The FTP Reader component is a simple example component that connects with a FTP, FTPS, or SFTP server and retrieves files from a target directory for processing. As each file is read, its contents will be pushed downstream for further processing, and the original file will be either deleted These files are downloaded to a local folder and then either deleted from the server or moved to a designated folder on the server. The downloaded files are then read and pushed downstream for further processing in the same way the https://interfaceware.atlassian.net/wiki/x/AQDqsw processes files. By default, this component polls every 60 seconds.
...
| Expand |
|---|
| title | STEP 1: Import the File Retriever component |
|---|
|
Using +COMPONENT, import the From FTP Readercomponent. |
| Expand |
|---|
| title | STEP 2: Set up the component configurations |
|---|
|
Fields marked with an asterisk are required  Image AddedConnection Configurations Protocol*: specify the protocol you’ll be using (FTP, FTPS, SFTP) Server*: the server you’ll be connecting with Port*: the port to use Username*: the remote user to login as. This account will need permissions to read, delete, and/or move files to the specified SourceDir FTPDirectory Certificate: the name of your certificate file, if in use. By default, this should be a PEM certificate. The default configurations can be changed by modifying the net.[protocol].init function directly in the script. This configuration is only applicable for FTP and FTPS. PrivateKey: the name of your private key file, if in use. By default, this should be a PEM certificate and not be password-locked. The default configurations can be changed by modifying the net.[protocol].init function directly in the script. Password: the password to authenticate with RetryCount: the number of times to retry the connection if it drops. By default, it will retry 10 3 times.
FTP Workflow Configurations SourceDirFTPDirectory*: the source directory on the FTP server you’ll be retrieving files from FileExtensionFileType: the types of files you wish to retrieve (e.g. txt)DeleteProcessedFiles. You can also specify multiple files (e.g. txt,hl7) or all file types (e.g. *). MoveFTPProcessedFiles: set to true if you wish to delete processed filesmove processed files (i.e. files that have already been downloaded to the local server) to a folder on the FTP server designated by FTPProcessedDirectory. If false, processed files will be deleted. This is set to true by default. FTPProcessedDirectory: the directory on the FTP server that processed files will be moved into. You will need to create this folder if you choose to use this workflow.
Local File Workflow Configurations LocalDirectory*: the local directory that the files will be downloaded to from the FTP server. Files in this directory will then be picked up for processing. MoveProcessedFiles: set to true if you wish to move processed files to a local folder designated by ProcessedFileDir ProcessedDirectory. If false, processed files will be deleted. This is set to false true by default. ProcessedFileDirProcessedDirectory: the local directory on the server that processed files will be moved tointo. You will need to create this folder if you choose to use this workflow. MinimumFileAge*: the time to wait in seconds after last modified date before processing. By default, this is 0 seconds. IgnoreSegments*: set to true to remove specified segments when processing file. By default, this is false. InputFileEncoding*: encoding of input files. If left blank, it will be updated to the default encoding.
|
| Expand |
|---|
| title | STEP 3: Start the component and view the processed files' contents in the logs |
|---|
|
Once the component has been started, it will pull download files that match the provided parameters to the specified local directory. It will then process the downloaded files and push their contents to the component’s queue for further processing. Open the component’s logs to see the queued files and verify that the contents are being read as expected. |
| Expand |
|---|
|
Encoding Name | Code |
|---|
Chinese (Big 5 HKSCS) | BIG5-HKSCS | Chinese (GB 18030) | GB18030 | Chinese (GB 2312) | GB2312 | Chinese (GBK) | GBK | Japanese (ISO-2022-JP) | ISO-2022-JP | Korean (ISO-2022-KR) | ISO-2022-KR | Western (ISO-8859-1) | ISO-8859-1 | Central European (ISO-8859-2) | ISO-8859-2 | South European (ISO-8859-3) | ISO-8859-3 | Central European (ISO-8859-4) | ISO-8859-4 | Cyrillic (ISO-8859-5) | ISO-8859-5 | Arabic (ISO-8859-6) | ISO-8859-6 | Greek (ISO-8859-7) | ISO-8859-7 | Hebrew (ISO-8859-8) | ISO-8859-8 | Turkish (ISO-8859-9) | ISO-8859-9 | Western (Windows-1252) | Windows-1252 | Chinese (Big 5) | big5 | Japanese (EUC-JP) | eucJP | Japanese (Shift JIS) | sjis | 8-Bit Unicode (UTF-8) | UTF-8 |
|
Adapting the Component
There are a few ways you can adapt and expand this component for more complex workflows:
...
| Expand |
|---|
| title | Customize the FTP/FTPS/SFTP connection |
|---|
|
By default, the FTP/FTPS/SFTP connection is authenticated using username and password authentication and uses the default configurations. In some cases, additional configurations may be desired, such as: Specifying a FTP(S) mode Using public and private key authentication Disabling verify_peer or verify_host Increasing the timeout and more. Full details can be found by accessing the net.ftp.init, net.ftps.init, or net.sftp.init function help in the Translator
These configurations can be customized by modifying the Input parameter table in the main functionloadConfigs() function in FTP/FTPprocess.lua:  Image Removed Image Added |
| Expand |
|---|
| title | Add more file retrieval conditions |
|---|
|
Currently, the component retrieves files from the FTP server based on two conditions: The file is retrievable The file extension matches the provided extension (e.g. txt)
This is typically sufficient for most cases, but if additional conditions are needed, they can be added to the MatchRules function as shown below:  Image RemovedFTPmatchRules function in FTP/FTPmatchRules.lua:  Image Added |
| Expand |
|---|
| title | Customize the ignored segments or message delimiter via config.lua |
|---|
|
Ignored segments: the ignored segments list can be modified to remove additional segments from a message when IgnoredSegments = true Replace message delimiter: replaces the delimited at the end of the message with ‘\n’
|