The File Retriever FTP Reader component is a simple example component that connects with a FTP, FTPS, or SFTP server and retrieves files from a target directory for processing. As each file is read, its contents will be pushed downstream for further processing, and the original file will be either deleted or moved to a designated folder. By default, this component polls every 60 seconds.
Running the Component
| Info |
|---|
The below steps assume an existing FTP/FTPS/SFTP server is available to connect to and the necessary user permissions and folder structure are in place. |
| Expand |
|---|
| title | STEP 1: Import the File Retriever component |
|---|
|
Using +COMPONENT, import the File Retriever FTP Reader |
| Expand |
|---|
| title | STEP 2: Set up the component configurations |
|---|
|
Connection Configurations Protocol: specify the protocol you’ll be using (FTP, FTPS, SFTP) Server: the server you’ll be connecting with Port: the port to use Username: the remote user to login as. This account will need permissions to read, delete, and/or move files to the specified SourceDir Password: the password to authenticate with RetryCount: the number of times to retry the connection if it drops. By default, it will retry 10 times.
Workflow Configurations SourceDir: the source directory you’ll be retrieving files from FileExtension: the types of files you wish to retrieve (e.g. txt) DeleteProcessedFiles: set to true if you wish to delete processed files. If false, processed files will be moved to a folder designated by ProcessedFileDir. This is set to false by default. ProcessedFileDir: the directory on the server that files will be moved to. You will need to create this folder if you choose to use this workflow.
|
| Expand |
|---|
| title | STEP 3: Start the component and view the processed files' contents in the logs |
|---|
|
Once the component has been started, it will pull files that match the provided parameters and push their contents to the component’s queue for further processing. Open the component’s logs to see the queued files. |
Adapting the Component
There are a few ways you can adapt and expand this component for more complex workflows:
...