Break down the different parts of an integration
Every integration
...
can be broken down into three parts - a Source, Processor and Destination:
1. Source
...
Two types of sources
Polling
File system (local, sftp, ftp, etc.)
Database
API endpoint
Listening (i.e. event triggered)
LLP feed
API requests
Filter (we should rebrand this to something better) - Processor?
...
Actual filtering
Mapping
...
Responsible for gathering data from the source system. There are two broad categories for how data can be gathered:
| Expand | ||
|---|---|---|
| ||
Polling interfaces are set to run at regular intervals and query the source for data to be processed. Data can be queried in various ways, such as:
|
| Expand | ||
|---|---|---|
| ||
Listening interfaces are event triggered and listen for data sent by a source system. For example:
|
2. Processor
Once data is received, there is often some kind of processing that needs to be done in order for the data to be compatible with intended downstream systems. Some examples can include:
Filtering unwanted messages.
Mapping from one field to another, or more generally, converting from one message type to another
To a lookup
Transforming data (e.g. data formatting, string manipulation
, removing special characters, code mapping, database lookup, etc.).
Transformation
Date formats.
3. Destination
With the data in the correct format, all that is left is to send the data downstream to the destination. This can be done in a variety of ways such as:
LLP listener
API endpoint
Database
Drop a fileInserting/Updating a database
Dropping a file with the data in a file system (local directory, SFTP, etc.)
Sending through TCP (i.e. LLP, HTTP, etc.)