Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Break down the different parts of an integration

Every integration

...

can be broken down into three parts:

  1. Source

...

  • Two types of sources

    • Polling

      • File system (local, sftp, ftp, etc.)

      • Database

      • API endpoint

    • Listening (i.e. event triggered)

      • LLP feed

      • API requests

Filter (we should rebrand this to something better) - Processor?

...

Actual filtering

Mapping

...

  1. Processor

  2. Destination

Source

Responsible for gathering data from the source system. There are two broad categories for how data can be gathered:

  1. Polling interfaces are set to run at regular intervals and query the source for data to be processed. Some examples:

    1. Polling a source directory (local, SFTP, etc.) for files to be processed

    2. Querying a database for records to be processed

    3. Making API queries

  2. Listening interfaces are event triggered and listen for data sent by a source system. Some examples:

    1. LLP listeners receive data through LLP from an LLP sender

    2. Hosting an API endpoint that receives HTTP requests

Processor

Once data is received, there is often some kind of processing that needs to be done in order for the data to be compatible with downstream systems. Some examples can include:

  • Filtering unwanted messages

  • Mapping from one field to another, or more generally, converting from one message type to another

  • To a lookup

  • Transformation

    Date formats

    Transforming data (e.g. data formatting, string manipulation,

    removing special characters

    code mapping, database lookup, etc.)

Destination

...

LLP listener

...

API endpoint

...

Database

...

With the data in the correct format, all that is left is to send the data downstream to the destination. Some examples of how this can be done:

  • Sending through TCP (i.e. LLP, HTTP, etc.)

  • Inserting into a database

  • Dropping a file with the data in a file system (local, SFTP, etc.)