Customizing the File Reader

This is good example how we can Eliminate bottlenecks with IguanaX. This is the problem. A large lab needs to have a file reading component which can feed files from different customers based on prefix of the file, i.e. some files start with JJ333, others XYZ etc. depending on the source.

How can we make a super streamlined component which:

Matches the requirements
Is easy to implement
Simplifies operations

Let’s do it!

Build a custom configuration

See Custom Fields. We take the base File Reader and add Client ID List which will be a comma delimited list of the prefixes we want to filter on:

We need to modify the matching algorithm - what do we start with?

The original file reader component is deliberately simple and uses /wiki/spaces/IXB/pages/3181903910 to make it more obvious how to modify the code to meet new needs. So this is what we are starting off with. We need this function to return true when the matching criteria are met.

https://bitbucket.org/interfaceware/fromfile/src/main/MatchRules.lua

--What file extensions do we match?
local FileExtensionMatchSet={
   txt=true,
   log=true,
   hl7=true
}

function MatchFile(FileName)
   --local JustFileName = FILfilename(FileName)
   local Extn = FILextension(FileName)
   Extn = Extn:lower()
   return FileExtensionMatchSet[Extn]
end

Start the new MatchFile logic with getting the list of prefixes and removing white space

We’ll apply /wiki/spaces/IXB/pages/3181903910 and write this as a function:

function APPidList()
   local IdList = component.fields()["Client Id List"]
   IdList = IdList:split(",")
   for i = 1, #IdList do
      IdList[i] = IdList[i]:trimWS()
   end
end

These are the concepts we have used:

Writing a function
Using APP as a prefix, see Using prefixes for naming code
Custom Fields
String:split()
Iterating through a Lua table as list
Trimming white space

Make a function which takes the ID and sees if the filename starts with it

So now we make a function to determine if the file name matches

function APPmatchPrefix(Prefix, FileName)
   local Part = FileName:sub(1, #Prefix)
   trace(Part, Prefix)
   return (Part == Prefix)
end

Concepts used:

Now put all the matching logic together

We write the new MatchFile function based on the functions we have written already:

function MatchFile(FileName)
   local IdList = APPidList()
   local JustFileName = FILfilename(FileName)
   for i = 1, #IdList do
      if APPmatchPrefix(IdList[i], JustFileName) then 
         return true;
      end
   end
end

Concepts used:

A function
FIL Library used to get the file name
For loop with Lua table as list using # Operator on Lua Tables
if statements

Commit your changes, alter the component to run on the new commit and run it!