Versions Compared

Version Old Version 1 New Version 2
Changes made by

Aryn Wiebe

Aryn Wiebe

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

For workflows that require handling multiple simultaneous tasks, such as a multi-threaded web server managing concurrent connections, parallel processing is a powerful technique to improve efficiency and scalability. This approach is only suitable for when the order in which messages are processed is not important.

In IguanaX, parallel processing within components is achieved using the component.clone method to create multiple Lua instances (or threads) for processing. Messages can then be distributed across a pool of these threads with component.call, which invokes the main() function of the processing component and passes the data to be processed. Each thread processes messages independently, allowing for faster performance in high-throughput scenarios.

Let’s take a look at a simple example:

...

titleSTEP 1: Add two custom components to your Iguana, name one "Dispatcher" and the other "Worker"
Expand
titleSTEP 3: Worker

Typically, the component.clone call would be a part of the initialization logic so that when the component is turned on, multiple Lua threads are spawned. After this, whenever the component receives a message, the first available Lua thread will take it and process it according to the script.

Info

component.clone has a maximum thread count of 100. You must provide the count parameter a number between 2 and 100.

Code Block
languagelua
function processMessage(Data)
   -- Data comes in as a json string, parse to get the parameters
   local ParsedData = json.parse{data=Data}
   local TaskId = ParsedData.id
   local Payload = ParsedData.payload
   -- process data 
end

function main(Data)
   if Data == "INIT" then
      -- Make 10 threads
      component.clone{count=10}
      return
   end
   processMessage(Data)
end
Expand
titleSTEP 3: Dispatcher

In the dispatcher, the data is prepared and component.call is used to call the main function within the Worker component.

For component.call to work, the target component must be running.

Code Block
languagelua
function main(Data)
   if Data == "INIT" then
      return
   end
   
   -- Create sample JSON to dispatch
   local CallData = {}
   CallData.id = 123
   CallData.payload = Data
   CallData = json.serialize{data=CallData}
   
   -- This will call the main function of Worker and immediately return
   component.call{name="Worker", data=CallData, func="main", async=true}
end

link Call a function within another Component

Multithreaded web service For an example use case, refer to the Web Service Multithreaded component.