How to strip out non-ASCII characters
- Richard Wang
- meer.ushter
This document discuss about:
We have seen that customers want to use Iguana to parse data that includes non-ASCII characters (ex. XML)
Due to the inclusion of the non-ASCII characters, the translator is not able to process those messages, thus resulting in Iguana channels stopping and producing error messages in the Iguana logs.
Design
Exclude all non-ASCII characters using lua string.gsub()
To learn more about string.gsub() see our documentation here.
How to
Using the above Lua function, we look to replace the non-ASCII character with nothing (““ string)
First, we need to call the Lua function and place our first parameter, which in this case would be the Data Variable, that consists of our message in the To Translator.
string.gsub(Data,Secondly, we need to define the ASCII Table to compare our string, so we use
"[^%z\1-\127]"to compare all of the character from the ASCII Table starting from Dec 0’s Char to Dec 127 Char. Reference to the ASCII TABLE.
Our code will then look like thisstring.gsub(Data, "[^%z\1-\127]",Finally, we must provide a character to replace the non-ASCII character in this case we can use (““) as we want to replace it with “nothing”, from which you should get the following as a result:
string.gsub(Data, "[^%z\1-\127]", "")Now, for our final structure in order to retrieve the Data, we need to remove the non ASCII characters and then assign the modified data to a variable for further use, our code could then resemble the following: