Using an array to concatenate a long string

So in languages like Lua and Javascript strings are immutable. This can really play havoc with how malloc works which is used to allocate memory on the heap if you are concatenating many small strings into a much much larger string.

With each append, the operating system has to deallocate the last large string and allocate a new one. Particularly bad since these languages use garbage collection rather than allowing memory to be deallocated explicitly.

So this harmless looking code can be a performance killer for the entire operating system if the LargeArray array variable is very large:

function EXAMPLEfunc(LargeArray){
  var Result = '';
  for (var i =0; i < LargeArray.length-1; i++){
    Result += LargeArray[i];
  }
  return Result;
}

This is not clear why unless you understand how malloc works. But this approach is much much faster when dealing with a large array.

function EXAMPLEfunc(Foo, Array){
  for (var i =0; i < Foo.length-1; i++){
    Array.push(Foo[i]);
  }
  return Array.join('');
}

This is because there is no string allocation with line 3 - we’re just building a list. In line 5 the join command is able to iterate through all the strings and add their lengths together and allocate one large block of memory with only one call to malloc under the hood.

For a good explanation of this same phenomena in the context of Lua read:

https://www.lua.org/pil/11.6.html

Reminds me of the painter example

Back to Basics

Concepts

Using an array to concatenate a long string

Related pages