Understanding search paths

Owned by Eliot Muir
Sept 24, 2021
2 min read

How can a computer find resources like executables, dynamic libraries, header files, static libraries in the system of directories on disc? A common solution to this question is to define a “search path” for these resources so that when we ask the computer or compiler program to find these resources that it knows to search through that list of directories to find the resource asked for.

Let’s give a few examples to illustrate this concept.

Say I am using the command line in Windows and I want to invoke the “GIT” source code executable. Then the GIT command program needs to be in a search path which for windows is the “PATH” environmental variable. We would use the phrase make sure that “git is on the path” in the machine.

Say we need to access an oracle database driver on Mac OS X. The oracle database driver is shared object and it needs to be in a directory listed with the LD_LIBRARY_PATHvariable. i.e. the OCI driver needs to be in the dynamic linked library path. See (https://developer.apple.com/library/archive/documentation/DeveloperTools/Conceptual/DynamicLibraries/100-Articles/UsingDynamicLibraries.html ).

For C++ compilation we need to tell the compiler where to find header files so that when it sees something like #include <COL/COLstring.h> that it knows how to find COLstring.h in the COL directory. For this problem we use a trick which I call search path namespacing. This is where we add the parent directory where all the header files can be found under by using the expression -I../ which says add the expression for going up one level in the directory tree, now we can look for header files using the relative path COL/COLstring.h which resolves to ../COL/COLstring.h which in absolute terms is probably somewhere in your home directory.

Search path namespacing is a general purpose solution which could be applied to solving many search path problems.

For instance for linking at against third party libraries we could follow a convention of putting all the third party libraries into a common directory or under a common directory. For instance ~/ThirdParty/Libs - we can add ~/ThirdParty/Libs to the linker library search path using -L~/ThirdParty/Libs and then link to each library using -lzlib/zlib.