Hot Code Reloading in C++

Why do we need Hot Code Reloading in the first place?

Well, to say shortly, for certain applications, critical sections of code need to be often rerun to check for errors and perform rolling fixes without waiting. However it becomes quite difficult if your "giant" application needs to be recompiled repeatedly for each iteration of fix taking precious amounts of time from your project hours.
Also, certain applications like "Game Engines" require features like dynamic loading to apply some new logic without restarting the entire game engine each time the code is compiled somewhat how "Unity" or "Godot" (and maybe Unreal, I didn't look into their system in details) provides dynamic code reloading upon saving changes and change the logic on the fly without changing game state or restarting the engine entirely.

How Hot Code Reloading works?

If this theory is boring then Skip straight to code

Shared Libraries

Shared Libraries are linked binaries, that do not contain their own starting point and can be loaded by other programs in a shared memory space and get pointers to symbols in the executable to execute it's code.
Sounds complicated? Well in simple words it's an executable with no main and can be loaded by the host system to be shared among multiple programs and execute the same small binary for multiple programs, that's where the name Shared Library comes from.
Different host systems have different ways to handle such libraries and have different names for it:

On Linux: Shared libraries -- Example libshared.so
On Windows: Dynamic libraries -- Example libshared.dll
On Mac OS and friends: Shared libraries or Dynamically loaded modules (there's a difference between them but support linux libs too) -- Example: libshared.dylib or libshared.so

In the following topic we will mostly discuss the Linux way of handling shared libraries and build a hot code reloader for linux.

How Linux finds Shared libraries

First after compilation either the program requests for the library with dlopen call or the program itself is linked with the library, for the host system to look for and then the host system finds the library in the following order:

If a slash is found in the name then it is treated as relative or absolute path and the shared object is loaded using that pathname

If no slash is found:
- Using the directories specified in the DT_RPATH dynamic section attribute of the binary if present and DT_RUNPATH attribute does not exist.
- Using the environment variable LD_LIBRARY_PATH, unless the executable is being run in secure-execution mode (see below), in which case this variable is ignored.
- Using the directories specified in the DT_RUNPATH dynamic section attribute of the binary if present. Such directories are searched only to find those objects required by DT_NEEDED (direct dependencies) entries and do not apply to those objects' children, which must themselves have their own DT_RUNPATH entries. This is unlike DT_RPATH, which is applied to searches for all children in the dependency tree.
- From the cache file /etc/ld.so.cache, which contains a compiled list of candidate shared objects previously found in the augmented library path. If, however, the binary was linked with the -z nodefaultlib linker option, shared objects in the default paths are skipped. Shared objects installed in hardware capability directories (see below) are preferred to other shared objects.
- In the default path /lib, and then /usr/lib. (On some 64-bit architectures, the default paths for 64-bit shared objects are /lib64, and then /usr/lib64.) If the binary was linked with the -z nodefaultlib linker option, this step is skipped.

How linux loads the shared library to memory

After locating the library with the above specifications, it simply loads the library in a dynamic location somewhere in RAM and marks it's symbols available in the system.
Now calling dlsym fetches the pointer to these dynamic symbols and passes them to the requesting program or for pre linked programs, the elf binary is already bootstrapped with the elf interpreter to get these symbols during runtime.

Shared Library working — An image representation of dynamic libraries in memory

Hot Code Reloading in C++

For windows

The analog for dlopen, dlsym and dlclose are LoadLibrary, GetProcAddress and CloseLibrary.
You can use them or use my own C++ header only wrapper HotCodeReload to make your life easier and load libraries effiently to your code. Here's a link to HotCodeReload Github

dlopen, dlsym, dlclose

The functions are present in the <dlfcn.h> header

dlopen -- Loads a shared library to a void * to the library
Synopsis: void *dlopen(const char *filename, int flags);
For more details dlopen man7

dlsym -- Gets the address of a symbol in shared library
Synopsis: void *dlsym(void *restrict handle, const char *restrict symbol);
For more details dlsym man7

dlclose -- Closes the symbol handle(void *)
Synopsis: int dlclose(void *handle);
For more details dlclose man7

Implementation

Source of shared library we want to load

./libshared.so


#ifdef __cplusplus // force C style naming convention for CPP to find for the symbols
extern "C"         // in dlsym calls, CPP mangles symbol names to support OOP
{
#endif

int sq(int x)
{
    return x * x;
}

#ifdef __cplusplus
}
#endif

Loading the library:


#include <dlfcn.h>
// ...
int main()
{
    void *lib = dlopen("./libshared.so", RTLD_LAZY); // or RTLD_NOW to load symbols immediately
    if(lib == nullptr)
    {
        std::cerr << "Failed to load library!\n";
        return 1;
    }
}

Calling dlopen with RTLD_NOW will force dlopen to resolve all symbols before returning, failure to do so will return NULL, also if the library is not found then NULL is returned.

Getting the symbols


typedef int (*fun_t)(int);
fun_t sqFunc;
*(&sqFunc) = (fun_t)dlsym(lib, "sq"); // use this weird dereference of address syntax to make the compiler
                                 // cast void (*) to int (*)(int) without complaints
if(sqFunc == nullptr) // always check for errors
{
    std::cerr << "Failed to get symbol sq\n";
    dlclose(lib);
    return 1;
}

// Now you can normally call the function
int sqOfFive = sqFunc(5);
std::cout << "Square of 5 is: " << sqOfFive << '\n';

The dlsym function returns a void (*) pointer to the symbol requested, here we ask the compiler to cast it to a int (*)(int) to call the function

Closing the library


dlclose(lib);
lib = nullptr; // its always a good practise to mark freed stuff nullptr

Simply make sure to call dlclose at the end to close the library and mark the free for use from your program.

Complete dynamic loading example


#include <dlfcn.h>
// ...

int main()
{
    const char *libName = "./libshared.so";
    void *lib = dlopen(libName, RTLD_LAZY);

    if(lib == nullptr)
    {
        // log error
        return 1;
    }

    typedef int (*fun_t)(int);
    fun_t sqFunc;
    *(&sqFunc) = (fun_t)dlsym(lib, "sq");

    if(sqFunc == nullptr)
    {
        // log error
        dlclose(lib);
        return 1;
    }

    char ch = 0;
    do
    {
        std::cout << "Enter choice: ";
        std::cin >> ch;

        if(ch == 'r') // reload the library
        {
            if(lib != nullptr) dlclose(lib);
            lib = dlopen(libName, RTLD_LAZY);
            if(lib == nullptr)
            {
                // log error
                return 1;
            }

            *(&sqFunc) = (fun_t)dlsym(lib, "sq");
            if(sqFunc == nullptr)
            {
                // log error
                dlclose(lib);
                return 1;
            }
            std::cout << "Reloaded!\n";
            continue;
        }

        std::cout << "Output: " << sqFunc(5) << '\n';
    } while(ch != 'q');

    dlclose(lib);
    return 0;
}

Now for hot reloading, run the program, and then press any key except r or q to view the output of sq function for 5.
In order to change code on fly, recompile libshared.so without stopping the main program and press r to reload the library in main program
Now pressing any other key will yield in a different result with changed code!
To quit the main program press q to bail out of the loop and return

Example output

g++ main.cpp -o main.exe
g++ shared.cpp -shared -fPIC -o libshared.so
./main
Enter choice: s
Output: 25

Now, change the code of shared.cpp like as:


#ifdef __cplusplus // force C style naming convention for CPP to find for the symbols
extern "C"         // in dlsym calls, CPP mangles symbol names to support OOP
{
#endif

int sq(int x)
{
    return x + x; // <<<=== now we are adding each other instead!
}

#ifdef __cplusplus
}
#endif

Enter choice: r
Reloaded!
Enter choice: s
Output: 10
Enter choice: q

Now as you can see, the code has been updated without stopping main.

Preserving states

State can be preserved on the main.cpp side by storing the data in a struct or std::map as the data in main.exe side will remain throught the execution of the program.

Further learning sources:

TSoding hot code reloading in C -- youtube.com

Slembcke's Computational Corner -- slembcke.github.io

Google.com -- google.com
Yes I am serious!