Disclaimer: this article was originally published by yours truly on Medium as part of my employment at Doctolib.

Calling into thread-unsafe DLLs with node-ffi

Well, that’s a mouthful… Anyway let’s start with some context. I am a French software developer working for Doctolib in our Berlin offices with a team of developers and product owners.

We build Zipper, a standalone program that stands between the Doctolib website in a browser and our partners’ software to bind everything together, all thanks to Native Messaging. These bridges help our users save time by removing the need for double entry of patient data, in Doctolib and in their own tools, with easy navigation between the two.

Sometimes we need to have Zipper hook into native libraries, some of which are proprietary. For this purpose node-ffi usually works fine, until we need to asynchronously call into thread-unsafe libraries. Node.JS being inherently concurrent, mixing these causes trouble.

Use case

We build Zipper using Webpack, then package it with pkg which bundles Javascript code and the V8 engine so as to produce a standalone executable. You can even use pkg to include assets in your binaries for easy distribution!

Since pkg runs with Node.JS we can do everything a Node.JS program can do, including loading and calling into DLLs (if you don’t know what this means, hang on; I will explain it later). Recently though, we needed to interact with some software by simulating user input so we turned to AutoIt, a scripting language designed to interact with Windows GUI elements, whose functions are also available as a DLL. It turns out this library is not thread-safe and that by using node-ffi naively we would get into trouble (crashes, mostly) by issuing concurrent calls. But before going any further let’s just have a refresher on what DLLs are and how they are used, especially with Node.JS.

Intro to node-ffi

Dynamic-link libraries (DLLs) contain code a running Windows program can load and execute. Some are provided by the operating system, some are provided by third parties and are installed either by the programs that need it or separately by the user. “Dynamic-link” means the libraries are loaded at runtime as opposed to being included directly into your executable, which has an interesting consequence: as long as the interface is the same, you can swap library versions and still have your program work fine with them without rebuilding it.

node-ffi is the de facto standard for loading and calling into DLLs (and their equivalent on other systems) from Node.JS. It provides you with an object whose functions represent functions from the library, which you can call synchronously or asynchronously. Let’s see an example.

toto.dll is a library that was provided to us by a third party, along with toto.h, a C header file which contains the definitions of the functions from the library.

/* toto.h */
int toto_foo(int, int);
void toto_bar(char*);

This simple library provides two functions:

toto_foo has two integer parameters and returns an integer.
toto_bar accepts a single pointer argument and returns nothing.

Using node-ffi we can load this library like this:

/* toto.js */
import { Library } from 'node-ffi'
import { refType } from 'ref'

const charPointer = ref.refType('char')
export default Library('toto.dll', {
  toto_foo: ['int', ['int', 'int']],
  toto_bar: ['void', [charPointer]],
})

And voilà! We now have a Javascript module which, when loaded, loads the library, locates the functions inside and exposes them as Javascript functions which can be called either synchronously (which is Bad) or asynchronously. Note that using the ref module node-ffi supports using pointers, simply pass the external function a Buffer and node-ffi will get the pointer to the Buffer’s data and pass it to the function.

/* index.js */
import toto from './toto'

// synchronous calls
const fooResult = toto.toto_foo(42, 413)
console.log(`synchronously got ${fooResult}`)

// asynchronous calls
toto.toto_foo.async(42, 413, (error, result) => {
  console.log(`asynchronously got ${result}`)
})

// using pointers
const buffer = Buffer.alloc(1337)
toto.toto_bar(buffer)
console.log(`synchronously used/modified ${buffer}`)

This works fine, until you find yourself with a thread-unsafe library.

What if the library is not thread safe?

Libraries have initialization code and deinitialization code, can allocate or deallocate memory, and have access to all the same memory as your main process. But most importantly, they can hold global state. Anyone who’s ever worked with concurrency most certainly knows that concurrency and global state cause much sadness and suffering when put together.

Oh and by the way, DLLs can contain unsafe code which can crash, like actually crash as in, the operating system kills your program. When this happens you do not get an exception or a rejected promise. Your Javascript program stops. So when you need to use a library which breaks when used with multiple threads, you run into trouble.

Let’s assume toto_foo is thread-unsafe. Maybe it uses some global state or does some I/O that is not properly synchronized. The following code will randomly crash or misbehave because Node.JS may have multiple threads calling into the library simultaneously, which the library does not expect.

/* index-broken.js */
import toto from './toto'

for (let i = 0; i < 5; i++) {
  toto.toto_foo.async(0, i, (error, result) => {
    console.log(result)
  })
}

Possible solutions

Using synchronous calls

The obvious solution in that case would be to use synchronous calls.

import toto from './toto'

for (let i = 0; i < 5; i++) {
  console.log(toto.toto_foo(0, i))
}

Note that this blocks a Javascript thread, which means that while the function is running, no other Javascript code can execute. The event loop itself is blocked. This might be fine, as long as you know that function will not block for long. However you can’t do this if your function does I/O, heavy computations, sleeps, etc. Also remember there is an overhead to calling functions over FFI.

Sadly for our purposes this could not work as we use many AutoIt functions which wait for specific events to happen, and would block our process from performing any of the other tasks it needs to perform at any time.

Serializing asynchronous calls

Asynchronicity does not prevent us from serializing all the calls to the library only with Javascript. We can fairly simply write a wrapper to the library which hides the synchronous functions and wraps the asynchronous functions to have their calls wait in a queue while a call is in progress.

import toto from './toto'
import { promisify } from 'util'

let queue = Promise.resolve()

async function enqueueCall(call, callback) {
  await queue
  try {
    const result = await call()
    try { callback(null, result) } catch {}
  } catch(error) {
    try { callback(error, null) } catch {}
  }
}

function wrapAsyncCall(functionName) {
  const wrappedFunction = promisify(toto[functionName])
  return (...args) => {
    const callback = args.pop()
    queue = queue.then(
      async () => {
        const result = await wrappedFunction(...args)
        try { callback(null, result) } catch {}
      },
      error => {
        try { callback(error, null) } catch {}
      }
    )
  }
}

const wrappedFunctions = {}
for (const functionName in toto) {
  wrappedFunctions[functionName] = wrapAsyncCall(functionName)
}

export default wrappedFunctions

Fixing the library

In the case of a free and open-source library, or a library you built yourself, you can of course fix the library to make it thread-safe. There is no way I can cover this subject in a single blog post, or even many. For each library adding support for multi-threading will be a different problem which requires solid knowledge of concurrent programming and of the internals of the library being modified, plus lots of time, especially for larger libraries.

Wrapping the library

This is the solution we eventually went for. We actually had cases where we needed to wait on some event using AutoIt, while simultaneously issuing other calls that would lead this event to happen. However, the DLL’s implementation of the waiting function was blocking. Node-ffi lets us run this blocking function asynchronously by running it in a separate thread.

However, if we serialize the calls, this will inevitably lead to a deadlock: if we simultaneously run

a call that waits for an event
a call that contributes to producing said event

and we serialize the calls, the second call will never happen and the first one will never return (unless it times out, which isn’t what we want either).

Because we do not have access to the sources of the AutoIt library we could not try and make it multithreaded, so we decided we would write a wrapper around the library which exposes an identical interface (making our wrapper a drop-in replacement for the real library). I will only give a high-level overview of this solution because it is quite a bit more complex than the previous ones I presented. If you are curious you can get the code to our wrapper on Github.

We were thinking: this library is a high-level wrapper for Windows system calls which are thread-safe, so the issue had to be in the library implementation, likely in the form of global state or the like. So we thought a possible solution would be to load the library multiple times, each time instantiating a duplicate of its internal state. And so as a proof-of-concept we built a wrapper with no internal state which for every call to the library would

Load the library (with LoadLibrary).
Get the function we’re calling.
Call it.
Unload the library (with FreeLibrary).

This did not work. It turns out calling LoadLibrary multiple times to load the same library always returns the same instance. The more flexible LoadLiraryEx does not have an option to override this either so we decided to trick Windows into believing we were loading a different library. Thus our second proof-of-concept attempt was still a stateless wrapper which would do this at each call.

Find the library.
Copy it to a temporary file.
Load the temporary file as a library.
Get the function we want to call.
Call it.
Unload the library.
Delete the temporary file.

It roughly looks like this:

#include <windows.h>

const LPWSTR dll_path = "./toto.dll"

int __stdcall toto_foo(int a, int b)
{
  WCHAR tmp_path[MAX_PATH + 1] = {0};
  GetTempFileNameW(L".", L"toto.tmp", 0, tmp_path);
  CopyFileW(dll_path, tmp_path, false);
  const HANDLE dll_handle = LoadLibraryW(tmp_path);
  int _stdcall (*fun)(int, int) =
    GetProcAddress(dll_handle, "toto_foo");
  int result = (*fun)(a, b);
  FreeLibrary(dll_handle);
  DeleteFileW(tmp_path);
  return result;
}

Of course this is getting ridiculously inefficient because it copies, loads, and deletes a file for each and every call to the library but it works! We could do many simultaneous calls and nothing broke (almost, more on that later). We later improved the performance by keeping instances of the library in a pool so that we don’t need to copy and load it for every call.

One thing that broke with this approach is that this library does indeed have internal state; it has functions which change the behaviour of other functions by changing said state. However, our wrapper does not yet have a feature allowing us to dispatch a sequence of function calls to the same library instance. This is something we will fix by adding some API to our wrapper that lets the Javascript program “reserve” an instance and call multiple functions on it with the guarantee that all these calls will be dispatched to the same instance.

Conclusion

While this strategy worked fine for our purposes, it is only a first working solution. It has allowed us to use AutoIt to interact with multiple GUI elements simultaneously, speeding up these interactions significantly! (One particular form used to take about a second and a half to fill, and is now complete in about 100 milliseconds.) There is much room for improvement: we could for example build a generic tool that would apply this technique to arbitrary libraries.

This was an interesting problem for us to solve as it shows that diving into the lower-level workings of Javascript and programs in general you can solve hard problems in creative ways.

October 10, 2019