Practical ZeroMQ

The recommended approach to learning ZeroMQ is usually to read ‘The Guide’, which can be found here: https://zguide.zeromq.org/. It’s a resource well worth reading, and is by no means a quick introduction. My intent in this series of articles is to provide a complimentary approach to the usual guide, and discuss the use-cases and practical concerns of ZeroMQ that I have encountered in my usage.

Language Interop

I see ZeroMQ as ideal for ad-hoc interoperability between languages. There are library bindings for almost every language in common usage, and in some cases even native ZMTP (ZeroMQ Message Transport Protocol) implementations (ex. https://github.com/zeromq/zmq.rs in rust or https://github.com/zeromq/chumak in erlang). Importantly, the performance overhead of ZeroMQ is also small enough to make this an option.

Let’s look at a concrete example. Typically we might interface with a C function from Python via the C FFI, and call the function like normal Python code. Alternatively, we can start a separate process in C that accepts input over ZeroMQ, processes the input internally, and then returns the result over ZeroMQ. We’ll address the performance characteristics of this approach.

Here is our C function:

/* worker.c */
char* insertion_sort(char arr[], int n) {
  int i, key, j;
  for (i = 1; i < n; i++) {
    key = arr[i];
    j = i - 1;
    while (j >= 0 && arr[j] > key) {
      arr[j + 1] = arr[j];
      j = j - 1;
      arr[j + 1] = key;
    }
  }
  return arr;
}

And here is a very basic ZeroMQ ‘server’ around this function:

/* worker_server.c */
#include <zmq.h>
#include <czmq.h>
#include <unistd.h>
#include <assert.h>
#include "worker.h"
void recv_message(zsock_t *router) {
  while (1) {
    zmsg_t *req = zmsg_recv(router);
    zframe_t *address = zmsg_pop(req);
    zframe_t *arr_frame = zmsg_pop(req);
    zframe_t *n_frame = zmsg_pop(req);
    char *arr = zframe_data(arr_frame);
    int n = atoi(zframe_data(n_frame));
    char *sorted = insertion_sort(arr, n);
    zframe_t *res_data = zframe_from(sorted);
    zmsg_t *res = zmsg_new();
    zmsg_add(res, address);
    zmsg_add(res, res_data);
    zmsg_send(&res, router);
  }
}
int main(void) {
  void *ctx = zmq_ctx_new();
  void *router = zmq_socket(ctx, ZMQ_ROUTER);
  int rc = zmq_bind(router, "tcp://127.0.0.1:8888");
  assert (rc != -1);
  printf("server started on port 8888\n");
  recv_message(router);
  return 0;
}

And finally, two methods in Python to interface with each:

def test_ffi(test_str: bytes, test_size: int):
    insertion_sort(test_str, test_size).  # call directly
    return
def test_zmq(dealer: zmq.Socket, test_str: bytes, test_size: bytes):
    dealer.send_multipart([test_str, test_size]).  # call over ZMQ
    res = dealer.recv_multipart()
    return

I tested each Python method over the same input of various sizes. To understand the effect of input size on performance, consider that input size can be seen as a mechanism to control how much of the overall execution time is spent inside the C function. However, input size also adds overhead in the form of more bytes on the wire. With that in mind, we’ll plot the percent difference in execution time of ZeroMQ to FFI against the input size.

We see execution time using ZeroMQ stabilize at slightly less than double that of using FFI. This is reached around a test size of 200, where each call of the insertion_sort function is taking approximately 0.2 milliseconds.

In conclusion — ZeroMQ adds approximately double the runtime overhead of using FFI, assuming the foreign function has a runtime of at least 0.2 milliseconds.

Can you live with that? Generally speaking, we resort to using the C FFI when we need the performance boost it gives us, and it probably remains the best option for doing so from Python. That said, the motivation behind interfacing between two languages is not always performance. ZeroMQ can be used to glue together code in different languages without a terrible amount of infrastructure, and as demonstrated, modest performance overhead.

Considerations:

• In these tests, I performed no data serialization, and serialization of course entails further overhead.