January 16, 2013

OpenSSL, cURL, and multithreading.

OpenSSL is one of the most widely used crypto library. Supports many digest and cipher algorithms, many types of protocols, very light and extremely optimized for many platforms (take a look at the arm assembly or x86_64 assembly codes for different algorithms.. gives you a sense of the awesome work put into it by experienced people through the years), and yet, out of the box, it is not thread-safe.

Couple of points to note here:
- This does not mean that OpenSSL cannot be used in a thread-safe way. It can be, but a little work has to be put in.
- Related links: http://horstr.blogspot.in/2008/04/on-libcurl-openssl-and-thread-safety.html,
http://curl.haxx.se/libcurl/c/threaded-ssl.html, http://stackoverflow.com/questions/3281373/segmentation-fault-in-libcurl-multithreaded.
- cURL is a URL handler, like wget, and it fetches the contents of urls. For HTTPS connections, you can make curl use either OpenSSL, or GnuTLS, or other supported implementations of SSL. OpenSSL is default.
- When using curl in multi mode, you can create several easy handlers (like one for each url), and attach them  to a single multihandle, so as to give commands to the multi handle which it internally performs on each of the urls. Since OpenSSL is not threadsafe by default, this multi handle usage of curl can cause rare random crashes.
- To avoid all these, OpenSSL provides callback mechanism to set your own locking functions (like simple pthread mutexes, or your own mutexes/locks if you prefer). This will enable multiple parallel initializations and cleanups without causing a hitch.

Now the question arises, who sets these callback functions? Curl certainly doesn't set it, and shouldn't, in my opinion as it should remain light, portable, and shouldn't define its own constraints. The final app can always set its own callbacks. Imagine, then, if multiple apps are running as threads in a single process... like maybe in a browser/platform or something... If each of the apps set their own callbacks, there is the possibility that these locks are not mutually interoperable. So shouldn't the browser/platform set its own callbacks? Should they or shouldn't they be overrideable by the apps' callbacks? ugh. I've never liked mutexes and semaphores. too messy :(

1 comment:

Gururaj said...

Mutex and semaphores: I remember we had this OS course, where we touched upon them. Time to revisit emo.