libight: a new hope

This blog post is part of a series of three regarding ooniprobe-mobile.

On May 28th 2014, Arturo Filastò and I were sitting in a pub in Turin drinking Weißbier. He had been invited to present his Open Observatory of Network Interference (OONI) project at the 21st Nexa Center's lunch seminar, and he also gave a talk in our Rivoluzione Digitale course.

isle of skye
The pub in which we were, The Isle of Skye Pub, probably one of the best in Turin.

Two months earlier, I completed my PhD at Politecnico di Torino, and when I was discussing with Arturo, sadly my daily job was more focused on finalizing my doctoral thesis than on writing software.

I have to admit that the months during which I was editing my doctoral thesis were amongst the least pleasant months of my life, because I really do not like dealing with such large bloats of text (unless it's code, ça va sans dire).

Yet, as all the kids born in Liguria ought to know, from diamonds it stems nothing, from manure bloom flowers: while writing the doctoral thesis I started cheating and writing not about the Neubot that had been implemented, but about the one I'd have liked to see. And this was the beginning of libneubot.


Fabrizio De André, the late Italian singer and poet I cited above, is really one of my favorite ones.
Here's his cover of Bob Dylan's Desolation Row.

Libneubot? Well, even earlier than writing the thesis, in Autumn '13, the importance of working on a mobile version of Neubot was clear to me — and we also did receive specific funding, later in '15 used to cover my salary.

And, to address this mobile need, the idea I had was indeed libneubot. A C library meant to expose specific functionalities to Neubot, written in Python, and, in the future, to an Android app, written in Java.

Now, notwithstanding what some academic types will tell you, in the real world (TM), it does not suffice to have a neat idea to thrive, as Edison famously codified. Luckily, at the time I was assisted by a very brilliant student, Alessandro Quaranta, and together we were adding the missing 99% of transpiration to reach a workable product.

And, in that nice spring evening of '14, libneubot was indeed one of the topics I was discussing with Arturo, the other topic being MORFEO.

tesla and edison
There are lots of known facts (and also some alt-facts) surrounding the different approach of Tesla and Edison. I don't feel qualified enough to enter into the debate, but, for what I have understood, I'd say I'm more, with all due respect, an Edison-like type that transpires a lot than a Tesla-like type that relies on theory.

MORFEO? Yes, MORFEO. Or, Morpheus, in English (even though it would be more fair to say that that is ancient Greek). Anyhow, in our case, the MORFEO we were talking about was not the ancient god of dreams, but rather a subproject of EINS for which we were awarded funding.

EINS. You seem confused at this point. But don't panic. EINS was a European project that the Nexa Center was participating in. Sometime around the spring of '13, Federico Morando and Raimondo Iemma (my then fellow researchers at Nexa Center) suggested that I write a proposal for an upcoming open call within the EINS project. They said it would've been nice to partner with someone else, and I proposed Arturo, who I had met in Autumn '11 during the EU hackathon, and with whom I subsequently had a number of technical discussions centered around our own projects, respectively Neubot and OONI, which shared several similar design traits and challenges, even though they were focusing on different domains (performance and censorship). Anyhow, to cut it short, MORFEO proposed to add censorship measurements (assisted by Arturo) to Neubot, but this was not awarded for a long time, to the point that I forgot it existed, and was finally surprisingly awarded in spring '14.

fate presto
Lest we forget, the days of the EU hackathon were also very dramatic days for my country amidst the economic crisis. Berlusconi was stepping down and I recall spending the night hacking and constantly refreshing the browser for more news. This is the front page of a prominent Italian news paper, il Sole 24 ore, the day after the EU hackathon. Historians will tell us if the involved parties were acting in the national interest. Despite many hours reading and reflecting on the subject, I still feel like the whole matter was very complicated.

So, now we've reached the bottom of it, you have perfect information, and we can flash forward to that May '14 evening, where we were in fact discussing about what to actually do inside of MORFEO. And, it turned out that we were both interested in implementing measurements on mobile, but we were not able to do that so far. So, we were like: wait a minute, why don't we do this within the context of MORFEO?

The idea was promising, especially because it was clear to both of us that the effort of porting our measurement methodologies to mobile was complex enough that a partnership between two different teams was in order to guarantee increased chances of success.

Thus, we started discussing the pros and cons, and at a certain point I threw libneubot at the discussion, saying something like: well, a student and I have already a prototype, if you wish we can start looking into it, to move from discussion to real feasibility assessment.


Using libneubot was pretty much like realizing that the synthesizer was the sound of the future. When the discussion started, I did not really planned on proposing using libneubot, but it quickly become very obvious.

The legend says that we reached a point where we were either ordering another whisky or heading back to my home to hack.

We chose the latter. After a brief discussion, it was clear to both of us that it was not going to be fair if the common library to port our projects on mobile platforms was called with the name of either project, so we settled to renaming libneubot to libight. And if you go back in the history of the repository, you can clearly find the moment in which we implemented this decision.

Another decision that we took almost immediately was to reorganize the codebase so as to make it more similar to the one of Tor project under the argument that, well, libneubot was a mess with tons of files in the same directory and that was clearly not easy to understand for persons other than me and my student Alessandro Quaranta.

Other changes that were implemented pronto were, for example, adding support for GNU build tools and removing support for automatic mapping of Python APIs to C APIs. Additionally, at a certain point Arturo (who was browsing at a blazing fast speed for inspiration regarding how to structure the project) suggested to modify the project to use C++11 rather than plain C. I, who do not like object oriented programming and prefer much more functional programming, resisted for a while, but then I learned about the availability of lambdas in C+11, and so it was decided. Libight was to be written in C++11.

cxx
When we started talking about C++, that's what I feared. My previous experience was with C++98, with which I was not comfortable. Now, after having used C++11 for many years, I have to admit I enjoy it.

That night we followed suit with many more commits, discussing the general design, finding consensus, and sketching out the skeleton of what is now known as measurement-kit, the engine of the recently released ooniprobe-mobile and hopefully of many more pieces of software in the future. Many things happened next, but I don’t think it makes sense to recount them as part of this post, which is already quite long. This is it, for now. To be continued.

Thanks to Maria who helped with copy-editing.