I program my home computer, beam myself into the future

At some point, the Linux kernel decided that the maximum CPU usage (derived from “/proc/<PID>/stat”) for my FX-8320E CPU should be 400% instead of 800%. Poppycock, I say! This Piledriver beauty has 8 (integer) cores and Conky divides a process’ CPU usage to the number of cores, ending up with a maximum value of 50% instead of 100%.

So it should be as simple as doubling the “CPU%” value for each ${top cpu <#>} in ~/.conkyrc, right? Unfortunately, Conky’s configuration file lacks support for even the simplest math operation. The only way I could double those value was with a Lua script:

function conky_double(...)
        return string.format('%6.2f', 2 * tonumber(conky_parse('${' .. table.concat({...}, ' ') .. '}')));
end

This Lua function is called “double” inside the config file and it’s used to replace something like “${top cpu 1}” with “${lua double top cpu 1}”. So the function takes an arbitrary number of parameters  (passed as strings), insert spaces between them, places the result between “${” and “}”, gets conky to parse this as if it were a regular config command, converts the resulting string in a number, doubles it, formats it and returns it.

Now all that’s left to do is tell Conky to load this Lua script, by adding this inside conky.config:

lua_load = '/path/to/conky.lua'

and, of course, put “lua double” in front of those “top cpu” calls so processes can be shown with the right CPU usage percentage.

scan all the things!

w_scan is a very useful tool, because it can scan DVB-C/T/S and ATSC channels without any prior knowledge of frequencies. It’s been a while since I last used it, but now I got a new TV tuner card and did a new scan with it, only to discover that the Rai channels were no longer detected.

The first problem was that the last version of w_scan discards duplicate transponders, and the one it keeps is the first one it finds. In my area, I get a signal from multiple antennas, and the one with the strongest signal is at a higher frequency so it gets discarded as a duplicate.

Once I fixed this problem by disabling check_duplicate_transponders() in the source code, I found another one: the second transponder, the one I wanted scanned for channels, was advertising a third one, with a shitty signal strength. w_scan took this advertisement and completely replaced the second transponder with the third one.

So I fixed that too, and put it all under an optional flag (-K, —keep-duplicate-transponders) that finally scans everything I want to scan. The patch is here and if you’re a Gentoo user you can install the patched package with “layman -a stefantalpalaru; emerge w_scan”.

Now “w_scan -ft -c IT -E0 -O0 -t2 -K -L > channels.xspf” gives me the perfect input file for “vlc –dvb-budget-mode channels.xspf” (the “–dvb-budget-mode” parameter is needed by Mediaset channels because they fucked up something in the implementation).

 

dynamic initial data for Django model formsets

Here’s how you can pass initial values to a model formset (mfs) after the object was instantiated:

mfs.initial_extra = [
    {
        'foo': 1,
        'bar': 'a',
    },
    {
        'foo': 2,
        'bar': 'b',
    },
]
mfs.extra += len(mfs.initial_extra)
mfs.forms = [mfs._construct_form(i) for i in xrange(mfs.total_form_count())] # 'forms' is a cached property

Tested with Django-1.7 but should work with older versions. For regular formsets replace ‘initial_extra’ with ‘initial’.

uWSGI and graceful reloading

The holy grail of web application deployment is restarting without dropping ongoing requests and without any downtime. This is called graceful restart or graceful reload and one of the easy ways to achieve it is to run multiple processes and have some of them stop accepting new requests and restart themselves when they finish serving the existing ones. In the mean time the remaining processes run as usual. When the first batch completes the restarts the rest can undergo the same procedure.

uWSGI

uWSGI is slowly becoming the de facto application server in the Python world and it shows great promise with many more languages. The Art of Graceful Reloading from the official documentation describes various strategies. The one I’m focusing on is described in the Subscription system section.

Long story short, the application processes run as vassals controlled by an emperor and subscribe to a fastrouter. A web server like Nginx uses the fastrouter as a backend. During the graceful reload the vassals unsubscribe from the fastrouter in order not to receive new requests and subscribe again after restarting.

Triggering a vassal’s graceful reload is as simple as touching its configuration file. Telling when a certain vassal has finished reloading is more complicated.

uwsgi_reload

Here’s where my new project comes in. uwsgi_reload boils down to a script and example configuration files detailing the deployment of a Django project with uWSGI and Nginx. The script does not communicate with the application so it should work with most (all?) the frameworks/languages supported by uWSGI.

This is what the script does:

– it makes sure only one instance is running at any given time – with a simple and elegant file lock on its own file.

– it divides the vassals into two groups – the first is reloaded in parallel to speed things up and then the second (containing by default only one vassal) is reloaded sequentially.

– when the script exits all the vassals were restarted or the timeout was reached

These guarantees make it ideal for a deployment pipeline. If multiple developers trigger it, only one gets to execute it. It’s as fast as possible because of the parallelization and it waits for the end of the operations so you know for sure when the new version of the site is up and running.

under the hood

With all its features and configurability uWSGI is not perfect. It suffers from high complexity and documentation not always up to date. To do something as simple as finding out when a vassal is ready to accept new requests I had to look in the emperor stats for the last modification timestamp and the ‘accepting’ flag. The subscription and ‘death_mark’ status come from the fastrouter statistics. Most of the development time was spent trying to get this reliable information about a vassal’s status.

Convincing the vassals to only subscribe when the application server is ready to accept requests took some digging in mailing lists for the obscure configuration lines. Convincing Django to give up its lazy loading ways and warm up before entering the field was another story. Save yourself the trouble and use the files in the examples directory.

migrating a Cherokee configuration to Nginx

I’m releasing today cherokee2nginx – a script that converts your old cherokee.conf to a new and shiny nginx.conf. Well, not that shiny since the conversion is incomplete and this initial version only covers the parts I needed for a single server, but it sure beats having to do everything by hand.

From the parts that need manual editing after the conversion, the ɯɯɯ.domain.tld to domain.tld redirect is worth mentioning because the proper way to do it for Nginx looks like this:

server {
    listen 80;
    server_name www.domain.tld;
    return 301 http://domain.tld$request_uri;
}

from stick figure rights to human rights

Randall Munroe reiterates an often invoked defense of censorship in his last xkcd comic: freedom of speech only applies to interactions with the government. By this logic, non-governmental entities are free to censor any sort of speech they don’t like. In order to understand why this approach is corrupting a basic human right, we need to go back to the beginning.

In 1689 England’s Bill of Rights stated that “the freedom of speech and debates or proceedings in Parliament ought not to be impeached or questioned in any court or place out of Parliament”. This was obviously limited to members of Parliament and to the proceedings of that institution.

In 1789 the French Revolution brought the famous Déclaration des droits de l’homme et du citoyen which stated: “The free communication of ideas and opinions is one of the most precious of the rights of man. Every citizen may, accordingly, speak, write, and print with freedom, but shall be responsible for such abuses of this freedom as shall be defined by law.”. With this, freedom of speech is no longer a political necessity but a fundamental human right bestowed upon all citizens all the time.

In 1948 the French notion of human rights was adopted by most of the planet through the Universal Declaration of Human Rights. Here’s the relevant section: “Everyone has the right to freedom of opinion and expression; this right includes freedom to hold opinions without interference and to seek, receive and impart information and ideas through any media and regardless of frontiers.”. Pretty straightforward, right? But free speech is a powerful weapon and outside the legal framework of limits and punishments, it’s easy to be bothered by people saying absurd things and then claiming it’s their right to do so.

Guess what? It really is their right to do so. The dark side is very tempting with its justifiable censorship that surely won’t affect us sensible people, but freedom is much more important than comfort. So important, in fact, that we should defend the freedom of expression of people “shown the door” by a majority that labeled them “assholes”. Even if we agree with the labeling. There are many acceptable ways of dealing with speech we don’t agree with. Censorship is not one of them.