2011-03-05 Monitoring Memory of Suspicious Processes

Monitoring The Memory of Suspicious Processes

If you are operating many GNU/Linux boxes, it's not uncommon to have issues with some processes leaking memory. It's often the case for long-running processes handling large amount of data and usually using small chunk of memory segment while not freeing them back to the operating system. If you played with the Python "gc.garbage" or abused the Perl Scalar::Util::weaken function but to reach that stage, you need to know which processes ate the memory.

Usually looking for processes eating the memory, you need to have a look at the running process using ps, sar, top, htop… For a first look without installing any additional software, you can use ps with its sorting functionality:

%ps -eawwo size,pid,user,command --sort -size | head -20

 SIZE   PID USER     COMMAND
224348 32265 www-data /usr/sbin/apache2 -k start
224340 32264 www-data /usr/sbin/apache2 -k start
162444  944 syslog   rsyslogd -c4
106000 2229 datas     redis-server /etc/redis/redis.conf
56724 31034 datap    perl ../../pdns/parse.pl
32660  3378 adulau   perl pdns-web.pl daemon --reload
27040  4400 adulau   SCREEN
20296 20052 unbound  /usr/sbin/unbound
...

It's nice to have a sorted list by size but usually the common questions are:

• Is that normal?
• What's the evolution over time?
• Does the value increased or reduced over time?
• Which memory usage is evolving badly?

My first guess was to get the values above in a file, add a timestamp in front and make a simple awk script to display the evolution and graph it. But before jumping into it, I checked in Munin if there is a default plugin to do that per process. But there is no default plugin… I found one called multimemory that basically doing that per process. To configure it, you just need to add it as plugin with the processes you want to monitor.

[multimemory]
env.os linux 
env.names apache2 perl unbound rsyslogd

If you want to test the plugin, you can use:

%munin-run multimemory
perl.value 104148992
unbound.value 19943424
rsyslogd.value 162444
apache2.value 550055

You can connect to your Munin web page and you'll see the evolution for each monitored process name. After that's just a matter of digging into "valgrind --leak-check=full" or use your favorite profiling tool for Perl, Ruby or Python.

Tags: unix command-line memory monitoring

2011-03-06 Why The Philosophical Works Should Be Free

Roberto Di Cosmo recently published a work called "Manifeste pour une Création Artistique Libre", the work is not really a manifesto in the traditional sense but more a work about the potential licensing scheme at the Internet age. My blog entry is not about the content of the work itself but more about the non-free license used by the author. On the linuxfr.org website many people (including myself) made comments about how strange is to publish a work about free works while the manifesto itself is not free (licensed under the restrictive CC-BY-NC-ND). The author replies to the questions explaining his rationals to choose the non-free license with an additional "non printing" clause to the CC-BY-NC-ND.

I have a profound respect to Roberto's works regarding the promotion and support to the free software community but I clearly disagree with the facts stating philosophical works must not have any derivative and cannot be a free work. I also know that Richard Stallman disallows derivative work on his various works. If you carefully check the history of philosophical works, there are a lot of essays from various philosophers having some revision due to external contributions (e.g. Ivan Illich has multiple works evolving over time due to interaction or discussions with people). It's true that the practice was not very common to publish about the evolution of the works. But that was mainly due to the slowness of the publishing mechanisms and not by the works themselves.

The main argument used to avoid freeing the works is usually the integrity of the author's work. A lot of works have been modified over time to reflect the current use of the language or make a translation to another language. Does this affect the integrity of the author's work? I don't think so. Especially for any free works (including free software) attribution is required in any case. So by default, the author (and the reader) would see the original attribution and the modification over time (recently improved in the free software community by the extensive use of distributed version control system like git).

Maybe it's now the time to reconsider that free software is going far beyond the simple act of creating software but also touching any act of thinking or creation.

Tags: freedom freesociety society copyright freesoftware