This week I had to write a little tool that would collect input on different channels, via socket / netcat, via a http server, .... Calls for a parralel Design -- Maybe a good place to write something real in Erlang. While erlang itself was really nice to write -- I do like Prolog as well as the bunch of functional languages -- doing networking in erlang seems a bit special, the interfaces just aren't thin wrappers around the libc stuff.

Getting a Socket Text interface

What sounds like a easy challenge to accomplish was actually harder than expected. All I found was some way to pass binaries representing erlang code over a socket and evaluating it remotle. While it's nice that such things are as easy to do as they are it doesn't help me with my task of moving simple strings.

start() ->
    {ok, Listen} = gen_tcp:listen(51622, [binary, {packet, 0},
                                         {reuseaddr, true},
                                         {active, true}]),
    spawn(fun () -> accept_loop(Listen) end).

accept_loop(Listen) ->
    {ok, Socket} = gen_tcp:accept(Listen),
    accept_loop(Listen),
    handle(Socket)

handle(Socket) ->
    receive
        {tcp, Socket, Bin} ->
            io:format("~p~n", binary_to_list(Bin));
        {tcp_closed, Socket} ->
            Buffer
    end.

So the socket is now up and receives text as wanted. However, as we are already runnign a parralel program it would be nice to be hable to handle multiple socket connections in parralel right? For that we just need to add a spawn() at the right place. The right place is not the handle(Socket) but the accept_loop(Listen) because the process that called accept will receive all the tcp messages.

This last part was quite obvious after finding the documentation of the {active, _} properties for the socket. always here means that you'll receive all data from the socket as erlang Messages, once delivers one package and waits until it is activated again and false requires calling a method -- this would have been possible as well when forking handle(Socket).

The web server

Ok we also want a webserver. We do not want to run some webapplication inside appache or so, just do some post/get and simple pages. Erlang here provides a built-in httpd with some mod_esi that calls some function depending on the URL used. It doesn't do anything fancy like templating or DB backends or stuff, just taking strings and building the http answers.

Unfortunately there are no examples around and basically noone seems to be using this combination (apart from some hacks like mine probably). So as I needed to get some additional information into the handler function (a Pid to connect to some service), I, as a novice, just couldn't find a way. Asking on IRC the solution was rather simple: Just using erlang's process registry. For more complex stuf gproc might prove usefull here.

Result

I guess I've got a huge step farther in programming erlang now. The manpages are easily found by your search engine -- for python I have to either walk through the (well structured) online documentation or search for the right link in the search results, for erlang they're typically first. Also Joe Armstrong's Books as proven usefull. The most dificult part probably is getting around all the nice extras you can do (transfering functions over sockets et al) and find out how to do the thing you need.

There are many ways to run some commands simultaneously on multiple hosts like cssh or dsh. They come handy for example when you are installing software updates on a set of hosts.

dsh is a rather simple comandline tool allowing to execute a command over ssh on multiple hosts. However it doesn't allow any interactive input -- so you can't look at the potentially upgrading packages and press y to accept and you can't go through debconf promts or similar.

This is solved by cssh which opens a XTerm for every host and a input area that is broadcastet to all of them. this is working really well -- you can execute your update on all hosts and still do individual adjustments just as needed: switch focus from the broadcasted input to one of the terminal windows and anything you type just goes there.

Now cssh has a big disadvantage: it requires a running X server (and doesn't do too well with a fullscreen windowmanager). Requiring X is quite a blocker if you need to run that ssh multiplexer on a remote host, for example if the firewalling doesn't allow direct connections. Fortunately you can make tmux behave as we want -- in a simple terminal:

First you need a script spawning the ssh sessions in separate tmux panes and direct input to all of them -- here called ssh-everywhere.sh (you could also write a tmux config I guess):

#/bin/sh
# ssh-everywhere.sh
for i in $HOSTS
do
  tmux splitw "ssh $i"
  tmux select-layout tiled
done
tmux set-window-option synchronize-panes on

Now start the whole thing:

tmux new 'exec sh ssh-everywhere.sh'

And be done.

Update

If you want to type in just one pane (on one host) you can do that as well: C-b : set-window-option synchronize-panes off and moving to the right pane (C-b + Arrow keys)

So this is a collection of things I came about when trying to get a Debian GNU/Hurd virtual machine running with kvm. Most of it is properly documented if you manage to find that particular piece of information.

Kernel Version

Due to a bug in linux 2.6.37 and .38 hurd will only boot if you supply -no-kvm-irqchip which is not that easy if you are using libvirt. A wrapper `kvm` script in the PATH will do, as will using a 2.6.39 kernel.

sudo

sudo will hang before returning from executing some command. I'm now using screen and sudo -i which keeps you a working tty gets you root and hasn't caused mayor trouble yet

sshd

openssh-server won't come up complaining about missing PRNG – and indeed there's no /dev/{u,}random in the default install. fix is to install random-egd from ports.

#!/usr/bin/python

from btsutils.debbugs import debbugs
import mailbox
import re
import sys

mailbox =  mailbox.Maildir(sys.argv[1], factory=False)
bts = debbugs()

for key in mailbox.keys():
    message = mailbox[key]
    if not 'S' in message.get_flags():
        if message['X-Debian-PR-Message']:
            try:
                bugnr = message['X-Debian-PR-Message'].split()[1]
            except IndexError:
                continue
        else:
            test = re.search('Bug#(\d{6})', message['Subject'])
            if test:
                bugnr = test.group(0)[4:]
            else:
                continue

        try:
            bug = bts.get(bugnr)
        except AttributeError:
            print bugnr
            continue

        if bug.getStatus() == u'done':
            message.set_flags(message.get_flags() + 'S')
            mailbox[key] = message


mailbox.flush()

Run it like python cleanbugsmail.py ~/Maildir/.debian.bugs. Anyone aware of a better solution?