Roberto De Almeida

After reading Angels & Demons (I almost typed daemons) I decided to make an ambigram out of name, ie, a drawing that remains unaltered after a 180 degrees rotation. Turns out that my name is pretty easy for this:

You should be able to read "Roberto de almeida" above! The drawing is a little sketchy because I did with the mouse using Inkscape, since I don't have a scanner at hand.

I'm doing an experiment with my recent papers: each has a Trac instance, so that code and the article text (a LaTeX file) can be shared between colaborators using Subversion. Data is hosted on an Opendap server, running my trusty pydap. But the best part is that the whole process, from downloading data to composing the article pdf, is done through a Makefile. The Makefile creates a virtual environment, downloads the required Python modules, and bootstraps the analyses. Just type make and enjoy.

Now imagine if all scientists embraced the open-source philosophy of open collaboration and real meritocracy and did the same.

Seabird is a common brand of CTD, an equipment designed to measure conductivity, temperature and depth on the ocean. From conductivity it's possible to calculate the water salinity, and from that you obtain a profile of the density of the water. Small changes in density are responsible for part of the ocean currents that flow around the world.

The Seabird processing software generates files with the cnv extension, containing a header with the metadata followed by the data as ASCII or in binary format. The file header can look like this:

* Some info
* Some kind of variable: with data
** Two stars:  must really important
# but there's also hashes
# and keys here like = this
# file_type: binary
*END*

The data follows, either as packed 4-byte floats or column separated ASCII values. Pretty simple stuff.

This week I wrote a pydap plugin to serve Seabird files. The parsing code, which could be useful to other oceanographers using Python, looks like this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19

header, data = content.split('*END*\r\n', 1)

# Process headers.
p = re.compile("""
    (?:\*{1,2}|\#)\s    # starts with "* " or "** " or "# "
    (.*?)               # everything until next token
    \s*(?::|=|$)\s*     # ":  " or " = " or EOL
    (.*)                # all the rest, if any
""", re.VERBOSE)
attributes = (p.match(line).groups() for line in header.split('\r\n') if line.strip())
attributes = dict((k, lazy_eval(v.strip())) for (k, v) in attributes)

file_type = attributes.get('file_type', 'binary').lower()
nquan = attributes.pop('nquan')
if file_type == 'binary':
    data = fromstring(data, 'f')
elif file_type == 'ascii':
    data = fromstring(data, sep=' ')
data.shape = (-1, nquan)

The code works by splitting the metadata and the data at the *END* point. The headers are processed using a somewhat generous regular expression that looks for (key,value) pairs, leaving the value as an empty string when necessary. The function lazy_eval comes from pydap, and consists of a safe eval that returns only strings, numbers, lists and tuples.

The data handling is even more simple. We decode the data using numpy's fromstring, depending on if it's in ASCII or binary. The data is then reshaped, according to the number of variables described in the metadata. Pretty straightforward.

Paul Graham has issued a silly challenge to compare show how Arc compares with other languages. The idea is to

generate a page with an input field and a submit button. If the user clicks on submit, he gets a second page with a link saying "click here." If he clicks on that, he gets a third page saying "you said: ..." where ... was whatever he put in the input field. This has to happen without the value being passed in the url; it should not be possible to change the behavior of the third page by editing the url in the second.

Here's my take. Instead of using Python I used Javascript, together with the fantastic jQuery library:

1
2
3
4
5
6

$('<form><input /><input type=submit /></form>').appendTo('body').submit(function() {
    $('form input').hide().parent().append('<a href="#">continue</a>').find('a').click(function() {
        $('form').html('you said: ' + $('form input').val());
    });
    return false;
});

The code runs completely on the client-side, and uses a loose definition of "page", since the browser never reloads. After coming up with this solution I saw a couple more of Javascript solutions on the challenge page, and one of them also uses jQuery. But I was more interested in this Seaside solution from Lukas Renggli. I tried playing with Squeak/Seaside some time ago, but the documentation consisted of out-of-date tutorials spread over a dozen different websites. Maybe I should give it another try.

Magicdate is a simple Python module that I wrote, inspired by Simon Willison's dateparse.js, to parse fuzzy dates like "last sunday" or "Jan 4th". Particularly, I like to use it with my Python scripts to parse command line arguments; it even comes with a custom parser for optparse:

1
2
3
4
5

from optparse import OptionParser
import magicdate
parser = OptionParser(option_class=magicdate.MagicDateOption)
parser.add_option('-s', '--start', dest='start', type='magicdate', default=None)
parser.add_option('-e', '--end', dest='end', type='magicdate', default='today')

So it's now possible to do:

1

$ some-script.py --start "1 week and 2 days ago" --end yesterday

Today I added support for the "x time ago" syntax, per Kurt's request (the only person I know who has a CV as a Google Earth KML file!). The new syntax was pretty easy to implement, since the code iterates over pairs of regexps/parsing functions to handle the processing when a match is found. All I had to do was add a new function and upload the code to the Cheeseshop.

The module still needs a homepage (the Cheeseshop entry points to a 404, although the download link works). I need to organize all my projects on my homepage, but it'll have to wait a little.

And speaking of 404s, two weeks ago I went to a music school close to home, to have pandeiro (a brazilian percussion instrument) lessons. When a woman sent me to room 404 I had to fight the urge to make the joke "I'll never find it!". :)