Working with abstractions

It’s lovely how when you use the correct abstractions everything falls right into place. Pydap is a library that abstracts a remote dataset as local Python objects. I’ve been developing it since 2003, the library is currently in its third major version, and the number of lines of code has been steadily declining as I get the right abstractions, avoiding the need for code that handle “special cases” and exceptions to the rule.

As a nice example, I have some code to handle sequential data that looks like this:

def get_row(data, col, level):
    if level == 1:
        return [ line[col] for line in data ]
    else:
        return [ get_row(value, col, level-1) for value in data ]
 
def combine_rows(data, level):
    if level == 1:
        return zip(*data)
    else:
        return [ combine_rows(value, level-1) for value in zip(*data) ]
 

Basically, if I have a sequence of variables “a”, “b” and “c”, this code will combine the data from each variable into a list of tuples and vice-versa. It works with arbitrarily nested sequences up to any level, which explains its recursive nature.

After some modifications in the code I wanted to reuse these two functions with some variables that could also be outside a sequence. In this case, the level would be zero, so I had to add:

def get_row(data, col, level):
    if level == 0:
        return data[col]
    elif level == 1:
        return [ line[col] for line in data ]
    else:
        return [ get_row(value, col, level-1) for value in data ]
 
def combine_rows(data, level):
    if level == 0:
        return data
    elif level == 1:
        return zip(*data)
    else:
        return [ combine_rows(value, level-1) for value in zip(*data) ]
 

And then I realized that the special case for level 1 could be removed, with this simply rewritten as:

def get_row(data, col, level):
    if level == 0:
        return data[col]
    else:
        return [ get_row(value, col, level-1) for value in data ]
 
def combine_rows(data, level):
    if level == 0:
        return data
    else:
        return [ combine_rows(value, level-1) for value in zip(*data) ]
 

This convinced me, in a poetic and inductive way, that the code was indeed right, and that I had chosen the right abstractions for my code.