Tuesday, June 8, 2010

import this and The Zen of Python

Richard Jones is working on a talk for PyCon Australia and asked me about the history of the Zen of Python, Tim Peters' eternal words of wisdom, often quoted, on the essential truths of Python. At first, I couldn't find a reference to the first publication of this list, but then I did a better search of my archives and found that it was originally sent to the python-list mailing list on June 4, 1999, under the subject "The Python Way".

Interestingly enough, because I couldn't find that first reference immediately, I went back into my archives and researched the "this" module. Did you know that if you type the following at a modern Python interpreter, you get the Zen of Python?

% python
Python 2.6.5 (r265:79063, Apr 16 2010, 13:57:41)
[GCC 4.4.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import this
The Zen of Python, by Tim Peters

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!

The story behind "import this" is kind of funny, and occurred totally behind the scenes, so I thought it might be interesting to relate how it happened. Maybe something to add to Guido's History of Python blog.

Anyway, back in fall 2001, Foretec was organizing the International Python Conference #10 (IPC 10, precursor to Pycon). Foretec was a conference organizing company owned by CNRI, which earlier had employed Guido, Fred, Jeremy and myself, and which by 2000 we had left to form Pythonlabs at various friendly (and occasionally disfunctional) corporate homes. By September 13, 2001 (yes, how weird!) we were working for our friends at Zope Corporation (perhaps not yet rebranded from Digitial Creations), but Foretec was still organizing the IPCs. They wanted to have a slogan for the conference, which could be printed on a t-shirt, and wanted to gather submissions from the Python community. Pythonlabs agreed to judge the entries and select a winner. However, I think Guido's wife was due any day and he didn't have much time or energy to go through the entries.

We got something like 500 entries, almost all of them terrible. Tim's exact words were "Jeez Louise, I can't look at these for more than 5 minutes w/o my brain turning to mush" and yet he still managed to do an initial cut down to something like 130 entries. While we had agreed to choose a winner, we procrastinated until the last minute and by then it was obvious that Tim and I were the only ones crazy enough to still care. Tim suggested that we trade the list back and forth between the two of us, each cutting the list in half until there was just one left. Tim is much better at math than me, and I had forgotten about Python's integer division so I was left to choose from between the last two entries: "Bite off all you like - Chewing is optional" (yes, I said most were terrible ;), and "import this". At the last minute, Tim resurrected a fun one that we had both noticed approvingly early on: "Let's we study about Python program".

While that last one had lots of appeal, I liked the irreverent, almost sneering tone of "import this". I saw the potential for a great, Michael Jackson-esque t-shirt.

As soon as we'd chosen "import this" I realized we just had to implement it. Python 2.2 was about to be released and I proposed that we turn off checkin notifications and sneak in a "this.py" module which when imported just printed the Zen of Python. Tim or Guido suggested further that we rot13 the contents of the module just for a little extra obfuscation, and we told no one outside our little group. According to my intergoogles spelunking, as soon as IPC 10 was concluded, we commemorated the event by committing this.py to what was to become Python 2.2.1, thus adding to the affront of new features in point releases. IIRC, it took a long time for someone to find our little easter egg.

That was all back in the day when the Python community had a sense of humor.

Monday, June 7, 2010

Experimental Virtual Machines

I'm doing some work these days on trying to get Python 2.7 as the default Python in the next version of Ubuntu, Maverick Meerkat (10.10). This work will occasionally require me to break my machine by installing experimental packages. That's a good and useful thing because I want to test various potentially disruptive changes before I think about unleashing them on the world. This is where virtual machines really shine!

To be efficient, I need a really fast turnaround from known good state, to broken state, back to known good state. In the past, I've used VMware Fusion on my Mac to create a VM, then take a live snapshot of the disk before making my changes. It was really easy then to revert to the last known good snapshot, try something else and iterate.

But lately Fusion has sprouted a nasty habit of freezing the host OS, such that a hard reboot is necessary. This will inevitably cause havoc on the host, by losing settings, trashing mail, corrupting VMs, etc. VMware can't reproduce the problem but it happens every time to me, and it hurts, so I'm not doing that any more :).

Back to my Lucid host and libvirt/kvm and the sanctuary of FLOSS. It's really easy to create new VMs, and there are several ways of doing it, from virt-manager to vmbuilder to straight up kvm (thanks Colin for some recipes). The problem is that none of these are exactly fast to go from bare metal to working Maverick VM with all the known good extras I need (like openssh-server and bzr, plus my comfortable development environment).

I didn't find a really good fit for vmbuilder or the kvm commands, and I'm not smart enough to use the libvirt command line tools, but I think I've figured out a hack using virt-manager that will work well enough.

1. Create a disk for the baseline VM (named 'scars' in my case :) manually
% qemu-img create -f qcow2 scars.qcow2 20G

2. Create the baseline VM using virt-manager
* I use dhcp internally, so I give this thing a mac address, assign it 1GB
of RAM and 1 processor.
* For storage, I tell it to use the scars.qcow2 file I created above
* Boot from the maverick ISO of your choice, install everything you want,
and get your development environment in place
* Shut this machine down

3. Clone your baseline VM
* In the virt-manager Manager window, right click on your baseline VM and
select Clone
* You will not be given an opportunity to select a disk or a mac address,
so for now just go with the defaults.
* Do not start your clone

4. Create an 'overlay' disk that is a backed by your baseline disk.
% qemu-img create -f qcow2 -b scars.qcow2 scars.ovl

5. Edit your clone
* Delete the disk given to your clone by default
* Create a new virtio storage that points to scars.ovl
* Delete the nic given to your clone by default
* Create a new virtio network device with the mac address of your
baseline. You'll get a warning about a mac address collision, but this
can be ignored (see below).

6. Boot your clone

At this point you'll have a baseline which is your known good system, and a clone/overlay which you can break to your heart's content. When it's time to iterate back to a known good state, shut down your clone, delete the overlay disk, and create a new one from the baseline qcow2 disk. This is pretty fast, and your turn around time is not much more than the time it takes to shutdown one machine and boot another. It actually feels a lot faster by the wall clock than Fusion ever was to snapshot and restore.

One downside is that you cannot run both VMs at the same time. I think mostly this is because of the MAC address collision, but also because creating the overlay requires that both machines be powered off.

The other downside seems to be that if you want to update your known good baseline, say by installing more packages or apt-get update/upgrade, you will have to recreate your overlay disk for your next experiment. Changes to the underlying disk do not seem to propagate to the overlay automatically. Maybe that's intentional; I can't find much documentation on it. (Note too that the manpage for qemu-img does not describe the -b option.)

I guess the last downside is that I spent way too much time trying to figure all this out. The Googles were not a lot of help but did give me the qemu-img clue. But at least now you don't have to! :)