Category: technology

My Natty Upgrade Test

Written by Barry Warsaw in technology on Fri 15 April 2011. Tags: test drive, ubuntu, virtual machine,

Ubuntu 11.04 (code name: Natty Narwhal) beta 2 was just released and the final release is right around the corner. Canonical internal policy is that we upgrade to the latest in-development release as soon as it goes beta, to help with bug fixing, test, and quality assurance.

Now, I've been running Natty on my primary desktops (my two laptops) since before alpha 1, and I've been very impressed with the stability of the core OS. One of my laptops cannot run Unity though, so I've mostly been a classic desktop user until recently. My other laptop can run Unity, but compiz and the wireless driver were too unstable to be usable, that is until just before beta 1. Still, I diligently updated both machines daily and at least on the classic desktop, Natty was working great. (Now that beta 1 is out, the wireless and compiz issues have been cleared up and it's working great too.)

The real test is my beefy workstation. This is a Dell Studio XPS 435MT 12GB, quad-core i7-920, with an ATI Radeon HD 4670 graphics card, running dual-headed into two Dell 20" 1600x1200 flat panel displays. During the Maverick cycle I was a little too aggressive in upgrading it, because neither the free nor the proprietary drivers were ready to handle this configuration yet. I ended up with a system that either couldn't display any graphics, or didn't support the dual heads. This did eventually all get resolved before the final release, but it was kind of painful.

So this time, I was a little gun shy and wanted to do more testing before I committed to upgrading this machine. Just before Natty beta 1, I dutifully downloaded the daily liveCD ISO, and booted the machine from CD. On the surface, things seemed promising …

Continue reading »


Release that rewrite on 11.11.11

Written by Barry Warsaw in technology on Mon 04 April 2011. Tags: floss, mailman, opensource, python,

I know that the Mailman 3 project is not alone in procrastinating getting out a release of its major rewrite. It's hard work to finish a rewrite on your own copious spare time. I was just chatting with Thomas Waldmann of the Moin project on IRC, and he lamented a similar story about the Moin 2 release. Then he said something that really made me sit up straight:

<ThomasWaldmann> 11.11.11 would be a great date for something :)

Yes, it would! We have the 2011 Google Summer of Code happening soon (students, you have until April 8th to submit your applications) so many free and open source software projects will get some great code coming soon. And November is far enough out that we can plan exactly what a "release" means. Here's what I propose:

Let's make November 11, 2011 the "Great FLOSS Release Day". If you're working on an open source project undergoing a major new version rewrite, plan on doing your release on 11.11.11. It can be a beta or final release, but get off your butts and make it happen! There's nothing like a good deadline to motivate me, so Mailman 3 will be there. Add a comment here if you want your project to be part of the event!


Charming Snakes and Shaving Yaks

Written by Barry Warsaw in technology on Mon 28 March 2011. Tags: debugging, python, ubuntu,

For the last couple of days I've been debugging a fun problem in the Ubuntu tool called Jockey. Jockey is a tool for managing device drivers on Ubuntu. It actually contains both a command-line and a graphical front-end, and a dbus backend service that does all the work (with proper authentication, since it modifies your system). None of that is terribly relevant to the problem, although the dbus bit will come back to haunt us later.

What is important is that Jockey is a Python application, written using many Python modules interfacing to low-level tools such as apt and dbus. The original bug report was mighty confusing. Aside from not being reproducible by myself and others, the actual exception made no fricken sense! Basically, it was code like this that was throwing a TypeError:

_actions = []
# _actions gets appended to at various times and later...
for item in _actions[:]:
    # do something

Everyone who reported the problem said the TypeError was getting thrown on the for-statement line. The exception message indicated that Python was getting some object that it was trying to convert to an integer, but was failing. How could you possible get that exception when either making a copy of a list or iterating over that copy? Was the list corrupted? Was it not actually a list but some list-like object that was somehow returning non-integers for its min and max indexes?

To make matters worse, this little code snippet was in Python's standard library, in the subprocess module. A quick search of Python's bug database did reveal some recent threads about changes here, made to ensure that popen objects got properly cleaned up by the garbage collector if they weren't cleaned up explicitly by the program. Note that we're using Python 2.7 here, and after some reading …

Continue reading »


What we do

Written by Barry Warsaw in technology on Thu 09 September 2010. Tags: canonical, python, ubuntu,

My friends and family often ask me what I do at my job. It's easy to understand when my one brother says he's a tax accountant, but not so easy to explain the complex world of open source software development I live in. Sometimes I say something to the effect: well, you know what Windows is, and you know what the Mac is right? We're building a third alternative called Ubuntu that is free, Linux-based and in most cases, much better. Mention that you won't get viruses and it can easily breathe new life into that old slow PC you shudder to turn on, and people at least nod their heads enthusiastically, even if they don't fully get it.

I've been incredibly fortunate in my professional career, to have been able to share the software I write with the world for almost 30 years. I started working for a very cool research lab with the US Federal government while still in high school. We had a UUCP connection and were on the early Arpanet, and because we were funded by the US taxpayer, our software was not subject to copyright. This meant that we could share our code with other people on Usenet and elsewhere, collaborate with them, accept their suggestions and improvements, and hopefully make their lives a little better, just as others around the world did for us. It was free and open source software before such terms were coined.

I've never had a "real job" in the sense of slaving away in a windowless cube writing solely proprietary software that would never see the light of day. Even the closed source shops I've worked at have been invested somehow in free software, and with varying degrees of persuasion, have both benefited from and contributed to the …

Continue reading »


import this and the Zen of Python

Written by Barry Warsaw in technology on Tue 08 June 2010. Tags: python, history,

Richard Jones is working on a talk for PyCon Australia and asked me about the history of the Zen of Python, Tim Peters' eternal words of wisdom, often quoted, on the essential truths of Python. At first, I couldn't find a reference to the first publication of this list, but then I did a better search of my archives and found that it was originally sent to the python-list mailing list on June 4, 1999, under the subject "The Python Way".

Interestingly enough, because I couldn't find that first reference immediately, I went back into my archives and researched the "this" module. Did you know that if you type the following at a modern Python interpreter, you get the Zen of Python?

% python3 -c "import this"
The Zen of Python, by Tim Peters

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!

The story behind "import this" is kind of funny, and occurred totally behind the scenes, so I thought it might be interesting to relate how it happened …

Continue reading »


Experimental Virtual Machines

Written by Barry Warsaw in technology on Mon 07 June 2010. Tags: canonical, python, ubuntu,

I'm doing some work these days on trying to get Python 2.7 as the default Python in the next version of Ubuntu, Maverick Meerkat (10.10). This work will occasionally require me to break my machine by installing experimental packages. That's a good and useful thing because I want to test various potentially disruptive changes before I think about unleashing them on the world. This is where virtual machines really shine!

To be efficient, I need a really fast turnaround from known good state, to broken state, back to known good state. In the past, I've used VMware Fusion on my Mac to create a VM, then take a live snapshot of the disk before making my changes. It was really easy then to revert to the last known good snapshot, try something else and iterate.

But lately Fusion has sprouted a nasty habit of freezing the host OS, such that a hard reboot is necessary. This will inevitably cause havoc on the host, by losing settings, trashing mail, corrupting VMs, etc. VMware can't reproduce the problem but it happens every time to me, and it hurts, so I'm not doing that any more :).

Back to my Lucid host and libvirt/kvm and the sanctuary of FLOSS. It's really easy to create new VMs, and there are several ways of doing it, from virt-manager to vmbuilder to straight up kvm (thanks Colin for some recipes). The problem is that none of these are exactly fast to go from bare metal to working Maverick VM with all the known good extras I need (like openssh-server and bzr, plus my comfortable development environment).

I didn't find a really good fit for vmbuilder or the kvm commands, and I'm not smart enough to use the libvirt command line tools, but I think …

Continue reading »


We have a winner!

Written by Barry Warsaw in technology on Wed 26 May 2010. Tags: mailman,

In the early part of 2010, we started a contest for a new GNU Mailman logo. Our old logo, donated by the Dragon de Monsyne had served us well for many years, but it felt like we needed a refresh. Besides, we wanted a nice scalable graphic that we could use in many more situations. So we solicited entries and then conducted a poll. Today I am very pleased to announce the winner!

Mailman logo

By better than 2-to-1, this submission by Andrija Arsic was voted as best logo by the Mailman community. Congratulations Andrija!

While we have yet to re-brand the website and software to include the new logo, we'll start using it immediately. If you'd like to help with any redesign, please contact us at mailman-developers@python.org.

A little bit about Andrija: originally from Trstenik, Serbia and now studying IT technology in Belgrade, Andrija is a self-employed, part-time graphic designer, specialising in the fields of corporate identity (logo) design, web design, print design and branding with the majority of his time spent designing and implementing marketing promotions for businesses such as logos, websites, letterhead, business cards, packaging and more. I'm glad that he also contributes to free software, as I think his winning logo is spectacular.

My thanks and appreciation to all the artists who contributed logos to the contest. All the designs are very nice, and in their own way, capture the spirit of GNU Mailman.


From Python package to Ubuntu package in 3-ish easy steps

Written by Barry Warsaw in technology on Mon 24 May 2010. Tags: canonical, ubuntu, debian, python, packaging,

My friend Tim is working on a very cool Bazaar-backed wiki project and he asked me to package it up for Ubuntu. I'm getting pretty good at packaging Python projects, but I always like the practice because each time it gets a little smoother. This one I managed to package in about 10 minutes so I thought I'd outline the very easy process.

First of all, you want to have a good setup.py, and if you like to cargo cult, you can start with this one. I highly recommend using Distribute instead of setuptools, and in fact the former is what Ubuntu gives you by default. I really like adding the distribute_setup.py which gives you nice features like being able to do python setup.py test and many other things. See lines 18 and 19 in the above referenced setup.py file.

The next thing you'll want is Andrew Straw's fine stdeb package, which you can get on Ubuntu with sudo apt-get install python-stdeb. This package is going to bootstrap your debian/ directory from your setup.py file. It's not perfectly suited to the task (yet, Andrew assures me :), but we can make it work!

These days, I host all of my packages in Bazaar on Launchpad, which is going to make some of the following steps really easy. If you use a different hosting site or a different version control system, you will have to build your Ubuntu package using more traditional means. That's okay, once you have your debian/ directory, it'll be fairly easy (but not as easy as described here). If you do use Bazaar, you'll just want to make sure you have the bzr-builddeb plugin. Just do sudo apt-get install bzr-builddeb on Ubuntu and you should get everything you need.

Okay, so now you …

Continue reading »


Gentoo No More

Written by Barry Warsaw in technology on Sun 23 May 2010. Tags: canonical, servers, ubuntu,

Today I finally swapped my last Gentoo server for an Ubuntu 10.04 LTS server. Gentoo has served me well over these many years, but with my emerge updates growing to several pages (meaning, I was waaaay behind on updates with almost no hope of catching up) it was long past time to switch. I'd moved my internal server over to Ubuntu during the Karmic cycle, but that was a much easier switch. This one was tougher because I had several interdependent externally facing services: web, mail, sftp, and Mailman.

The real trick to making this go smoothly was to set up a virtual machine in which to install, configure and progressively deploy the new services. My primary desktop machine is a honkin' big i7-920 quad-core Dell with 12GB of RAM, so it's perfectly suited for running lots of VMs. In fact, I have several Ubuntu, Debian and even Windows VMs that I use during my normal development of Ubuntu and Python. However, once I had the new server ready to go, I wanted to be able to quickly swap it into the real hardware. So I purchased a 160GB IDE drive (since the h/w it was going into was too old to support SATA, but still perfectly good for a simple Linux server!) and a USB drive enclosure. I dropped the new disk into the enclosure, mounted it on the Ubuntu desktop and created a virtual machine using the USB drive as its virtio storage.

It was then a pretty simple matter of installing Ubuntu 10.04 on this USB drive-backed VM, giving the VM an IP address on my local network, and installing all the services I wanted. I could even register the VM with Landscape to easily keep it up-to-date as I took my sweet time …

Continue reading »


MBP

Written by Barry Warsaw in technology on Wed 09 December 2009.

So I got my new Mac Book Pro, named it "hemispheres" as mentioned in my previous post. I'm very happy with this machine, and here are my early impressions.

The SSD is pure win. It's quiet, cool, and fast. The biggest immediate downside of course is the cost, but as a refurb, this option was made much more affordable. It'll be interesting to see how it performs over time as there are known issues with the long-term use of the technology.

My two biggest concerns were the screen and the keyboard. I absolutely love my previous generation MBP keyboard, as it's about the only laptop I can comfortably use for 8 hours a day. The unibody MBPs have a completely different keyboard, with a different feel, but I hadn't used one for a long hacking session so I was unsure how it would feel after a few hours under my hands. I'm very relieved to say that while it has a different feel, it's still incredibly comfortable to use. Apple seems to know what they're doing.

I have had a few problems hitting the right keys, most notably not quite getting the shift-2 chord right to produce an @ sign. The reach is a little bit longer it seems so I often end up just typing a "2", somehow missing the shift combination. I have a few other common typing errors, but for the most part I'm getting used to it.

The other big uncertainty was the antiglare screen. I absolutely love the matte screen on my old MBP and hate hate HATE the glossy screens that seem to come on most laptops today. Honestly, if the MBP didn't offer a matte screen as an option, I probably wouldn't have bought it. Of course, now it's a $50 upcharge, but I …

Continue reading »