From Python package to Ubuntu package in 3-ish easy steps
Written by Barry Warsaw in technology on Mon 24 May 2010. Tags: canonical, ubuntu, debian, python, packaging,
My friend Tim is working on a very cool Bazaar-backed wiki project and he asked me to package it up for Ubuntu. I'm getting pretty good at packaging Python projects, but I always like the practice because each time it gets a little smoother. This one I managed to package in about 10 minutes so I thought I'd outline the very easy process.
First of all, you want to have a good setup.py, and if you like to cargo cult, you can start with this one. I highly recommend using Distribute instead of setuptools, and in fact the former is what Ubuntu gives you by default. I really like adding the distribute_setup.py which gives you nice features like being able to do python setup.py test and many other things. See lines 18 and 19 in the above referenced setup.py file.
The next thing you'll want is Andrew Straw's fine stdeb package, which you can get on Ubuntu with sudo apt-get install python-stdeb. This package is going to bootstrap your debian/ directory from your setup.py file. It's not perfectly suited to the task (yet, Andrew assures me :), but we can make it work!
These days, I host all of my packages in Bazaar on Launchpad, which is going to make some of the following steps really easy. If you use a different hosting site or a different version control system, you will have to build your Ubuntu package using more traditional means. That's okay, once you have your debian/ directory, it'll be fairly easy (but not as easy as described here). If you do use Bazaar, you'll just want to make sure you have the bzr-builddeb plugin. Just do sudo apt-get install bzr-builddeb on Ubuntu and you should get everything you need.
Okay, so now you have the requisite packages, and a setup.py, let's build us a deb and upload it to our personal package archive so everyone on Debian and Ubuntu can easily try it out.
First, let's create the debian directory. Here's the first little icky bit:
$ python setup.py --command-packages=stdeb.command sdist_dsc
Notice that this leaves us with a deb_dist/ directory, not the debian/ directory we want. The latter is in there, just buried a bit. Let's dig it out:
$ mv deb_dist/wikkid-0.1/debian . $ rm -rf deb_dist $ bzr add debian $ bzr commit -m'Debianize'
Note that wikkid-0.1 will be replaced by the name of your package. In order to build the .deb package, you need an orig.tar.gz file. Packaging sort of assumes that you've got an original upstream tarball somewhere and you're just adding the necessary Debian goo to package the thing. In this case, we don't have an upstream tarball, although we could easily create one, and upload it to the Cheeseshop or Launchpad or wherever. However, that just slows us down so let's skip that for now! (Aside: if you do have an upstream tarball somewhere, you'll want to add a debian/watch file which points to it; that'll eliminate the need to do the next step, by downloading the tarball instead.)
Let's create the tarball right now and copy it to where the following step will expect it:
$ python setup.py sdist $ mv dist/Wikkid-0.1.tar.gz ../wikkid_0.1.orig.tar.gz
Here's the second icky bit. Building a Debian source package imposes a very specific naming convention on the tarball. Wikkid's setup.py happens to build a tarball with an incompatible name, while the sdist command leaves it in a place where the next step can't find it. The rename just gets everything into the proper place. YMMV.
Now we can build the Debian source package. It's the source package that we'll upload to our Launchpad PPA. Launchpad will then automatically (if we've done everything right) build the binary package from the uploaded source package, from which Ubuntu and Debian users can easily install.
Oops! Before we do this, please edit your debian/changelog``y file and change ``unstable to lucid. You should also change the version number by adding a ~ppa1 to the end of it. Yeah, more ickiness.
Alright now we're ready to build our source package:
$ bzr bd -S
Now let's upload it (assuming you've enabled a PPA):
$ cd .. $ dput ppa:barry/python wikkid_0.1-1~ppa1_source.changes
That's it! If you've done everything successfully, you'll have the package in your PPA in 5 minutes or so. Then anybody who's added your PPA can just apt-get install wikkid (or whatever your package is called).
I do hope to work with the appropriate developers to make some of the ickiness go away. Please do contact me if you want to help!
Let's say you publish your tarball on the Cheeseshop or Launchpad, and you don't want to have to build a different tarball locally in order to package it. Here's what I think works:
Create a debian/watch file that points to the download location you publish to. If your package is not yet available in Debian or Ubuntu, then use this command to build your source package:
$ bzr bd -S -- -sa
The bit at the end tells the Debian packaging primitives to include your tarball when your source package is uploaded. The debian/watch file is used to download your published tarball and automatically renamed to the required .orig.tar.gz name. When you dput your package, your tarball will be uploaded too, and everything should build properly.
Addendum 2 (2010-06-10)
Andrew Straw has added a debianize command to his stdeb package, which makes things much nicer. With this you can create the debian/ directory right next to your setup.py. AFAIK, this version of stdeb isn't released yet, so you need to install his git head in a virtualenv, and it has a few minor buglets, but it does seem like the best-of-breed solution. I'll post another article with a more detailed follow up later.
Addendum 3 (2016-04-19)
As I convert this blog to the new platform, and am re-reading this article, I'm struck by how much has changed in the intervening six years. I think Python packaging for Debian and Ubuntu has improved greatly, with better guides and tools.
And since then, the Distribute package has merged back into setuptools, with the latter really becoming the defacto standard for upstream packaging of Python libraries and applications.
It's also interesting to note that I've since almost completely converted to using the git version control system, and I host all my personal projects (and big open source projects like GNU Mailman) on GitLab. We must evolve.