Posts Tagged With 'debian'
Way back in 2011 I wrote an article describing how you can build Debian
packages with local dependencies for testing purposes. An example would be a
new version of a package that has new dependencies. Or perhaps the new
dependency isn't available in Debian yet. You'd like to test both packages
together locally before uploading. Using sbuild and autopkgtest you can
have a high degree of confidence about the quality of your packages before you
Here I'll describe some of the improvements in those tools, and give you
simplified instructions on how to build and test packages with local
Several things have changed. Probably the biggest thing that simplifies the
procedure is that GPG keys for your local repository are no longer needed.
Another thing that's improved is the package testing support. It used to be
that packages could only be tested during build time, but with the addition of
the autopkgtest tool, we can also test the built packages under various
scenarios. This is important because it more closely mimics what your
package's users will see. One thing that's cool about this for Python
packages is that autopkgtest runs an import test of your package by
default, so even if you don't add any explicit tests, you still get
something. Of course, if you do want to add your own tests, you'll need to
recreate those default tests, or check out the autodep8 package for some
I've moved the repository of scripts over to git.
The way you specify the location of the extra repositories holding your local
debs has changed. Now, instead of providing a directory on the local file
system, we're going to fire up a simple Python-based HTTP server and use that
as a new repository URL. This won't be …
Continue reading »
For UDS-R for Raring (i.e. Ubuntu 13.04) in Copenhagen, I sponsored
three blueprints. These blueprints represent most of the work I will be doing
for the next 6 months, as we're well on our way to the next LTS, Ubuntu 14.04.
I'll provide some updates to the other blueprints later, but for now, I want
to talk about OAuth and Python 3. OAuth is a protocol which allows you to
programmatically interact with certain website APIs, in an authenticated
manner, without having to provide your website password. Essentially, it
allows you to generate an authorization token which you can use instead, and
it allows you to manage and share these tokens with applications, so that you
can revoke them if you want, or decide how and which applications to trust to
act on your behalf.
A good example of a site that uses OAuth is Launchpad, but many other sites
also support OAuth, such as Twitter and Facebook.
There are actually two versions of OAuth out there. OAuth version 1 is
definitely the more prevalent, since it has been around for years, is
relatively simple (at least on the client side), and enshrined in RFC 5849.
There are tons of libraries available that support OAuth v1, in a multitude
of languages, with Python being no exception.
OAuth v2 is much less common, since it is currently only a draft
specification, and has had its share of design-by-committee controversy.
Still, some sites such as Facebook do require OAuth v2.
One of the very earliest Python libraries to support OAuth v1, on both the
client and server side, was python-oauth (I'll use the Debian package names
in this post), and on the Ubuntu desktop, you'll find lots of scripts and
libraries that use python-oauth. There are major problems with …
Continue reading »
So, now all the world now knows that my suggested code name for Ubuntu 12.10,
Qwazy Quahog, was not chosen by Mark. Oh well, maybe I'll have more luck
with Racy Roadrunner.
In any case, Ubuntu 12.04 LTS is to be released any day now so it's time
for my semi-annual report on Python plans for Ubuntu. I seem to write about
this every cycle, so 12.10 is no exception. We've made some fantastic
progress, but now it's time to get serious.
For Ubuntu 12.10, we've made it a release goal to have Python 3 only on the
desktop CD images. The usual caveats apply: Python 2.7 isn't going away; it
will still probably always be available in the main archive. This release
goal also doesn't affect other installation CD images, such as server, or
other Ubuntu flavors. The relatively modest goal then only affects
packages for the standard desktop CD images, i.e. the alternative installation
CD and the live CD.
Update 2012-04-25: To be crystal clear, if you depend on Python 2.7, the
only thing that changes for you is that after a fresh install from the
desktop CD on a new machine, you'll have to explicitly apt-get install
*python2.7. After that, everything else will be the same.
This is ostensibly an effort to port a significant chunk of Ubuntu to Python
3, but it really is a much wider, Python-community driven effort. Ubuntu has
its priorities, but I personally want to see a world where Python 3 rules the
day, and we can finally start scoffing at Python 2 :).
Still, that leaves us with about 145 binary packages (and many fewer source
packages) to port. There are a few categories of packages to consider:
- Already ported and available.
- This is …
Continue reading »
Time for another installment of my ongoing mission to convert the world to
Python 3! This time, a little Debian packaging-fu for modifying an existing
Python 2 package to include support for Python 3 from the same source package.
Today, I added a python3-feedparser package to Ubuntu Precise. What's
interesting about this is that, despite various reported problems, upstream
feedparser 5.1 claims to support Python 3, via 2to3 conversion. And indeed
it does (although the test suite does not).
Before today, Ubuntu had feedparser 5.0.1 in its archive, and while some
work has been done to update the Debian package to 5.1, this has not been
released. The uninteresting precursor to Python 3 packaging was to upgrade
the Ubuntu version of the python-feedparser source package to 5.1. I'll spare
you the boring details about missing data files in the upstream tarball, and
other problems, since they don't really relate to the Python 3 effort.
The first step was to verify that feedparser 5.1 works with Python 3.2 in a
virtualenv, and indeed it does. This is good news because it means that the
setup.py does the right thing, which is always the best way to start
supporting Python 3. I've found that it's much easier to build a solid Debian
package if you have a solid setup.py in upstream to begin with.
Now, what I'd like to do is to give you a recipe for modifying your existing
debian/ directory files to add Python 3 support to a package that already
exists for Python 2. This is a little trickier for feedparser because it used
an older debhelper standard, and carried some crufty old stuff in its
rules file. My first step was to update this to debhelper compatibility
level 8 …
Continue reading »
Update 2016-11-28: I've updated this article with new instructions!
sbuild is an excellent tool for locally building Ubuntu and Debian packages.
It fits into roughly the same problem space as the more popular pbuilder, but
for many reasons, I prefer sbuild. It's based on schroot to create chroot
environments for any distribution and version you might want. For example, I
have chroots for Ubuntu Oneiric, Natty, Maverick, and Lucid, Debian Sid,
Wheezy, and Squeeze, for both i386 and amd64. It uses an overlay filesystem
so you can easily set up the primary snapshot with whatever packages or
prerequisites you want, and the individual builds will create a new session
with an overlaid temporary filesystem on top of that, so the build results
will not affect your primary snapshot. sbuild can also be configured to save
the session depending on the success or failure of your build, which is
fantastic for debugging build failures. I've been told that Launchpad's build
farm uses a customized version of sbuild, and in my experience, if you can get
a package to build locally with sbuild, it will build fine in the main archive
or a PPA.
Right out of the box, sbuild will work great for individual package builds,
with very little configuration or setup. The Ubuntu Security Team's wiki
page has some excellent instructions for getting started (you can stop
reading when you get to UMT :).
One thing that sbuild doesn't do very well though, is help you build a stack
of packages. By that I mean, when you have a new package that itself has new
dependencies, you need to build those dependencies first, and then build your
new package based on those dependencies. Here's an example.
I'm working on bug 832864 and I wanted to see if I could build the …
Continue reading »
So, yesterday (June 21, 2011), six talented and motivated Python hackers from
the Washington DC area met at Panera Bread in downtown Silver Spring,
Maryland to sprint on PEP 382. This is a Python Enhancement Proposal to
introduce a better way for handling namespace packages, and our intent is to
get this feature landed in Python 3.3. Here then is a summary, from my own
spotty notes and memory, of how the sprint went.
First, just a brief outline of what the PEP does. For more details please
read the PEP itself, or join the newly resurrected import-sig for more
discussions. The PEP has two main purposes. First, it fixes the problem of
which package owns a namespace's __init__.py file,
e.g. zope/__init__.py for all the Zope packages. In essence, it eliminate
the need for these by introducing a new variant of .pth files to define a
namespace package. Thus, the zope.interfaces package would own
zope/zope-interfaces.pth and the zope.components package would own
zope/zope-components.pth. The presence of either .pth file is enough
to define the namespace package. There's no ambiguity or collision with these
files the way there is for zope/__init__.py. This aspect will be very
beneficial for Debian and Ubuntu.
Second, the PEP defines the one official way of defining namespace packages,
rather than the multitude of ad-hoc ways currently in use. With the pre-PEP
382 way, it was easy to get the details subtly wrong, and unless all
subpackages cooperated correctly, the packages would be broken. Now, all you
do is put a * in the .pth file and you're done.
Sounds easy, right? Well, Python's import machinery is pretty complex, and
there are actually two parallel implementations of it in Python 3.3, so
gaining traction on …
Continue reading »
My friend Tim is working on a very cool Bazaar-backed wiki project and he
asked me to package it up for Ubuntu. I'm getting pretty good at packaging
Python projects, but I always like the practice because each time it gets a
little smoother. This one I managed to package in about 10 minutes so I
thought I'd outline the very easy process.
First of all, you want to have a good setup.py, and if you like to cargo
cult, you can start with this one. I highly recommend using
Distribute instead of setuptools, and in fact the former is what Ubuntu gives
you by default. I really like adding the distribute_setup.py which gives
you nice features like being able to do python setup.py test and many other
things. See lines 18 and 19 in the above referenced setup.py file.
The next thing you'll want is Andrew Straw's fine stdeb package, which you
can get on Ubuntu with sudo apt-get install python-stdeb. This package is
going to bootstrap your debian/ directory from your setup.py file.
It's not perfectly suited to the task (yet, Andrew assures me :), but we can
make it work!
These days, I host all of my packages in Bazaar on Launchpad, which is going
to make some of the following steps really easy. If you use a different
hosting site or a different version control system, you will have to build
your Ubuntu package using more traditional means. That's okay, once you have
your debian/ directory, it'll be fairly easy (but not as easy as described
here). If you do use Bazaar, you'll just want to make sure you have the
bzr-builddeb plugin. Just do sudo apt-get install bzr-builddeb on
Ubuntu and you should get everything you need.
Okay, so now you …
Continue reading »