TedPage: Releasing an Indicator

In order to reduce the bus factor a bit, I'm documenting how I do releases for the indicators. This is not a complex process, but not obvious either. This post will involve lots of Bazaar, Launchpad and Ubuntu. If that's not what you want, skip to the next post in your reader.

Understanding Ubuntu Distributed Development

I think the first think to understand is what our goals are in making a set of releases. It's a set of artifacts to mark a point in the continual development of the software. But, that point is marked differently by different groups of people. Largely this is mirrored by the "trunks" of Ubuntu Distributed Development. It's important to realize that Bazaar doesn't really have the concept of "trunk" in it. We name a branch in Launchpad that to make it easier to communicate, but Bazaar doesn't treat that branch any differently than any other. So for the different audiences, they each have their own trunk, and releases there mean different things. The trunks we're using are:

• Upstream Trunk. Traditionally this is what software developers think of as trunk. The place where the artificats that are edited by the developers exist
• Upstream Import Trunk. This is a branch that is maintained by the UDD tools and represents the tarball releases. The tarballs are slightly different than upstream in that they contain additional files generated by the build.
• Upstream packaging. A packaging branch that is mostly a pure representation of upstream software, perhaps more up-to-date that distro. This allows us to do test builds and maintain updates for things like our Jenkins builds independent of having to ship the change to every Ubuntu user.
• Ubuntu packaging. A branch maintained by the Ubuntu Desktop team that represents what is sent to Ubuntu users.

As we move through the release process we'll go through these branches. Some will end up slightly obscured, but I'll try to point them out along the way.

Releasing Upstream

Building and shipping the upstream tarballs is mostly straight forward, but I want to take the time to be explicit and note a couple of tricks that are useful along the way. For our examples here I'm going to release indicator-messages version 1.2.3. First grab the trunk:

$ bzr branch lp:indicator-messages trunk

Then you can edit configure.ac to update the version and commit that. It's important to commit before distcheck since our ChangeLog is built dynamically at dist and you want the top of it to be the version number that is released.

$ vi configure.ac
$ bzr commit -m 1.2.3

Note, we're not tagging, and that's for good reason! Tags are a pain to move, and as releases go there's a reasonable chance that distcheck will fail, but let's check first:

$ ./autogen.sh
$ make distcheck

Distcheck will build the tarball, uncompress it and ensure it builds and can run the test suite. Most of the time this works so you can just:

$ bzr tag 1.2.3
$ bzr push

But, sometimes it doesn't. And you'll need to fix it. I'm going to leave fixing it as an exercise for the reader, but what I want to talk about is how that should look on the branch. You don't want your fix to occur after you set the version in the history. So you can step back by doing:

$ bzr uncommit
$ bzr shelve

This saves your changes. So then when you want to come back to releasing again

$ bzr unshelve
$ bzr commit -m 1.2.3

And you're back to where we left off. We need to put the tarball into Launchpad, and fortunately there's a handy script for that:

$ lp-project-upload indicator-messages 1.2.3 indicator-messages-1.2.3.tar.gz 1.2.4

An interesting thing here is the "1.2.4", where does that come from? It's creating the next milestone in Launchpad for you so that we can start assigning bugs that are Fix Committed to it and they're already there come release time. Also you'll need to take the bugs that are Fix Committed on the 1.2.3 release. You can do that with this script if there's a bunch, but if there's not too many just do it by hand.

Building a test package

We've built and released a tarball, and it passes all the tests, and all the code's been reviewed, but until you install it on your machine you're not putting any skin in the game. Let's really believe in that release. First we need to get the upstream branch. In this example we're assuming the Ubuntu release is precise because that's the current right now

$ bzr branch lp:~indicator-applet-developers/ubuntu/precise/indicator-messages/upstream upstream

You should at this point make sure that this branch is in sync with the Ubuntu Desktop branch. Frequently they'll make updates to the packaging or cherry pick revisions they're interested in from the developers. This isn't a necessary step to make everything work, but it makes the merge for them easier, so you should do it.

$ bzr merge lp:~ubuntu-desktop/indicator-messages/ubuntu
$ bzr commit -m "Sync from Ubuntu Desktop" 

We're now going to use the UDD toolchain to pull in the tarball and merge it into the packaging. I'm purposely providing two parameters here that are optional. The first is the tag that should be used to pull from. Chances are that if you just edited trunk the revision there would be exactly the same, but let's not take that chance. The second is the version where I'm just being more explicit about what we're doing.

$ bzr merge-upstream indicator-messages-1.2.3.tar.gz lp:indicator-messages -r tag:1.2.3 --version 1.2.3

At this point the tarball has been imported and merged into the current branch. One innocuous warning that you'll see in this step is about criss-cross merges. This usually happens when the desktop team cherry picks a branch. I have argued that this shouldn't be a warning and largely I regard this as Bazaar saying "if there are conflicts they're your fault," rarely are there ever conflicts from this.

We're now at the point where we start switching from the Bazaar-based tools to the Debian ones. We need to talk about what's changed in the Debian ChangeLog by using

$ dch -a

There will already be a section made for you buy the import, but you should continue to describe the upstream release with what has changed, specifically which bugs have been fixed. Also, I always change the version number from "1.2.3-0ubuntu1" to "1.2.3-0ubuntu1~ppa1". This is so that when I install the packages locally they'll get overridden by the ones that are built in the Ubuntu builders. Also it creates a unique namespace if I mess up that I can increment. You can then commit with the message you put into the changelog with

$ debcommit

A couple of quick things that you can check at this point. Debian packages have a way to include patches, we should have merged those upstream and if you leave them there they won't apply when building. So you can just remove them and comment as such

$ rm -rf debian/patches
$ dch -a
$ debcommit

Also you can check to ensure that you're shipping actually the tarball that you build, plus only the Debian packaging. All of the Debian packaging should be in the the debian/ directory so we can look to see if anything is not in that directory

$ bzr diff -r tag:upstream-1.2.3 | lsdiff

The astute reader will notice that I just used a tag that I didn't build. Hmm. This is where the Upstream Import branch comes into play. When we did the merge-upstream above we actually edited two branches. The first is the import branch where the clean tarball was placed. Then it was pulled into our packaging branch from that revision. I find it handy to look at the revision history at this point

$ bzr visualize

Now let's build ourselves a package. Usually if it builds I also tag it as a version in the revision history so that I can keep track of what I've installed. So to ensure I don't mess that up I chain the commands

$ bzr build-deb && dch -r && debcommit -r

You should now have packages in the directory above the current directory that you can install.

$ sudo dpkg -i *1.2.3-0ubuntu1~ppa1*deb

Now you can test on your machine if things are generally working.

Pushing downstream

If everything looks good you should push that branch back up and then propose a merge into the Ubuntu Desktop branch. This tells the Ubuntu Desktop team that we've got a package that we think works and they should consider for usage in Ubuntu. They'll run their own set of acceptance tests, and I'm sure they'll find you if they fail.