Applications under systemd

When we started to look at how to confine applications enough to allow an appstore that allows for anyone to upload applications we knew that Apparmor could do the filesystem and IPC restrictions, but we needed something to manage the processes. There are kernel features that work well for this, but we didn’t want to reinvent the management of them, and we realized that Upstart already did this for other services in the system. That drove us to decide to use Upstart for managing application processes as well. In order to have a higher level management and abstraction interface we started a small library called upstart-app-launch and we were off. Times change and so do init daemons, so we renamed the project ubuntu-app-launch expecting to move it to systemd eventually.

Now we’ve finally fully realized that transition and ubuntu-app-launch runs all applications and untrusted helpers as systemd services.

bye, bye, Upstart. Photo from: https://pixabay.com/en/goodbye-waving-boy-river-boat-705165/

For the most part, no one should notice anything different. Applications will start and stop in the same way. Even users of ubuntu-app-launch shouldn’t notice a large difference in how the library works. But for people tinkering with the system they will notice a few things. Probably the most obvious is that application log files are no longer in ~/.cache/upstart. Now the log files for applications are managed by journald, which as we get all the desktop services ported to use systemd, will mean that you can see integrated events from multiple processes. So if Unity8 is rejecting your connection you’ll be able to see that next to the error from your application. This should make debugging your applications easier. You’ll also be able to redirect messages off a device realtime, which will help debugging your application on a phone or tablet.

For those who are more interested in details we’re using systemd’s transient unit feature. This allows us to create the unit on the fly with multiple instances of each application. Under Upstart we used a job with instances for each application, but now that we’re taking on more typical desktop style applications we needed to be able to support multi-instance applications, which would have been hard to manage with that approach. We’re generating the service name using this pattern:

ubuntu-app-launch--$(application type)--$(application id)--$(time stamp).service

The time stamp is used to make a unique name for applications that are multi-instance. For applications that ask us to maintain a single instance for them the time stamp is not included.

Hopefully that’s enough information to get you started playing around with applications running under systemd. And if you don’t care to, you shouldn’t even notice this transition.

posted Mar 23, 2017 | permanent link

Presentations Updated

This post is mostly a mea culpa to all the folks that asked me after a presentation: “And those slides will be online?” The answer is generally “yes” but they were in a tweet or something equally as hard to find. But now I finally got to making an updated presentations page that is actually useful. Hopefully you can find the slides you are looking for there. And more importantly you can use them as a basis for your talk to a local group in your town.

As I was redoing this I thought it was a bit interesting how my title pages seem to alternate every couple of years between complex and simple. And I think I have a candidate for worst theme (though there was a close second). Also a favorite theme along with a reminder of all the fun it is to make a presentation with JessyInk.

I think that there are a couple missing that I can’t find, and also video links out on the Internet somewhere. Please drop me a line if you have any ideas, suggestions or I sent you files that I’ve now lost. Hopefully this is easier to maintain now so there won’t be the same delay.

posted Jan 16, 2017 | permanent link

The Case for Ubuntu Phone

There are times in standard social interactions where people ask what you do professionally, which means I end up talking about Ubuntu and specifically Ubuntu Phone. Many times that comes down to the seemingly simple question: “Why would I want an Ubuntu phone?” I’ve tried the answer “becasue I’m a thought leader and you should want to be like me,” but sadly that gets little traction outside of Silicon Valley. Another good answer is all the benefits of Free Software, but many of those are benefits the general public doesn’t yet realize they need.

Ubuntu Phone

The biggest strength and weakness of Ubuntu Phone is that it’s a device without an intrinsic set of services. If you buy an Android device you get Google Services. If you buy an iPhone you get Apple services. While these can be strengths (at least in Google’s case) they are effectively a lock in to services that may or may not meet your requirements. You certainly can get Telegram or Signal for either of those, but they’re never going to be as integrated as Hangouts or iMessage. This goes throughout the device including things like music and storage as well. Ubuntu and Canonical don’t provide those services, but instead provide integration points for any of them (including Apple and Google if they wanted) to work inside an Ubuntu Phone. This means as a user you can use the services you want on your device, if you love Hangouts and Apple Maps, Ubuntu Phone is happy to be a freak with you.

Carriers are also interested in this flexibility. They’re trying to put together packages of data and services that will sell, and fetch a premium price (effectively bundling). Some they may provide themselves and some by well known providers; but by not being able to select options for those base services they have less flexibility on what they can do. Sure, Google and Apple could give them a great price or bundle, but they both realize that they don’t have to. So that effectively makes it difficult for the carriers as well as alternate service providers (e.g. Dropbox, Spotify, etc) to compete.

What I find most interesting thing about this discussion is that it is the original reason that Google bought Android. They were concerned that with Apple controlling the smartphone market they’d be in a position to damage Google’s ability to compete in services. They were right. But instead of opening it up to competition (a competition that certainly at the time and even today they’re likely to win) they decided to lock down Android with their own services. So now we see in places like China where Google services are limited there is no way for Android to win, only forks that use a different set of integrations. One has to wonder if Ubuntu Phone existed earlier whether Google would have bought Android, while Ubuntu Phone competes with Android it doesn’t pose any threat to Google’s core businesses.

It is always a failure to try and convince people to change their patterns and devices just for the sake of change. Early adopters are people who enjoy that, but not the majority of people. This means that we need to be an order of magnitude better, which is a pretty high bar to set, but one I enjoy working towards. I think that Ubuntu Phone has the fundamental DNA to win in this race.

posted Jan 14, 2017 | permanent link

Snapping Unity8

For the last little while we’ve been working to snap up Unity8. This is all part of the conversion from a system image based device to one that is entirely based on snaps. For the Ubuntu Phones we basically had a package layout with a system image and then Click packages on top of it.

System Image with Clicks

Where Snaps change things is now all the various pieces of the core operation of the device can be confined and managed as individual bits. They can also be upgraded and rolled back on failure independently. And to get the benefits of that system we need to get Unity8 into a snap. For all the time that we spent harassing application developers about how they need to transform their apps so that they could work within the constraints of a Click, now we have to do that work ourselves. And it is definitely non-trivial. But we’re getting there.

Snaps for everything

All of the snaps needed to build a full system aren’t yet available so we created a small debian package that intergrates the Unity8 snap onto standard Ubuntu system. This package puts into place the configuration files for LightDM so that when a user selects the Unity8 Snap Session it calls into the unity8-session snap directly. This way Unity8 itself stays all together in the snap and we don’t have to worry about the other pieces of the core system that are also being developed at the same time. The package is available in the Stable Phone Overlay for Ubuntu 16.04 or in the Zesty archive and can be installed with apt install unity8-session-snap. It has a small helper script that will install the snaps you need that you can call as unity8-snap-install.

Right now we have the basic functionality mostly working. But there are a lot of warts. We’re working to remove all of them and turn the Unity8 snap into the best way to get the latest features of Unity8 in a safe manner.

Unity8 Snap Demo r309

I plan to post a bit more about what we’re doing and problems we’re solving along the way, even if I have some catching up to do. Trying to keep the posts small so I get them done.

posted Jan 5, 2017 | permanent link

Inkscape Snap

There is a new Inkscape release out! It is version 0.92 which has a lot of exciting features, my favorite being the Mesh Gradients. For this release I also worked on making a Snap package of Inkscape to make it easier for users to get the latest version.

The most exciting feature of Snaps for me as an upstream developer is how the Inkscape project can control which version users see to download on Ubuntu. Today, if someone was to open up the Software Center and install Inkscape they would get the 0.92.0 release. No other settings or configuration required. This breaks with the traditional Linux distribution model of the latest version only being available in development releases or by adding additional repositories, both of which are user hostile. For me, this is an exciting change that will hopefully make the Linux client more approachable and maintainable to a larger range of users. That’s the why, let’s talk a bit about how.

The Inkscape snap is built from a Snapcraft configuration file that is stored in the main repository (or in this case the 0.92.x branch). The configuration file tells Snapcraft which packages we want to pull in from the Ubuntu archives as dependencies and what parameters to call CMake with. Launchpad is then configured to build a snap on every commit to that branch and upload it to the Ubuntu Store in the candidate channel. When we decide the release is ready the Inkscape project can then promote the binary to the stable channel and all users will get updated to that version.

We’re also using the edge channel for our GTK+3 experimental builds on trunk. So if you’d like to start playing and testing those you can switch your Inkscape install by doing:

$ snap refresh --edge inkscape

Which is exciting, but can also break often. Hopefully this will allow users who are interested in testing the bleeding edge builds without having to build their own versions of Inkscape.

I’m excited about the way that Snaps bring a direction connection from upstreams to users in a way that hasn’t existed on the Free Software Desktop to date. No complication but provided in a safe, recreatable manner without configuring encryption keys and special archives. We’re living in the future.

posted Jan 4, 2017 | permanent link

Migrating to Jekyll

My blog has been running really old software. The software that we always say that people should not be using because it is not maintained or developed and probably has a million bugs. Of course, we would never do that ourselves.

With some cleaning and hopes of perhaps blogging a bit more I am happy to say that I have migrated my blog over to Jekyll. I like the concept of static generation, it matches how I want to maintain and worry about my sites, especially my blog.

This is a test post to see if everything actually works.

posted Jan 2, 2017 | permanent link

Creating a QML snap with Snapcraft

We want to enable all kinds of developers to quickly make applications and devices using Snappy as their basis. A quick way to make compelling user interfaces is by using QML, so it seemed like a natural fit to get QML working in snapcraft to eliminate complex setups, and just get things working. There is an Introduction to Snapcraft, I'm going to assume you've already read that.

To get started with an interesting demo I went and stole the Qt photoviewer demo and pulled it into its' own repository, then added a couple simple configuration files. This is a great demo because it is graphical and fun, but also shows pulling data from the network as all the photos are based on Flickr tags.

parts:
  qml:
    plugin: qml
  photoviewer:
    plugin: copy
    files:
      main.qml: main.qml
      PhotoViewerCore: PhotoViewerCore
snappy-metadata: meta

The snapcraft.yaml file includes two parts. The first part is the QML plug in which includes all the pieces needed to run QML programs from the Ubuntu archive. The second is the copy plugin which copies our QML files into the snap. We don't have a build system in this example so copy is all we need, more complex examples could use the cmake or autotools plugins instead.

The last item in the snapcraft.yaml tells Snapcraft where to find the packaging information for Snappy. In the meta directory we have a packages.yaml that is a standard Snappy package file.

name: photoviewer
version: 0.1
vendor: Ted Gould <ted@canonical.com>
frameworks: [mir]
binaries:
  - name: photoviewer
    exec: qmlscene main.qml --
    caps:
      - mir_client
      - network-client

It configures a binary that will be set up by Snappy, which is simply a call to qmlscene with our base QML file. This will then get wrapped up into a single binary in /apps/bin that we can execute.

We need to now turn this directory into a snap. You should follow the instructions to install snapcraft, and then you can just call it in that directory:

$ snapcraft

There are a few ways to set up a Snappy system, the one that I've used here is with QEMU on my development system. That makes it easy to develop and test with, and currently the Mir snap is only available for amd64. After getting snappy setup you'll need to grab the Mir framework from the store and install the snap we just built.

$ sudo snappy install mir
$ sudo snappy install --allow-unauthenticated photoviewer_1.0_amd64.snap

You can then run the photoviewer:

$ photoviewer.photoviewer

And you should have something like this on your display:

While this is a simple demo of what can be done with QML, it can be expanded to enable all kinds of devices from displaying information on a network service or providing UI for a small IoT device.

posted Aug 6, 2015 | permanent link

Just Say It!

While I love typing on small on screen keyboards on my phone, it is much easier to just talk. When we did the HUD we added speech recognition there, and it processed the audio on the device giving the great experience of controlling your phone with your voice. And that worked well with the limited command set exported by the application, but to do generic voice, today, that requires more processing power than a phone can reasonably provide. Which made me pretty excited to find out about HP's IDOL on Demand service.

I made a small application for Ubuntu Phone that records the audio you speak at it, and sends it up to the HP IDOL on Demand service. The HP service then does the speech recognition on it and returns the text back to us. Once I have the text (with help from Ken VanDine) I set it up to use Content Hub to export the text to any other application that can receive it. This way you can use speech recognition to write your Telegram notes, without Telegram having to know anything about speech at all.

The application is called Just Say It! and is in the Ubuntu App Store right now. It isn't beautiful, but definitely shows what can be done with this type of technology today. I hope to make it prettier and add additional features in the future. If you'd like to see how I did it you can look at the source.

As an aside: I can't get any of the non-English languages to work. This could be because I'm not a native speaker of those languages. If people could try them I'd love to know if they're useful.


posted Jun 28, 2015 | permanent link

Role change: Now snappier

Happy to announce that I'm changing roles at Canonical, moving down the stack to join the Snappy team. It is in some ways a formalization of my most recent work which has been more on application lifecycle and containment than higher level stuff like indicators and other user services. I'll be working on the core snappy team to ensuring that snappy works for a wide variety of use cases, from small sensors embedded in your world to phones to services running in the cloud. For me Snappy formalizes a lot of trends that we're seeing all over computing today so I'm excited to get more involved with it.

To kick things off I'll be working on making Snaps easier to build and maintain using the native dependency systems that exist already for most languages. The beautiful part about bundling is that we no longer have to force our dependency system on others, they can choose what works best for them. But, we still need to make integrating with it easy.

New adventures bringing new challenges are where I like to roam. I'll still be around though, and might even contribute a patch or two to some of my old haunts.

posted Jun 9, 2015 | permanent link

Tracking Usage

One of the long standing goals of Unity has been to provide an application focused presentation of the desktop. Under X11 this proves tricky as anyone can connect into X and doesn't necessarily have to give information on what applications they're associated with. So we wrote BAMF, which does a pretty good job of matching windows to applications, but it could never be perfect because there simply wasn't enough information available. When we started to rethink the world assuming a non-X11 display server we knew there was one thing we really wanted, to never ever have something like BAMF again.

This meant designing, from startup to shutdown, a complete tracking of an application before it started creating windows in the display server. We then were able to use the same mechanisms to create a consistent and secure environment for the applications. This is both good for developers and users as their applications start in a predictable way each and every time it's started. And we also setup the per-application AppArmor confinement that the application lives in.

Enough backstory, what's really important to this blog post is that we also get an event when an application starts and stops which is a reliable event. So I wrote a little tool that takes those events out of the log and presents them as usage data. It is cleverly called:

$ ubuntu-app-usage

And it presents a list of all the applications that you've used on the system along with how long you've used them. How long do you spend messing around on the web? Now you know. You're welcome.

It's not perfect in that it uses all the time that you've used the device, it'd be nice to query the last week or the last year to see that data as well. Perhaps even a percentage of time. I might add those little things in the future, if you're interested you can beat me too it.

posted Nov 14, 2014 | permanent link

All the older posts...