24 hours a day, 7 days a week, 365 days per year...

October 22, 2017

retro-gtk: The Future, Marty!

Let's come back to retro-gtk. In the previous articles I explained how bad retro-gtk was, what I did to start improving it and more importantly what I did to prepare the terrain for further development. This article will detail the aforementioned planed improvements!

Unless stated otherwise, I don't plan these changes to be part of retro-gtk 0.14 and I have no idea when they will be implemented. If I say anything you know to be wrong or if you know something which could help the library's development, please share it in the comments!

Stabilization of the API

As stated in the previous article, I want retro-gtk's API to stop breaking as much as it did in the past. Starting with 0.14, we will avoid API and ABI breaks as much as possible, and if we do any we will document them properly. The API should be stable but given that some big changes are coming I don't feel comfortable promising proper stability just yet.


I requested to move retro-gtk to GNOME's GitLab. This would allow retro-gtk to stop parasiting gnome-games' Bugzilla and live its own life. This is particularly important as retro-gtk is expected to become more stable and hence more used. I hope it will be done before 0.14 get released.

Meson! 😃

I started porting retro-gtk's build system to Meson, nothing is on master yet and I am studying Meson as I do the port. I hope it will be done before 0.14 get released.

Y U NO Rust‽

The library has two major constraints: it has to use the low-level Libretro C API and it has to expose an introspected GObject C API. As far as I know Rust supports both these constraints, but I don't know Rust yet and the Vala code needed to be ported to some low-level language so I could develop the library more freely and make it move forward.

Rust support of GObject — even though it is very promising — is still in development, so porting retro-gtk to it while I had to learn the language would have delayed the needed low-level language port. I may port parts of the library to Rust later, once I took the time to learn it and once the GObject integration matured. I expect a C to Rust transition to be less painful while maintaining the ABI than the Vala to C was.

Hardware Rendering and Shaders

Currently retro-gtk renders the video output of the core by converting it with custom code and drawing it with Cairo. It was quick and easy to implement with the knowledge I had back then and it did the job so I let it that way so far, even though it's very clearly far from optimal. retro-gtk should instead use hardware acceleration to render the video.

The only problem is that I still know nothing about graphics APIs such as OpenGL. 🙂 I hope the learning curve won't be too steep, if you have any knowledge to share, please do! I'll rely on Emanuele's GtkGLArea presentation and Anton's OpenGL 4 Tutorials to start working.

This would allow retro-gtk to render the cores' video with shaders, such as CRT shaders, Game Boy shaders and more!

I plan to work on this during SUSE Hack Week 16, so it may land in 0.14! 🙂

Hardware Rendering for the Cores

Libretro defines RETRO_ENVIRONMENT_SET_HW_RENDER and struct retro_hw_render_callback to allow the cores to request a hardware rendering interface to the frontend. Some Libretro cores can't work without it, so retro-gtk should implement it.

This would allow retro-gtk (and by extension Games) to run Nintendo 64 emulators and many more Libretro cores it can't yet!

I also plan to work on it during SUSE Hack Week 16.

Gamepad Handling

retro-gtk handles some controllers internally via RetroCoreView, but it doesn't handle gamepads directly and rely on the application to implement gamepad objects. I am currently exporting the gamepad handling code of Games into its own library, it may be a good opportunity to let retro-gtk handle all its controllers — gamepad included — internally, avoiding to expose too much of its internals. I am not sure whether it is a good idea or not yet.

Run the Core in a Subproccess

This is by far the biggest change planned for retro-gtk, I have absolutely no idea when it will be implemented: I would like retro-gtk to run Libretro cores into their own subprocesses. I would like this to not change the API — or as little as possible — which is made easier by the new higher level API as it reduces the surface the IPC must cover, moving more of the code in the subprocess. Here are the effects I expect such a change to have.

Calling Back Core Support Improvements

As each core would be alone in its subprocess we can be sure of which core is calling back, so there is no need to track its identity, making retro-gtk able to handle Libretro cores calling back from a different thread. As a result the API is expected to be more asynchronous.

Avoid Static Memory Collisions

When a module is loaded twice in a process it is actually loaded just once, making the two instances sharing the same static memory space. Most Libretro cores where once standalone games or applications, hence they were free to use static variables and sometimes did so. Some of this is still present in some Libretro cores, making them buggy when two instances try to access the same static memory. retro-gtk avoid these clashes in a very dirty way: the module is copied to a temporary file which is loaded instead, tricking the loading mechanism in actually loading is twice… I hope that loading each module from a different process will solve that, but I really have no idea if it will: if you know anything about this please contact me.

Improved Stability

We could make retro-gtk resilient to crashes of the subprocess, whether it is caused by the Libretro core or the lower layers of retro-gtk, the applications could handle this as they want for example by kindly notifying the users. I hope it won't make debugging the cores harder and that meaningful backtraces and other useful debugging information will remain obtainable.

Improved Security?

libseccomp could be used in the subprocess to avoid the cores to run unauthorized syscalls, which could be used to mitigate exploits and other 0days. The problem is that we can't know which syscalls the Libretro cores need so it may be tricky to filter enough but not too much. It still needs some reflection.

Regarding Performances

On one hand, performances could get improved as the UI and the Libretro core would run on two different CPU cores at the same time. On the other hand, high performance IPCs are tricky to get right, especially as I have next to no experience in that field. Copies and locks should be avoided as much as possible but I don't expect to avoid them all. This strategy turned out well for web browsers so I can only hope it will here too. 🙂

Unit Testing, Code Coverage ans Non-Regression Tests

Because we should do all of that, period. Special Libretro cores may have to be crafted for testing, but we should use already existing ones as much as possible to avoid accidentally writing ones not following precisely Libretro's semantic and working only in retro-gtk.

Core Integration Tests and Non-Regression Tests

It would be really useful for retro-gtk to distribute a Libretro core integration testing tool. Games would no doubt benefit from it as we already encountered regressions in Libretro cores we ship with its Flatpak version, which is particularly bad when it causes your user's saves to be lost.


This article concludes the mini-series about retro-gtk. I hope this library to become the best way to build a GTK+ based Libretro frontend, and if all of these features get implemented I am sure it will be! 😊

October 21, 2017

[Problem3 SPOJ:ROBOTGRI] A problem for lovers of mazes!


This was a tough nut!

Its a graph problem. Its actually a combination of 2 problems in one! I have voted it as hard and given it my recommendation.

I went through the following resources (none has the actual solution but all serve as hints) to finally come up with a solution for it:

Tested and Accepted:,chandniverma/

Time complexity in the worst case: O(n^2 + E)
where n = number of rows (or columns) in the grid
and E = number of edges in the connected graph containing the starting cell 'S'.

Space complexity: O(n^2)

Rendering HTML5 video in Servo with GStreamer

At the Web Engines Hackfest in A Coruña at the beginning of October 2017, I was working on adding some proof-of-concept code to Servo to render HTML5 videos with GStreamer. For the impatient, the results can be seen in this video here

And the code can be found here and here.


Servo is Mozilla‘s experimental browser engine written in Rust, optimized for high-performance, parallelized rendering. Some of the parts of Servo are being merged in Firefox as part of the Project Quantum, and already provide a lot of performance and stability improvements there.

During the hackfest I actually spent most of the time trying to wrap my head around the huge Servo codebase. It seems very well-structured and designed, exactly what you would expect from starting such a project from scratch by a company that has decades of experience writing browser engines already. After also having worked on WebKit in the past, I would say that you can see the difference of a legacy codebase from the end of the 90s and something written in a modern language with modern software engineering practices.

To the actual implementation of HTML5 video rendering via GStreamer, I actually started on top of the initial implementation that Philippe Normand started before already. That one was rendering the video in a separate window though, and did not work with the latest version of Servo anymore. I cleaned it up and made it work again (probably the best task you can do to learn a new codebase), and then added support for actually rendering the video inside the web view.

This required quite a few additions on the Servo side, some of which are probably more hacks than anything else, but from the GStreamer-side is was extremely simple. In Servo currently all the infrastructure for media rendering is still missing, while GStreamer has more than a decade of polishing for making integration into other software as easy as possible.

All the GStreamer code was written with the GStreamer Rust bindings, containing not a single line of unsafe code.

As you can see from the above video, the results work quite well already. Media controls or anything more fancy are not working though. Also rendering is currently done completely in software, and a RGBA frame is then uploaded via OpenGL to the GPU for rendering. However, hardware codecs can already be used just fine, and basically every media format out there is supported.


While this all might sound great, unfortunately Mozilla’s plans for media support in Servo are different. They’re planning to use the C++ Firefox/Gecko media backend instead of GStreamer. Best to ask them for reasons, I would probably not repeat them correctly.

Nonetheless, I’ll try to keep the changes updated with latest Servo and once they add more things for media support themselves add the corresponding GStreamer implementations in my branch. It still provides value for both showing that GStreamer is very well capable of handling web use cases (which it already showed in WebKit), as well as being a possibly better choice for people trying to use Servo on embedded systems or with hardware codecs in general. But as I’ll have to work based on what they do, I’m not going to add anything fundamentally new myself at this point as I would have to rewrite it around whatever they decide for the implementation of it anyway.

Also once that part is there, having GStreamer directly render to an OpenGL texture would be added, which would allow direct rendering with hardware codecs to the screen without having the CPU worry about all the raw video data.

But for now, it’s waiting until they catch up with the Firefox/Gecko media backend.

2017-10-21 Saturday.

  • Up in the night with horrible neck pain; hmm, poked at an online bug in a curmudgeonly way - found a cut/paste coding horror & tried to fix it.

2017-10-20 Friday.

  • Mail chew; marketing / analytics training much of the day. Managed to get some admin done too.

Lightroom exporter/importer

Importing from legacy proprietary software is always important to liberate one's data and help migrate to libre solutions. To that effect, I wrote, in Rust, lrcat-extractor. A library (crate in Rust parlance) with a command line tool to extract and dump Lightroom catalogs, the central part of the user generated data.

Currently only the 2-version-ago Lightroom 4 catalogs are supported as it is the only version I have. Supporting Lightroom 6 is just a matter of me purchasing it and spending some time.

The goal is of course to use this crate in Niepce to import Lightroom 4 catalogs directly.

Head to the lrcat-extractor repository if you are curious.

Also as aside note, I also have the equivalent for the now discontinued Apple Aperture that I started to write almost two years ago as a practical exercise to learn Rust, and that I used as a base. Check out aplib-extractor.

The next step is to implement the actual import, probably filling the gaps in the API.

Update: and before someone says it, the timing of this project is quite coincidental.

October 20, 2017

Fedora meets RHEL

As we enter the final freeze before the Fedora 27 Workstation release, I’d like to highlight a new feature that will hopefully make Fedora more attractive for developers. Last month, I had written about our experiments to make it easier to consume gratis, self-supported Red Hat Enterprise Linux installations from Fedora Workstation. I am happy to report that this is now a reality.


Starting from Fedora 27 Workstation, you’ll be able to install an infinite number of RHEL 7.x VMs right from inside GNOME Boxes. All you need is an account on, and it will automatically set up a RHEL virtual machine that’s entitled to the RHEL Developer Suite subscription.


Thanks to Felipe Borges for a seemingly endless round of patch reviews, and Fabiano and Victor for occasionally lending us their brain.

DASH trick-mode playback in GStreamer: Fast-forward/rewind without saturating your network and CPU

GStreamer now has support for I-frame-only (aka keyframe) trick mode playback of DASH streams. It works only on DASH streams with ISOBMFF (aka MP4) fragments, and only if these contain all the required information. This is something I wanted to blog about since many months already, and it’s even included in the GStreamer 1.10 release already.

When trying to play back a DASH stream with rates that are much higher than real-time (say 32x), or playing the streams in reverse, you can easily run into various problems. This is something that was already supported by GStreamer in older versions, for both DASH streams as well as local files or HLS streams but it’s far from ideal. What would happen is that you usually run out of available network bandwidth (you need to be able to download the stream 32x faster than usual), or out of CPU/GPU resources (it needs to be decoded 32x faster than usual) and even if all that works, there’s no point in displaying 960 (30fps at 32x) frames per second.

To get around that, GStreamer 1.10 can now (if explicitly requested with GST_SEEK_FLAG_TRICKMODE_KEY_UNITS) only download and decode I-frames. Depending on the distance of I-frames in the stream and the selected playback speed, this looks more or less smooth. Also depending on that, this might still yield to many frames to be downloaded or decoded in real-time, so GStreamer also measures the distance between I-frames, how fast data can be downloaded and whether decoders and sinks can catch up to decide whether to skip over a couple of I-frames and maybe only download every third I-frame.

If you want to test this, grab the playback-test from GStreamer, select the trickmode key-units mode, and seek in a DASH stream while providing a higher positive or negative (reverse) playback rate.

Let us know if you run into any problems with any specific streams!

Short Implementation Overview

From an implementation point of view this works by having the DASH element in GStreamer (dashdemux) not only download the ISOBMFF fragments but also parses the headers of each to get the positions and distances of each I-frame in the fragment. Based on that it then decides which ones to download or whether to skip ahead one or more fragments. The ISOBMFF headers are then passed to the MP4 demuxer (qtdemux), followed by discontinuous buffers that only contain the actual I-frames and nothing else. While this sounds rather simple from an high-level point of view, getting this all right in the details was the result of a couple of months of work by Edward Hervey and myself.

Currently the heuristics for deciding which I-frames to download and how much to skip ahead are rather minimal, but it’s working fine in many situations already. A lot of tuning can still be done though, and some streams are working less well than others which can also be improved.

Porting to Rust

I have started something crazy a few weeks ago: porting my app from C++ to Rust.

I'll talk about it here, in several parts.

This is Part 1.


Once upon a time I started to write a Free Software photography application for GNOME. That was around 10 years ago.

At the time I chose to use C++ and Gtkmm 2.x. C++ because it provide unprecedented interoperability with existing C code while offering a stronger type system, RAII and template which allow safer programing pattern without a large overhead at runtime. Absolutely no regrets on that.

Over the years, despite never having been released, the code base was updated to C++11 and Gtkm 3.x.

Meanwhile Rust was developed at Mozilla. And Rust provide safety and high performance, but doesn't have the same ease of interoperability with a C codebase - albeit still make it easy.


Why am I porting to Rust?

  • Rust is designed to be a safe language. In this day an age of 0day due to the use of C (and C++) and their unsafe practice, it is something important.
  • I want the application to be multi-threaded (it already is in design), which is one of the design goal of Rust.
  • I started writing other stand alone components in Rust.

The goal is to progressively rewrite the application in Rust, mostly translating the logic from C++ to Rust. It is not the first time I do that across languages. I'll go from the back to the front, the front being the UI written with Gtkmm.

Several steps.

Build system

There have already been a few posts about integrating Rust into an automake based build system, by Federico or by me. The gist of it is to make sure you build your library with Rust and link it to your application.

Building Rust code is trivial using cargo. There is no point using anything else.


When doing an ad-hoc conversion, the biggest problem is calling code write in one language from the other.

You can not call Rust methods or generic functions from C/C++, so you'll need to have some interface in place. Rust has made it relatively easy by using C calling convention and making easy to declare a function with mangling disabled.


pub extern fn my_function(arg1: &Type1) -> RetType {


From C/C++ this would be something like:

extern "C" RetType my_function(const Type1* arg1);

Here we assume that RetType is a POD (Plain Old Datatype, like int, char, float), not a C++ class, nor a struct in Rust.

There is a tool called cbindgen that will generate C/C++ header from the Rust code. You can call it from the Rust build system to do that automatically. More on that later.

Calling C code from Rust is easy provided you have the proper interfaces. It is basically the reverse of the above.

But calling C++ methods on the other hand, you have to use bindgen.


Bindgen will generate from C/C++ header Rust modules to call C++ code.

The tool is pretty amazing to not say magical. Albeit there are some pitfalls.

For example a C++ class with a std::vector<> as a member will a problem if treated as opaque type. To be honest I am not sure if it would work as a non opaque type.

The use of boost might actually generate Rust code that doesn't compile.

Running the Rust test with cargo test will fail. I think it is Issue 1018.

There are a few things I recommend. Most notably, treat types as opaque whenever possible ; whitelist types and functions rather than blacklist. Your mileage may vary but this is what I got to work in my case.

Using these tools I managed to actually rewrite the large parts of the backend in Rust with equal functionality. Progressively more C++ code will be replaced.

I'll get into detail in a future post.

October 19, 2017

Looking back at Fedora Workstation so far

So I have over the last few years blogged regularly about upcoming features in Fedora Workstation. Well I thought as we putting the finishing touches on Fedora Workstation 27 I should try to look back at everything we have achieved since Fedora Workstation was launched with Fedora 21. The efforts I highlight here are efforts where we have done significant or most development. There are of course a lot of other big changes that has happened over the last few years by the wider community that we leveraged and offer in Fedora Workstation, examples here include things like Meson and Rust. This post is not about those, but that said I do want to write a post just talking about the achievements of the wider community at some point, because they are very important and crucial too. And along the same line this post will not be speaking about the large number of improvements and bugfixes that we contributed to a long list of projects, like to GNOME itself. This blog is about taking stock and taking some pride in what we achieved so far and major hurdles we past on our way to improving the Linux desktop experience.
This blog is also slightly different from my normal format as I will not call out individual developers by name as I usually do, instead I will focus on this being a totality and thus just say ‘we’.

  • Wayland – We been the biggest contributor since we joined the effort and have taken the lead on putting in place all the pieces needed for actually using it on a desktop, including starting to ship it as our primary offering in Fedora Workstation 25. This includes putting a lot of effort into ensuring that XWayland works smoothly to ensure full legacy application support.
  • Libinput – A new library we created for handling all input under both X and Wayland. This came about due to needing input handling that was not tied to X due to Wayland, but it has even improved input handling for X itself. Libinput is being rapidly developed and improved, with 1.9 coming out just a few days ago.
  • glvnd – Dealing with multiple OpenGL implementations have been a pain under Linux for years. We worked with NVidia on this effort to ensure that you can install multiple OpenGL implementations on the system and have your system be able to use the correct one depending on which GPU and driver you are using. We keep expanding on this solution to cover more usecases, so for Fedora Workstation 27 we expect to bring glvnd support to XWayland for instance.
  • Porting Firefox to GTK3 – We ported Firefox to GTK3, including making sure it works under Wayland. This work also provided the foundation for HiDPI support in Firefox. We are the single biggest contributor to Firefox Linux support.
  • Porting LibreOffice to GTK3 – We ported LibreOffice to GTK3, which included Wayland support, touch support and HiDPI support. Our team is one of the major contributors to LibreOffice and help the project forward on a lot of fronts.
  • Google Drive integration – We extended the general Google integration in GNOME 3 to include support for Google Drive as we found that a lot of our users where relying on Google Apps at their work.
  • Flatpak – We created Flatpak to lead the way in moving desktop applications into their own namespaces and containers, resolving a lot of long term challenges for desktop applications on Linux. We expect to have new infrastructure in place in Fedora soon to allow Fedora packagers to quickly and easily turn their applications into Flatpaks.
  • Linux Firmware Service – We created the Linux Firmware service to provide a way for Linux users to get easy access to UEFI firmware on their linux system and worked with great vendors such as Dell and Logitech to get them to support it for their devices. Many bugs experienced by Linux users over the years could have been resolved by firmware updates, but with tooling being spotty many Linux users where not even aware that there was fixes available.
  • GNOME Software – We created GNOME Software to give us a proper Software Store on Fedora and extended it over time to include features such as fonts, GStreamer plugins, GNOME Shell extensions and UEFI firmware updates. Today it is the main Store type application used not just by us, but our work has been adopted by other major distributions too.
  • mp3, ac3 and aac support – We have spent a lot of time to be able to bring support for some of the major audio codecs to Fedora like MP3, AC3 and AAC. In the age of streaming supporting codecs is maybe of less importance than it used to be, but there is still a lot of media on peoples computers they need and want access to.
  • Fedora Media Creator – Cross platform media creator making it very easy to create Fedora Workstation install media regardless of if you are on Windows, Mac or Linux. As we move away from optical media offering ISO downloads started feeling more and more outdated, with the media creator we have given a uniform user experience to quickly create your USB install media, especially important for new users coming in from Windows and Mac environments.
  • Captive portal – We added support for captive portals in Network Manager and GNOME 3, ensuring easy access to the internet over public wifi networks. This feature has been with us for a few years now, but it is still a much appreciated addition.
  • HiDPI support – We worked to add support for HiDPI across X, Wayland, GTK3 and GNOME3. We lead the way on HiDPI support under Linux and keep working on various applications to this date to polish up the support.
  • Touch support – We worked to add support for touchscreens across X, Wayland, GTK3 and GNOME3. We spent significant resources enabling this, both on laptop touchscreens, but also to support modern wacom devices.
  • QGNOME Platform – We created the QGNOME Platform to ensure that Qt applications work well under GNOME3 and gives a nice native and integrated feel. So while we ship GNOME as our desktop offering we want Qt applications to work well and feel native. This is an ongoing effort, but for many important applications it already is a great improvement.
  • Nautilus improvements. Nautilus had been undermaintained for quite a while so we had Carlos Soriano spend significant time on reworking major parts of it and adding new features like renaming multiple files at ones, updating the views and in general bring it up to date.
  • Night light support in GNOME – We added support for automatic adjusting the color and light settings on your system based on light sensors found in modern laptops. This integrated functionality that you before had to install extra software like Red Shift to enable.
  • libratbag – We created a library that enable easy configuration of high end mice and other kind of input devices. This has led to increased collaboration with a lot of gaming mice manufacturers to ensure full support for their devices under Linux.
  • RADV – We created a full open source Vulkan implementation for ADM GPUs which recently got certified as Vulkan compliant. We wanted to give open source Vulkan a boost, so we created the RADV project, which now has an active community around it and is being tested with major games.
  • GNOME Shell performance improvements – We been working on various performance improvements to GNOME Shell over the last few years, with significant improvements having happened. We want to push the envelope on this further though and are planning a major performance hackfest around Shell performance and resource usage early next year.
  • GNOME terminal developer improvements – We worked to improve the features of GNOME Terminal to make it an even better tool for developers with items such as easier naming of terminals and notifications for long running jobs.
  • GNOME Builder – Improving the developer story is crucial for us and we been doing a lot of work to make GNOME Builder a great tool for developer to use to both improve the desktop itself, but also development in general.
  • Pipewire – We created a new media server to unify audio, pro-audio and video. First version which we are shipping in Fedora 27 to handle our video capture.
  • Fleet Commander – We launched Fleet Commander our new tool for managing large Linux desktop deployments. This answer a long standing call from many of Red Hats major desktop customers and many admins of large scale linux deployments at Universities and similar for a powerful yet easy to use administration tool for large desktop deployments.

I am sure I missed something, but this is at least a decent list of Fedora Workstation highlights for the last few years. Next onto working on my Fedora Workstation 27 blogpost :)

List of GNOME-related projects fundraisers

I think it’s useful to have a list of projects fundraisers in GNOME or at least GNOME-related. Ideally it would be nice to have that list on the website, it looks to me an obvious thing to do, but after a discussion on the GNOME foundation-list, it seems unlikely to happen anytime soon.

So I’ve created this wiki page in the meantime. It also explains the difference with donations made to the GNOME Foundation.

The list includes the GtkSourceView fundraiser that I launched last month. I plan to write regular updates on that front on this blog, for example every two months. Stay tuned, and thanks for your support :-)

October 18, 2017

Fleet Commander ready for takeoff!

Alberto Ruiz just announced Fleet Commander as production ready! Fleet Commander is our new tool for managing large deployments of Fedora Workstation and RHEL desktop systems. So get our to Albertos Fleet Commander blog post for all the details.

Ubuntu GNOME Shell in Artful: Day 16

All good things must come to an end, however, in that particular case, it’s rather a beginning! We are indeed almost done in our road to Artful, which means that 17.10 is just around the corner: official Ubuntu 17.10 release is due tomorrow. Of course, it doesn’t mean we stop right away working on it: you will have bug fixes and security updates for 9 months of support! It’s thus time to close this series on Artful, and for this, we are going to tackle one topic we didn’t get to yet, which is quite important approaching the release: upgrading from a previous Ubuntu release! For more background on our current transition to GNOME Shell in artful, you can refer back to our decisions regarding our default session experience as discussed in my blog post.

Day 16: Flavors, upgrades and sessions!

Different kind of sessions

Any new Ubuntu installation will have two sessions available at most, whose base name is “Ubuntu”:

  • The “Ubuntu” session corresponds to GNOME Shell experience with our modifications (Ubuntu Dock, appindicator support, our theme, small behavior changes…). You have probably seen those and followed their development on previous blog posts. This is the default session running under Wayland.
  • The “Ubuntu on Xorg” session, being similar to the previous one, but running on Xorg as the name indicates :) Users who can’t run Wayland (using nvidia proprietary driver or unsupported hardware) should be automatically fallbacked and only presented with that session.

Ubuntu default installation sessions

Those two sessions are available when you install the ubuntu-session package.

However, more sessions are available in the Ubuntu archives around GNOME technologies: the Unity and vanilla GNOME ones. The first one is available as soon as you install the unity-session binary package. The vanilla GNOME sessions simply appears once gnome-session is installed. After a reboot, GDM is presenting all of them for selection when login in.

All available sessions

Let’s see how that goes on upgrades.

Upgrading from Ubuntu 17.04 or Ubuntu 16.04 (LTS)

People running today Ubuntu 17.04 or our last LTS, Ubuntu 16.04, are generally using Unity. As with every releases when upgrading people being on one default, we upgrade them to the next new default. It means that on upgrades, those users will reboot in our default Ubuntu GNOME Shell experience, having the “Ubuntu” and “Ubuntu on Xorg” sessions available. The “Ubuntu” session is the default and will lead you to our default and fully supported desktop:

Ubuntu GNOME Shell on 17.10

However, we don’t remove packages that are still available in the distribution on upgrades. Those users will thus have an additional “Unity” session option, which they can use to continue running Unity 7 (and thus, on Xorg only). Indeed, Unity is still present, in universe (meaning we don’t commit to strong maintenance or security updates), but we will continue to have a look at it on a best effort bases (at least until our next LTS). Some features are slightly modified to either don’t collide with GNOME Shell experience or to follow more the upstream philosophy, like the headerbars I mentioned in my previous blog post. In a nutshell, don’t expect the exact same experience that you used to have, but you will reach similar familiarity for the main concepts and components.

Unity on 17.10

Upgrading from Ubuntu GNOME 17.04 or Ubuntu GNOME 16.04

Those people were experiencing a more vanilla upstream GNOME experience than our Ubuntu session. It was a little bit 5050 in what to do for those users on upgrades as they were used to something different. The final experience that Ubuntu GNOME users will get is to have those 2 upstream vanilla GNOME session, (”GNOME” and “GNOME on Xorg”). Those will stay the default after upgrades.

Vanilla GNOME Shell session on 17.10

In addition to those sessions, we still want to give an easy option for our users to try our new default experience, and thus, the 2 “Ubuntu” sessions (Wayland & Xorg) are automatically installed as well on upgrade. The sessions are just around for user’s convenience. :)


I want to quickly mention and give kudos to Olivier who fixed a pet bug of mine, to ensure that Wayland to Xorg automatic fallbacking will always fallback to the correct sessions (Ubuntu will fallback to Ubuntu on Xorg and GNOME to GNOME on Xorg). His patches were discussed upstream and are now committed in the gdm tree. This will be quickly available as a stable release update conveniently as only impacting upgrades.

In a nutshell

To sum all that up:

  • New installs will have the “Ubuntu and Ubuntu on Xorg” options. Ubuntu under wayland being the default.
  • Upgrades from Ubuntu 16.04 and 17.04 will get Ubuntu (default), Ubuntu on Xorg and Unity sessions installed.
  • Upgrades from Ubuntu GNOME 16.04 and 17.04 will get GNOME (default), GNOME on Xorg, Ubuntu and Ubuntu on Xorg sessions installed.
  • Fallbacking when Wayland is not supported will fallback to the corresponding Xorg session.

And this is it for our “road to Artful” blog post long series! I hope you had as much fun reading it as I had writing and detailing the work done by the Ubuntu desktop team to make this transition, we hope, a success. It was really great as well to be able to interact and answers to the many comments that you posted on the dedicated section. Thanks to everyone participating there.

You can comment on the community HUB and participate and contribute from there! We will likely redo the same experiment and keep you posted on our technical advancement for the Ubuntu 18.04 LTS release. You should expect fewer posts as of course as the changes shouldn’t be so drastic as they were this cycle. We will mostly focus on stabilizing, bug fixes and general polish!

Until then, enjoy the upcoming Ubuntu 17.10 release, watch the website for the release announcement on desktop, servers, flavors, iot and clouds, join our community HUB… and see you soon around! :)


Automatic LUKS volumes unlocking using a TPM2 chip

I joined Red Hat a few months ago, and have been working on improving the Trusted Platform Module 2.0 (TPM2) tooling, towards having a better TPM2 support for Fedora on UEFI systems.

For brevity I won’t explain in this post what TPMs are and their features, but assume that readers are already familiar with trusted computing in general. Instead, I’ll explain what we have been working on, the approach used and what you might expect on Fedora soon.

For an introduction to TPM, I recommend Matthew Garret’s excellent posts about the topic, Philip Tricca’s presentation about TPM2 and the official Trusted Computing Group (TCG) specifications. I also found “A Practical Guide to TPM 2.0” book to be much easier to digest than the official TCG documentation. The book is an open access one, which means that’s freely available.

LUKS volumes unlocking using a TPM2 device

Encryption of data at rest is a key component of security.  LUKS provides the ability to encrypt Linux volumes, including both data volumes and the root volume containing the OS. The OS can provide the crypto keys for data volumes, but something has to provide the key for the root volume to allow the system to boot.

The most common way to provide the crypto key to unlock a LUKS volume,  is to have a user type in a LUKS pass-phase during boot. This works well for laptop and desktop systems, but is not well suited for servers or virtual machines since is an obstacle for automation.

So the first TPM feature we want to add to Fedora (and likely one of the most common use cases for a TPM) is the ability to bind a LUKS volume master key to a TPM2. That way the volume can be automatically unlocked (without typing a pass-phrase) by using the TPM2 to obtain the master key.

A key point here is that the actual LUKS master key is not present in plain text form on the system, it is protected by TPM encryption.

Also, by sealing the LUKS master key with a specific set of Platform Configuration Registers (PCR), one can make sure that the volume will only be unlocked if the system has not been tampered with. For example (as explained in this post), PCR7 is used to measure the UEFI Secure Boot policy and keys. So the LUKS master key can be sealed against this PCR, to avoid unsealing it if Secure Boot was disabled or the used keys were replaced.

Implementation details: Clevis

Clevis is a plugable framework for automated decryption that has a number of “pins”, where each pin implements an {en,de}cryption support using a different backend. It also has a command line interface to {en,de}crypt data using these pins, create complex security policies and bind a pin to a LUKS volume to later unlock it.

Clevis relies on the José project, which is an C implementation of the Javascript Object Signing and Encryption (JOSE) standard. It also uses the LUKSMeta project to store a Clevis pin metadata in a LUKS volume header.

On encryption, a Clevis pin takes some data to encrypt and a JSON configuration to produce a JSON Web Encryption (JWE) content. This JWE has the data encrypted using a JSON Web KEY (JWK) and information on how to obtain the JWK for decryption.

On decryption, the Clevis pin obtains a JWK using the information provided by a JWE and decrypts the ciphertext also stored in the JWE using that key.

Each Clevis pin defines their own JSON configuration format, how the JWK is created, where is stored and how to retrieve it.

As mentioned, Clevis has support to bind a pin with a LUKS volume. This means that a LUKS master key is encrypted using a pin and the resulting JWE is stored in a LUKS volume meta header. That way Clevis is able to later decrypt the master key and unlock the LUKS volume. Clevis has dracut and udisks2 support to do this automatically and the next version of Clevis will also include a command line tool to unlock non-root (data) volumes.

Clevis TPM2 pin

Clevis provides a mechanism to automatically supply the LUKS master key for the root volume. The initial implementation of Clevis has support to obtain the LUKS master key from a network service, but we have extended Clevis to take advantage of a TPM2 chip, which is available on most servers, desktops and laptops.

By using a TPM, the disk can only be unlocked on a specific system – the disk will neither boot nor be accessed on another machine.

This implementation also works with UEFI Secure Boot, which will prevent the system from being booted if the firmware or system configuration has been modified or tampered with.

To make use of all the Clevis infrastructure and also be able to use the TPM2 as a part of more complex security policies, the TPM2 support was implemented as a clevis tpm2 pin.

On encryption the tpm2 pin generates a JWK, creates an object in the TPM2 with the JWK as sensitive data and binds the object (or seals if a PCR set is defined in the JSON configuration) to the TPM2.

The generated JWE contains both the public and wrapped sensitive portions of the created object, as well as information on how to unseal it from the TPM2 (hashing and key encryption algorithms used to recalculate the primary key, PCR policy for authentication, etc).

On decryption the tpm2 pin takes the JWE that contains both the sealed object and information on how to unseal it,  loads the object into the TPM2 by using the public and wrapped sensitive portions and unseals the JWK to decrypt the ciphertext stored in the JWE.

The changes haven’t been merged yet, since the pin is using features from tpm2-tools master so we have to wait for the next release of the tools. And also there are still discussions on the pull request about some details, but it should be ready to land soon.


The Clevis command line tools can be used to encrypt and decrypt data using a TPM2 chip. The tpm2 pin has reasonable defaults but one can configure most of its parameters using the pin JSON configuration (refer to the Clevis tpm2 pin documentation for these), e.g:

$ echo foo | clevis encrypt tpm2 '{}' > secret.jwe

And then the data can later be decrypted with:

$ clevis decrypt < secret.jwe

To seal data against a set of PCRs:

$ echo foo | clevis encrypt tpm2 '{"pcr_ids":"8,9"}' > secret.jwe

And to bind a tpm2 pin to a LUKS volume:

$ clevis luks bind -d /dev/sda3 tpm2 '{"pcr_ids":"7"}'

The LUKS master key is not stored in raw format, but instead is wrapped with a JWK that has the same entropy than the LUKS master key. It’s this JWK that is sealed with the TPM2.

Since Clevis has both dracut and udisks2 hooks, the command above is enough to have the LUKS volume be automatically unlocked using the TPM2.

The next version of Clevis also has a clevis-luks-unlock command line tool, so a LUKS volume could be manually unlocked with:

$ clevis luks unlock -d /dev/sda3

Using the TPM2 as a part of more complex security policies

One of Clevis supported pins is the Shamir Shared Secret (SSS) pin, that allows to encrypt a secret using a JWK that is then split into different parts. Each part is then encrypted using another pin and a threshold is chose to decide how many parts are needed to reconstruct the encryption key, so the secret can be decrypted.

This allows for example to split the JWK used to wrap the LUKS mater key in two parts. One part of the JWK could be sealed with the TPM2 and another part be stored in a remote server. By sealing a JWK that’s only one part of the needed key to decrypt the LUKS master key, an attacker obtaining the data sealed in the TPM won’t be able to unlock the LUKS volume.

The Clevis encrypt command for this particular example would be:

$ clevis luks bind -d /dev/sda3 sss '{"t": 2, "pins": \
  {"http":{"url":"http://server.local/key"}, "tpm2": \

Limitations of this approach

One problem with the current implementation is that Clevis is a user-space tool and so it can’t be used to unlock a LUKS volume that has an encrypted /boot directory. The boot partition still needs to remain unencrypted so the bootloader is able to load a Linux kernel and an initramfs that contains Clevis, to unlock the encrypted LUKS volume for the root partition.

Since the initramfs is not signed on a Secure Boot setup, an attacker could replace the initramfs and unlock the LUKS volume. So the threat model meant to protect is for an attacker that can get access to the encrypted volume but not to the trusted machine.

There are different approaches to solve this limitation. The previously mentioned post from Matthew Garret suggests to have a small initramfs that’s built into the signed Linux kernel. The only task for this built-in initramfs would be to unseal the LUKS master key, store it into the kernel keyring and extend PCR7 so the key can’t be unsealed again. Later the usual initramfs can unlock the LUKS volume by using the key already stored in the Linux kernel.

Another approach is to also have the /boot directory in an encrypted LUKS volume and provide support for the bootloader to unseal the master key with the TPM2, for example by supporting the same JWE format in the LUKS meta header used by Clevis. That way only a signed bootloader would be able to unlock the LUKS volume that contains /boot, so an attacker won’t be able to tamper the system by replacing the initramfs since it will be in an encrypted partition.

But there is work to be done for both approaches, so it will take some time until we have protection for this threat model.

Still, having an encrypted root partition that is only automatically unlocked on a trusted machine has many use cases. To list a few examples:

  • Stolen physical disks or virtual machines images can’t be mounted on a different machine.
  • An external storage media can be bind to a set of machines, so it can be automatically unlocked only on trusted machines.
  • A TPM2 chip can be reset before sending a laptop to repair, that way the LUKS volume can’t be automatically unlocked anymore.
  • An encrypted volume can be bound to a TPM2 if there is no risk of someone having physical access to the machine but unbound again when there is risk. So the machine can be automatically unlocked on safe places but allow to require a pass-phrase on unsafe places.


I would like to thanks Nathaniel McCallum and Russell Doty for their feedback and suggestions for this article.

October 16, 2017

Build a workbench in 2 years

(This post has nothing to do with software, move along if you aren’t into woodworking…)

Here’s my somewhat different workbench design (based on Christopher Schwarz’s “Knockdown Nicholson” plans which became this article), thinking someone out there might be interested. Also it’s a lesson in how to make simple plans ten times more complicated — whether you’d like to avoid or imitate my example, I’ll leave to you.

Here’s my finished workbench. (Someone actually skilled could build something prettier! But I think my bench will work well for decades if I’m fortunate.)


And here’s a tweet with the pile of wood I started with two and a half years ago:

The Popular Woodworking article suggests “With $100 in lumber and two days, you can build this sturdy stowaway bench” to which I say, hold my beer. I can spend a lot more, and take way more time!

I have many excuses: my bench project is 2.5 years old, and my daughter is just over 2 years old. I started a new job and then left that to start a company. I probably had a few hours of shop time on average per week, and I worked on a lot of shop projects that weren’t the workbench itself.

Excuses only account for the calendar time though, not the shop time. This was closer to a 200-hour project for me than a two-day project.

How do you make a workbench project take ten times longer? (Other than “be an inexperienced hobbyist woodworker”?)

1. Use hardwood rather than southern yellow pine

I bought a pile of soft maple thinking it’d be a mostly-cosmetic choice. I’m very glad the finished bench is maple, but it slowed me down in several ways:

  • the maple is heavy and the bench parts are hard to move with only one person, leading to several times where I had to wait for help to come over
  • I had to mill all the boards from rough-sawn
  • I decided maple meant more worry about wood movement, leading to design complexity

When I started this, I’d just finished a couple of sawbenches and some bookshelves made of pine, and I was sick of the stuff; it’s horrible. (Bias I will admit to: I have childhood memories of walking through stands of loblolly pine trees in 95-degree Georgia heat, getting ticks and sweating; loblolly pine offers no shade to speak of. Loblolly “forests” are the worst.)

2. Make it 8 feet long

Unfortunately, both jointer planes and powered jointers are designed for up to 6′ boards. 8′ boards not only have more surface area, they are also too long for the jointers. 8′ seems like it should be 33% more work than 6′, but it isn’t linear like that because the required skill level goes up.

I started this project to solve a too-short-bench problem. My old bench is based on these Ana White plans. I fixed that one up to be coplanar on the front, added a vise, and added a bunch of weight; it’s hideous but it does permit handwork with my modifications… as long as your boards are no longer than about 2.5 feet. The first time I tried to make a project with longer boards, I discovered I’d need a new bench.

My bench isn’t only longer than the original plans; everything is larger-scale. Most of the 8/4 lumber came out about 1-3/4″ rather than 1-1/2″ like construction lumber. The legs are 8″ wide rather than 5-1/2″ wide, and the top is 2-1/4″ thick all the way to the edge.

3. No power saws

I started to do this entirely with hand tools; after a while I caved and got a nice jointer/planer machine.  Milling these boards completely by hand was beyond my hobbyist pay grade.  That said, every part of the bench still required significant hand-planing, and I didn’t use power saws or routers. I’d guess I spent hours just cleaning up ends with a shooting board.

If I built this bench again, I’d probably get a track saw, which would save a little sawing time and a LOT of cleanup-planing time.

4. Attach the top to the aprons rather than the leg assemblies

After I started the project, I realized that the original Knockdown Nicholson design doesn’t allow for much wood movement. Southern yellow pine doesn’t move too much, and I was worried maple would cause a problem. Maybe it would have, maybe not, I don’t know.

Rather than bolt the top to the leg assemblies, I used dowel nuts (the large 3/8-16 Veritas knockdown variety) to bolt “joists” between the aprons, and then lag-screwed the top to those joists.

Instagram Photo

There are advantages to the way I did it:

  • No counterbores on top to be ugly and fill with shavings
  • If the aprons move seasonally, the top should stay flat rather than being pushed up on the edges
  • The top is ultra-solid: 2-1/4 inches thick, then also held flat and supported by the four joists and the aprons
  • The joists are exactly the same length as the leg assemblies are wide, so they help hold the aprons flat

There are also disadvantages:

  • Lots of extra time: making the joists, drilling the two intersecting holes for the knockdown fasteners, making the notches in the aprons for the joists
  • Knockdown and assembly are harder (I intend to do this only when moving to a new house, so it’s OK, but it’d be an issue in a bench meant to regularly go places)

5. Build the leg assemblies with giant dovetails

Giant dovetails turn out to be much more time-consuming than regular dovetails. I started on this path because I didn’t have enough lumber to make the large screwed-on “plate” in the original plans.

I sawed most of the tails at least a little bit off square; squaring them up wasn’t easy at all, since they were wider than any chisel I owned. Similarly, the sockets were deeper than any router plane I had would go, with sides and bottom too large for my chisels. If you have timber-framing tools you might be able to do this more quickly than I did. This was another consequence of using the rough-sawn maple rather than construction lumber. Tools tend to top out at 1-1/2″ widths and depths, while the maple was more like 1-3/4″.

6. Overkill the tolerances

With more skill, I’d have known how to cut more corners. Instead, I made things as perfect as I could make them.  This was still far from perfect — I could point out flaws in the workbench for hours!

To build a bench really quickly I think you’d want to avoid milling or planing construction lumber at all. But gosh it’d have some huge gaps. (My old Ana-White-style workbench is like this, because I owned neither plane nor planer… I pulled everything square with clamps, then Kreg-screwed it in place.)

7. Build a workbench without a workbench

While building a workbench, I often thought “this sure would be easier if I had a workbench.”

Planing boards on sawbenches sucks. Hello back pain! My old workbench is only 3′ wide, so it wasn’t an option (that’s why I built the new bench in the first place). It’d almost be worth building a terrible-but-full-size Kreg-screwed temporary bench, purely to build the final bench on, and then burning the temporary bench. Or perhaps some sort of bench-height sawhorses-and-plywood contraption.

What went well

The bench works very well — everything that made it take more time, had at least some payoff. I’m glad I have an 8′ maple bench instead of a 6′ pine bench. I’m glad it’s as good as I knew how to make it. The obnoxious-to-build joists made the top prettier and flatter, and the giant dovetails made the leg assemblies rock solid.

It breaks down into 5 parts, just like Christopher Schwarz’s original, and the McMaster-Carr mounting plates work great.

I love the Benchcrafted swing-away seat, it gives me somewhere to sit down that takes up zero floor space when not in use. (Of course I overkilled attaching it, with a bolt all the way through the bench leg, and thick square washers.)

Lessons learned

Ordering a workbench from Plate 11 or Lie-Nielsen makes total sense and their prices are a bargain!

If you do build something, consider sticking to the simple plan.

And I’m now a whole lot better at planing, sawing, drilling, sharpening, and all sorts of other skills than I was when I started. The next project I make might go a little bit faster.


Who knew we still had low-hanging fruits?

Earlier this month I had the pleasure of attending the Web Engines Hackfest, hosted by Igalia at their offices in A Coruña, and also sponsored by my employer, Collabora, Google and Mozilla. It has grown a lot and we had many new people this year.

Fun fact: I am one of the 3 or 4 people who have attended all of the editions of the hackfest since its inception in 2009, when it was called WebKitGTK+ hackfest \o/


It was a great get together where I met many friends and made some new ones. Had plenty of discussions, mainly with Antonio Gomes and Google’s Robert Kroeger, about the way forward for Chromium on Wayland.

We had the opportunity of explaining how we at Collabora cooperated with igalians to implemented and optimise a Wayland nested compositor for WebKit2 to share buffers between processes in an efficient way even on broken drivers. Most of the discussions and some of the work that led to this was done in previous hackfests, by the way!


The idea seems to have been mostly welcomed, the only concern being that Wayland’s interfaces would need to be tested for security (fuzzed). So we may end up going that same route with Chromium for allowing process separation between the UI and GPU (being renamed Viz, currently) processes.

On another note, and going back to the title of the post, at Collabora we have recently adopted Mattermost to replace our internal IRC server. Many Collaborans have decided to use Mattermost through an Epiphany Web Application or through a simple Python application that just shows a GTK+ window wrapping a WebKitGTK+ WebView.


Some people noticed that when the connection was lost Mattermost would take a very long time to notice and reconnect – its web sockets were taking a long, long time to timeout, according to our colleague Andrew Shadura.

I did some quick searching on the codebase and noticed WebCore has a NetworkStateNotifier interface that it uses to get notified when connection changes. That was not implemented for WebKitGTK+, so it was likely what caused stuff to linger when a connection hiccup happened. Given we have GNetworkMonitor, implementation of the missing interfaces required only 3 lines of actual code (plus the necessary boilerplate)!


I was surprised to still find such as low hanging fruit in WebKitGTK+, so I decided to look for more. Turns out WebCore also has a notifier for low power situations, which was implemented only by the iOS port, and causes the engine to throttle some timers and avoid some expensive checks it would do in normal situations. This required a few more lines to implement using upower-glib, but not that many either!

That was the fun I had during the hackfest in terms of coding. Mostly I had fun just lurking in break out sessions discussing the past, present and future of tech such as WebRTC, Servo, Rust, WebKit, Chromium, WebVR, and more. I also beat a few challengers in Street Fighter 2, as usual.

I’d like to say thanks to Collabora, Igalia, Google, and Mozilla for sponsoring and attending the hackfest. Thanks to Igalia for hosting and to Collabora for sponsoring my attendance along with two other Collaborans. It was a great hackfest and I’m looking forward to the next one! See you in 2018 =)

Shaking the tin for LVFS: Asking for donations!

tl;dr: If you feel like you want to donate to the LVFS, you can now do so here.

Nearly 100 million files are downloaded from the LVFS every month, the majority being metadata to know what updates are available. Although each metadata file is very small it still adds up to over 1TB in transfered bytes per month. Amazon has kindly given the LVFS a 2000 USD per year open source grant which more than covers the hosting costs and any test EC2 instances. I really appreciate the donation from Amazon as it allows us to continue to grow, both with the number of Linux clients connecting every hour, and with the number of firmware files hosted. Before the grant sometimes Red Hat would pay the bandwidth bill, and other times it was just paid out my own pocket, so the grant does mean a lot to me. Amazon seemed very friendly towards this kind of open source shared infrastructure, so kudos to them for that.

At the moment the secure part of the LVFS is hosted in a dedicated Scaleway instance, so any additional donations would be spent on paying this small bill and perhaps more importantly buying some (2nd hand?) hardware to include as part of our release-time QA checks.

I already test fwupd with about a dozen pieces of hardware, but I’d feel a lot more comfortable testing different classes of device with updates on the LVFS.

One thing I’ve found that also works well is taking a chance and buying a popular device we know is upgradable and adding support for the specific quirks it has to fwupd. This is an easy way to get karma from a previously Linux-unfriendly vendor before we start discussing uploading firmware updates to the LVFS. Hardware on my wanting-to-buy list includes a wireless network card, a fingerprint scanner and SSDs from a couple of different vendors.

If you’d like to donate towards hardware, please donate via LiberaPay or ask me for PayPal/BACS details. Even if you donate €0.01 per week it would make a difference. Thanks!

Safe Browsing in Epiphany

I am pleased to announce that Epiphany users will now benefit from a safe browsing support which is capable to detect and alert users whenever they are visiting a potential malicious website. This feature will be shipped in GNOME 3.28, but those who don’t wish to wait that long can go ahead and build Epiphany from master to benefit from it.

The safe browsing support is enabled by default in Epiphany, but you can always disable it from the preferences dialog by toggling the checkbox under General -> Web Content -> Try to block dangerous websites.

Safe browsing is implemented with the help of Google’s Safe Browsing Update API v4. How this works: the URL’s hash prefix is tested against a local database of unsafe hash prefixes, and if a match is found then the full hash is further requested from the Google Safe Browsing server to be compared to the URL’s full hash. If the full hashes are equal, then the URL is considered unsafe. Of course, all hash prefixes and full hashes are cached for a certain amount of time, in order to minimize the number of requests sent to the server. Needless to say that working only with URL hashes brings a big privacy bonus since Google never knows the actual URLs that clients browse. The whole description of the API can be found here.


This year’s GUADEC came a bit unexpectedly for me. I wasn’t really planning to attend it because of my school and work, but when Iulian suggested that we should go, I didn’t have to think twice and agreed immediately. And I was not disappointed! Travelling to Manchester proved to be a great vacation where I could not only enjoy a few days off but also learn things and meet new and old friends.

Much like last year’s GUADEC, I attended some of the talks during the core days where I got to find out more things about new technologies such as Flatpak, Meson, BuildStream (I’m really looking forward to seeing how this one turns out in the future) etc, and also about the GNOME history and future prospects.

One of this year’s social events was GNOME’s 20th anniversary party held Saturday night at Museum of Science and Industry. I have to thank the organization team for arranging such a great party and taking care of everything. This was definitely the highlight of this year!

As usual, I’m gonna lay a few words about the location that we were in – Manchester. I found Manchester a nice and cozy city, packed with everything: universities, museums, parks, and restaurants of all kinds for all tastes. The weather was not the best that you can get, with all the rainy and sunny sessions that alternate on an hourly basis, but I guess that’s typical for UK. Overall, I think that Manchester is an interesting city where one would not ever get bored.

Thanks again to the GUADEC team and to GNOME for hosting such an awesome event!


Builder gains multi-touch gestures

If you’re running Wayland and have a touchpad capable of multi-touch, Builder (Nightly) now lets you do fun stuff like the following video demonstrates.

Just three-finger-swipe left or right to move the document. Content is sticky-to-fingers, which is my expectation when using gestures.

It might also work on a touchscreen, but I haven’t tried.

Ubuntu GNOME Shell in Artful: Day 15

Since the Ubuntu Rally in New York, the Ubuntu desktop team is full speed ahead on the latest improvements we can make to our 17.10 Ubuntu release, Artful Aardvark. Last Thursday was our Final Freeze and I think it’s a good time to reflect some of the changes and fixes that happened during the rally and the following weeks. This list isn’t exhaustive at all, of course, and only cover partially changes in our default desktop session, featuring GNOME Shell by default. For more background on our current transition to GNOME Shell in artful, you can refer back to our decisions regarding our default session experience as discussed in my blog post.

Day 15: Final desktop polish before 17.10 is out

GNOME 3.26.1

Most of you would have noticed already, but most of GNOME modules have been updated to their 3.26.1 release. This means that Ubuntu 17.10 users will be able to enjoy the latest and greatest from the GNOME project. It’s been fun to follow again the latest development release, report bugs, catch up regressions and following new features.

GNOME 3.26.1 introduces in addition to many bug fixes, improvements, documentation and translation, updates resizeable tiling support, which is a great feature that many people will surely take advantage of! Here is the video that Georges has done and blogged about while developing the feature for those who didn’t have a look yet:

A quick Ubuntu Dock fix rampage

I’ve already praised here many times the excellent Dash to Dock upstream for their responsiveness and friendliness. A nice illustration of this occurred during the Rally. Nathan grabbed me in the Desktop room and asked if a particular dock behavior was desired (scrolling on the edge switching between workspaces). First time I was hearing that feature and finding the behavior being possibly confusing, I pointed him to the upstream bug tracker where he filed a bug report. Even before I pinged upstream about it, they noticed the report and engaged the discussion. We came to the conclusion the behavior is unpredictable for most users and the fix was quickly in, which we backported in our own Ubuntu Dock as well with some other glitch fixes.

The funny part is that Chris witnessed this, and reported that particular awesome cooperation effort in a recent Linux Unplugged show.

Theme fixes and suggested actions

With our transition to GNOME Shell, we are following thus more closely GNOME upstream philosophy and dropped our headerbar patches. Indeed, as we previously, for Unity vertical space optimizations paradigm with stripping the title bar and menus for maximized applications, distro-patched a lot of GNOME apps to revert the large headerbar. This isn’t the case anymore. However, it created a different class of issues: action buttons are generally now on the top and not noticeable with our Ambiance/Radiance themes.

Enabled suggested action button (can't really notice it)

We thus introduced some styling for the suggested action, which will consequently makes other buttons noticeable on the top button (this is how upstream Adwaita theme implements it as well). After a lot of discussions on what color to use (we tied of course different shades of orange, aubergine…), working with Daniel from Elementary (proof!), Matthew suggested to use the green color from the retired Ubuntu Touch color palette, which is the best fit we could ever came up with ourself. After some gradient work to make it match our theme, and some post-upload fixes for various states (thanks to Amr for reporting some bugs on them so quickly which forced me to fix them during my flight back home :p). We hope that this change will help users getting into the habit to look for actions in the GNOME headerbars.

Enabled suggested action button

Disabled suggested action button

But That’s not all on the theme front! A lot of people were complaining about the double gradient between the shell and the title bar. We just uploaded for the final freeze some small changes by Marco making them looking a little bit better for both titlebars, headerbars and gtk2 applications, when focused on unfocused, having one or no menus. Another change was made in GNOME Shell css to make our Ubuntu font appear a little bit less blurry than it was under Wayland. A long-term fix is under investigation by Daniel.

Headerbar on focused application before theme change

Headerbar on focused application with new theme fix

Title bar on focused application before theme change

Title bar on focused application with new theme fix

Title bar on unfocused application with new theme fix

Settings fixes

The Dock settings panel evolved quite a lot since its first inception.

First shot at Dock settings panel

Bastien, who had worked a lot on GNOME Control Center upstream, was kindly enough to give a bunch of feedbacks. While some of them were too intrusive so late in the cycle, we implemented most of his suggestions. Of course, even if we live less than 3 kms away from each other, we collaborated as proper geeks over IRC ;)

Here is the result:

Dock settings after suggestions

One of the best advice was to move the background for list to white (we worked on that with Sébastien), making them way more readable:

Settings universal access panel before changes

Settings universal access panel after changes

Settings search Shell provider before changes

Settings search Shell provider after changes

i18n fixes in the Dock and GNOME Shell

Some (but not all!) items accessible via right clicking on applications in the Ubuntu Dock, or even in the upstream Dash in the vanilla session weren’t translated.

Untranslated desktop actions

After a little bit of poking, it appeared that only the Desktop Actions were impacted (what we called “static quicklist” in the Unity world). Those were standardized some years after we introduced it in Unity in Freedesktop spec revision 1.1.

Debian, like Ubuntu is extracting translations from desktop files to include them in langpacks. Glib is thus distro-patched to load correctly those translations. However, the patch was never updated to ensure action names were returning localized strings, as few people are using those actions. After a little bit of debugging, I fixed the patch in Ubuntu and proposed back in the Debian Bug Tracking System. This is now merged for the next glib release there (as the bug impacts both Ubuntu, Debian and all its derivatives).

We weren’t impacted by this bug previously as when we introduced this in Unity, the actions weren’t standardized yet and glib wasn’t supporting it. Unity was thus directly loading the actions itself. Nice now to have fixed that bug so that other people can benefit from it, using Debian and vanilla GNOME Shell on Ubuntu or any other combinations!

Translated desktop actions

Community HUB

Alan announced recently the Ubuntu community hub when we can exchange between developers, users and new contributors.

When looking at this at the sprint, I decided that it could be a nice place for the community to comment on those blog posts rather than creating another silo here. Indeed, the current series of blog post have more than 600 comments, I tried to be responsive on most of them requiring some answers, but I can’t obviously scale. Thanks to some of the community who already took the time to reply to already answered questions there! However, I think our community hub is a better place for those kind of interactions and you should see below, an automated created topic on the Desktop section of the hub corresponding to this blog post (if all goes well. Of course… it worked when we tested it ;)). This is read-only, embedded version and clicking on it should direct you to the corresponding topic on the discourse instance where you can contribute and exchange. I really hope that can foster even more participation inside the community and motivate new contributors!

(edit: seems like there is still some random issues on topic creation, for the time being, I created a topic manually and you can comment on here)

Other highlights

We got some multi-monitor fixes, HiDPI enhancements, indicator extension improvements and many others… Part of the team worked with Jonas from Red Hat on mutter and Wayland on scaling factor. It was a real pleasure to meet him and to have him tagging along during the evenings and our numerous walks throughout Manhattan as well! It was an excellent sprint followed by nice follow-up weeks.

If you want to get a little bit of taste of what happened during the Ubuntu Rally, Chris from Jupiter Broadcasting recorded some vlogs from his trip to getting there, one of them being on the event itself:

As usual, if you are eager to experiment with these changes before they migrate to the artful release pocket, you can head over to our official Ubuntu desktop team transitions ppa to get a taste of what’s cooking!

Now, it’s almost time to release 17.10 (few days ahead!), but I will probably blog about the upgrade experience in my next and last - for this cycle - report on our Ubuntu GNOME Shell transition!

Edit: As told before, feel free to comment on our community HUB as the integration below doesn’t work for now.

Blender Daily Doodles — Day 28

People following me on Instagram have been asking why I do the daily renders. You’re not gonna get better by thinking about it. The arsenal of tools and methods in Blender is giant and over the years I still struggle to call myself proficient in any of them.




Follow me on Instagram to see them come alive. I’m probably not gonna maintain the daily routine, but I will continue doing these.

October 15, 2017

Google App Engine: Using subdomains

Separating Frontend and Backend App Engine Services into Separate Subdomains

I’m in the middle of re-implementing my website into separate frontend (client) and backend (server) projects, which I’m deploying to separate App Engine “services”. Previously both the front end HTML and JavaScript, and the backend Java, were all in one (GWT) project, deployed together as one default App Engine service. I chose to separate the two services into their own subdomains. So the frontend is currently served from and its JavaScript code uses the REST API provided by the backend, served from This will later be and

This seemed wise at the time and I thought it was necessary but in hindsight it was an unnecessary detour. I would have been better off just mapping different paths to the services. I didn’t see how to do this at first with Google AppEngine, but it’s very easy. You just provide a dispatch.yaml file (also mentioned below) to map URL prefixes to the different services. For instance, /api/* would map to the backend service, and everything else would map to the frontend service. Larger systems might do this with a service gateway such as Netflix’s Zuul.

Nevertheless, this post mentions some of the issues you’ll have if you choose to split your frontend and backend into separate subdomains. It mentions App Engine, Go, and Angular (TypeScript), but the ideas are generally useful.

App Engine Service Names

Each service that you deploy to App Engine has an app.yaml file. This can have a line to name the service, which would otherwise just be called “default”. For instance, the first line of my frontend’s app.yaml file currently looks like this:

service: beta

The first line of my backend’s app.yaml file currently looks like this:

service: betaapi

When I deploy these services, via “gcloud app deploy” on the command line, I can then see them listed in the Google Cloud console, in the App Engine section, in the Services section.

Mapping Subdomains to App Engine Services

In the Google Cloud console, in the App Engine section, in the settings section, in the “custom domains” tab, you should add each subdomain. You’ll need to verify your ownership of each subdomain by adding DNS entries for Google to check.

App Engine very recently added the “Managed Security” feature, which automatically creates and manages (LetsEncrypt) SSL certificates, letting you serve content via HTTPS. Using this currently makes using subdomains more complicated, because it doesn’t yet support wildard SSL certificates. That’s likely to be possible soon when LetsEncrypt starts providing wildcards SSL certificates, so this section might become outdated.

Without Managed Security

If you aren’t using Managed Security yet, mapping subdomains to services is quite simple. Google’s documentation suggests that you just add a wildcard CNAME entry to the DNS for your domain, like so:

Record: *
 Type: CNAME

All subdomains will then be served by google. App Engine will try to map a subdomain to a service of the same name. So will map to a service named foo.

With Managed Security

However, if you are using Managed Security, you’ll need to tell App Engine about each individual subdomain so each subdomain can have its own SSL certificate. Unfortunately, you’ll then need to add the same 8 A and AAAA records to your DNS for your subdomain too.

Although App Engine does automatic subdomain-to-service mapping for wildcard domains, this doesn’t happen with specifically-listed subdomains. You’ll need to specify the mapping manually using a dispatch.yaml file, like so:

  - url: "*"
    service: beta
  - url: "*"
    service: betaapi

You can then deploy the dispatch.yaml file from the command line like so:

$ gcloud app deploy dispatch.yaml

I wish there was some way to split these dispatch rules up into separate files, so I could associate them with the separate codebases for the separate services. For now, I’ve put the dispatch.yaml file for all the services in the repository for my frontend code and I manually deploy it.

CORS (Cross Origin Resource Sharing)

By default, modern browsers will not allow Javascript that came from (or to make HTTP requests to another domain or subdomain, such as This same-origin policy prevents malicious pages from accessing data on other sites, possibly via your authentication token.

If you try to access a subdomain from JavaScript served from a different subdomain, you’ll see error messages about this in the browser console, such as:

No 'Access-Control-Allow-Origin' header is present on the requested resource. Origin '' is therefore not allowed access.

For instance, JavaScript served from cannot normally access content at

However, you can allow calls to a subdomain by using the CORS system. The browser will attempt the call, but will only provide the response back to the JavaScript if the response has the appropriate Allowed* headers. The browser may first attempt a separate CORS “pre-flight” request before actually issuing the request, depending on the request details.

If you configure your server to reply to a CORS requests with appropriate AllowedOrigins and AllowedMethods HTTP headers, you can tell the browser to allow the JavaScript to make the HTTP requests and receive the responses.

For instance, in Go, I’ve used rs/cors to respond to CORS requests, like so, passing in the original julienschmidt/httprouter that I’m using.

c := cors.New(cors.Options{
	AllowedOrigins: []string{"},
	AllowedMethods: []string{"GET", "POST", "OPTIONS"},	

handler := c.Handler(router)
http.Handle("/", handler)

I also did this in my original Java code by adding a ContainerResponseFilter, annotated with @Provider.

Cookies: CORS

Even when the server responds to CORS requests with AllowedOrigins and AllowedMethods headers, by default the browser will not allow Javascript to send cookies  when it sends HTTP requests to other domains (or subdomains). But you can allow this by adding an AllowCredentials header to the server’s CORS response. For instance, I added the AllowCredentials header in Go on the server side, like so:

c := cors.New(cors.Options{
	AllowCredentials: true,})

You might need to specify this on the client-side too, because the underlying XMLHttpRequest defaults to not sending cookies with cross-site requests. For instance, I specify withCredentials in Angular (Typescript) calls to http.get(), like so:

this.http.get(url, {withCredentials: true})

Note Angular’s awful documentation for the withCredentials option, though Mozilla’s documentation for the XMLHttpRequest withCredentials option is clearer.

Cookies: Setting the Domain

To use a cookie across subdomains, for instance to send a cookie to a domain other than the one that provided the cookie, you may need to set the cookie’s domain, which makes the cookie available to all subdomains in the domain. Otherwise, the cookie will be available only to the specific subdomain that set it.

I didn’t need to do this because I had just one service on one subdomain. This subdomain sets the cookie in responses, the cookie is then stored by the browser, and the browser provides the cookie in subsequent requests to the same subdomain.

OAuth2 Callback

If your subdomain implements endpoints for oauth2 login and callback, you’ll need to tell App Engine about the subdomain. In the Google Cloud console, in the “APIs & Services” section, go to the Credentials section. Here you should enter the subdomain for your web page under “Authorized JavaScript origins”, and enter the subdomain for your oauth2 login and callback subdomain under “Authorized redirect URIs”.

The subdomain will then be listed appropriately in the configuration file that you download via the “Download JSON” link, which you can parse in your code, so that the oauth2 request specifies your callback URL. For instance, I parse the downloaded config .json file in Go using google.ConfigFromJSON() from the package, like so:

func GenerateGoogleOAuthConfig(r *http.Request) *oauth2.Config {
	c := appengine.NewContext(r)

	b, err := ioutil.ReadFile(configCredentialsFilename)
	if err != nil {
		log.Errorf(c, "Unable to read client secret file (%s): %v", configCredentialsFilename, err)
		return nil

	config, err := google.ConfigFromJSON(b, credentialsScopeProfile, credentialsScopeEmail)
	if err != nil {
		log.Errorf(c, "Unable to parse client secret file (%) to config: %v", configCredentialsFilename, err)
		return nil

	return config

retro-gtk: Renaissance

This is the second article in a small series about retro-gtk, I recommend you to read the first one, retro-gtk: Postmortem, before this one.

In the previous article I listed some problems I encountered while developing and using retro-gtk; in this one I will present some solutions I implemented to fix them! ☺ All that is presented in this article is part of the newly-released retro-gtk 0.13.1, which is the first version of the 0.14 development cycle.

Changing the Scope

The Libretro API is tricky: lots of little details need to be handled properly and it isn't always very clear how to do so. By mimicking this API, retro-gtk inherited its complexity, making it way more complex than it should be as there aren't many different ways for a Libretro frontend to handle the cores correctly. retro-gtk was forwarding the complexity of the Libretro API to its users rather than abstracting it.

About a year ago I decided to slowly change the scope of the library. In the previous article, I described retro-gtk as "a GObject-based plugin system based on the Libretro plugin definition", and this still holds true, what changed is how its users will handle the cores. By taking inspiration from how it was used by the GamesRetroRunner class of Games, I slowly moved retro-gtk away from a Libretro reimplementation for frontends and turned it into a Libretro frontend as a library, offering higher level building block and taking care of most of the complexity of the original API internally.

Do you remember the pseudo-code example from the first article, implementing a load_game() function? It was overly complicated compared to what we actually wanted to do, wasn't it? Well, here is how to implement it in C with the new simplified API.

void load_game (RetroCore *core,
const gchar * const *media_uris)
GError *error = NULL;

retro_core_set_medias (core, media_uris);
retro_core_boot (core, &error);
if (error != NULL) {
g_debug ("Couldn't boot the Libretro core: %s", error->message);
g_error_free (error);

With the new API, even C code with GError handling is shorter than the previous pseudo-code example!

As you can see that's much simpler to use, most of the complexity is now handled internally by retro-gtk. Instead of having to use the many components inherited from the Libretro API, you now simply give the medias to the core prior to booting it, booting up the core will take care of its initialization and of loading the game. This also means that retro-gtk doesn't have to expose the game info or disk interface types, making the API smaller and hence simpler to understand.

Many other similar changes were implemented all around the API, way too many to list. Many features that were implemented as complex to use classes tied to RetroCore have been merged into it, removing lot's of the artificially introduced complexity in-between them.

A noteworthy improvement is the introduction of the RetroCoreView widget. Previously, the video output of the core was handled by RetroCairoDisplay, the audio output by RetroPaPlayer and widget specific input devices — forwarding keyboard and mouse inputs to the core or using the keyboard as a gamepad — were handled by input device objects taking a GtkWidget and listening to its events to implement a specific Libretro input device. It worked somewhat well but demanded lots of code to the user, and interaction between these objects was more complex than it should be.

RetroCoreView implement all these features in a single GTK+ widget with a simple API. There are two main functions to this widget. The first one is to allow you to set a core it should handle, it will display its video and play its audio without requiring you to take care of how to do so. The second one is to allow you to simply access the user inputs it receives by exposing it as controllers of the desired RetroControllerType.

Having all of this inside one widget avoid the user to deal with multiple layers of widgets and objects, rendering the video or capturing events. Handling the video output under the cover gives us more freedom on how to implement it. For example when a hardware accelerated renderer will be introduced, we should be able to change it without breaking the ABI and the users should automatically benefit from it, with no change to their codes or their binaries. This also allows to handle very easily interdependencies between the controllers and the outputs, for example a pointer controller is dependent on where the events are happening on the rendered video. All of this makes this widget simpler to use but also simpler to maintain as lot's of the code became way simpler in the transformation proccess.

For you curiosity, here is a slightly simplified version of RetroCoreView in the generated VAPI.

public class Retro.CoreView : Gtk.EventBox {
public CoreView ();
public Retro.Controller as_controller (Retro.ControllerType controller_type);
public uint64 get_controller_capabilities ();
public int16 get_input_state (Retro.ControllerType controller_type, uint index, uint id);
public void set_core (Retro.Core? core);
public void set_filter (Retro.VideoFilter filter);
public bool can_grab_pointer { get; set; }
public Gdk.Pixbuf pixbuf { get; set; }
public bool snap_pointer_to_borders { get; set; }

There are a few things I'm not sure how to handle properly yet:

  • how to make it clear the keyboard can't be used at the same time as a keyboard and a gamepad;
  • how to make it clear the mouse can't be used at the same time as a mouse and a pointer, the can-grab-pointer property toggles it for the moment but I don't think the wording is right;
  • how to handle user-defined filters (GLSL shaders, etc.) once we will support them.

Porting retro-gtk to C

Porting retro-gtk to C comes with downsides:

  • the porting process is a lot of work, lots of working code will be rewritten in a lower level language which means that lots of code will introduce bugs and memory leaks;
  • C being harder to read and write than Vala, ported code will be harder to maintain and new code will be harder to write correctly;
  • the Vala API will need to be generated from C code instead of Vala code, making it harder to produce and less accurate.

But this port also comes with advantages:

  • C being the native language of the libraries used by retro-gtk, porting retro-gtk to C avoids to access them via language bindings;
  • C being the native language of the Libretro API, there is no need to abstract it in order to use it inside retro-gtk, no need to tweak the low-level memory access the API requires;
  • the complexity of binding Libretro to Vala is removed, which counterweights a bit the complexity of writing C;
  • it's easier to maintain a stable C API and ABI from C than from Vala;
  • it removes the compile-time warnings caused by the Vala-generated C code;
  • the code should be a tiny bit faster (so little I don't expect it to be noticeable) as variable and string duplications from the Vala-generated C code are removed;
  • the compilation is faster as the Vala to C step is removed;
  • the dependency on Vala is removed, even if the one on vapigen is maintained.

Now, retro-gtk is a C library, it uses GTK-Doc to support documentation and introspection, it is introspectable via GObject Introspection and is usable in Vala by compiling a VAPI from the GIR file.

Emergent Advantages

The combination of these two changes, offering a higher level API which doesn't expose too much of the inner working of the library and developing retro-gtk with the same language as its dependencies, allows more room for the devs to work inside the library and to move things around without hitting and breaking the API ceilling. To continue on the room analogy, writing the API directly instead of compiling it from Vala allows to perfectly control what is part of it and what isn't, there are less risks for unexpected cables to hang from the API ceilling, cables we could hit while working. All of this should allow us to have a more stable API.

With the ability to control the API, and now that I am somewhat happy about it I want to change the stability promise of retro-gtk a bit: we will keep refactoring some bits of the API during the 0.14 development cycle, but after that we will try to keep it somewhat stable. What it means is that if we break it, we will try to keep the break as small as possible and we will document and advertise this explicit API break. If this not totally unstable state doesn't frighten you, you can start using retro-gtk for your own software with 0.14!

To celebrate this important milestone in the life of retro-gtk, the library just got its first logo!

So… What Now‽ 😃

So far I explained what was retro-gtk and what were its major problems in the first article, how I solved some of them but more importantly how I prepared the terrain to solve bigger ones in this article… but what's coming next? Now that the library can be improved upon more freely, the next article will detail the plans for its evolution, introducing shiny new big features to make it rock-solid!

October 14, 2017

Berlin 🚄 Strasbourg


I started my Monday morning by paying a visit to the 🏛 Berliner Schloss (Palace of Berlin). When I saw the construction site in 2011 they had only laid out the foundations and I wanted to check how much progress had been made. The walls are still exposed raw concrete in places, but the building is taking shape and built up to the roof. While it’s nothing compared to the 500 years it took to build the Strasbourg Cathedral, it is impressive that something can take such a long time to make.

Another thing I had seen in 2011 and wanted to go back to was the 🏛 Denkmal zur Bücherverbrennung am 10. Mai 1933 (Nazi book burning memorial) on Bebelplatz. I was moved when I stood there the first time, and once again this time. The event it refers to has such a powerful impact to me, and the monument succeeds at capturing it.

I quickly visited Gendarmenmarkt and Pariserplatz (where the famous Brandenburg gate stands), then moved to an area of the city I had not previously had time to explore. I went to see the 🏛 Reichstag, 🏛 Bundestag, 🏛 Bundeskanzleramt and 🏛 House of the cultures of the world. Unfortunately that’s when the rain decided to make a come back, so I didn’t really get to enjoy the view as much as I expected, but the massive buildings were still interesting to see.

… and back again

The main reason the trip on the way in was so adventurous was that there were renovations on some parts of the route and the trains weren’t running as per the usual schedule. That was last week and didn’t apply anymore this week. I was able to book a nightjet, which is a sleeper coach, i.e. a night train with beds. I wouldn’t say it was great, but it was alright. If you ever have to get on such a train, expect a hostel-like experience with bunk beds and tight shared space. It left at 23:10 and took me from Berlin all the way to Offenburg. The remaining part of the trip was a 25 minutes ride on a regional train. I was home at 7:30, fresh and ready to start my day of work.

Bad decisions make great stories.

I do not regret the development of the eastbound trip (and I didn’t have that many alternatives anyway) but I have to say it was nice having a smoother end to this journey.

Modern Text Editor Design

I’ve been fascinated about a few technologies in my career. I have a fondness for finding the right data-structure for a problem. Maybe it was because of all those years playing with cars that gave me the “I wanna go fast” mentality. It lead me to various jobs, including working on databases.

Another technology I love are text editors. There is some really fascinating technology going on behind the scenes.

Gtk4 development is heating up, and we are starting to see a toolkit built like a game engine. That’s pretty cool. But how will that change how we write editors? Should it?

In the Gtk3 cycle, I added support to GtkTextView that would render using Alex’s GtkPixelCache. It helped us amortize the cost of rendering into mostly just an XCopyArea() when drawing a frame. It’s why we have that nice 60fps two-finger-scrolling.

But now that we can have GPU textures, do we want to start doing paginated rendering like Apple did with Preview to make PDF rendering ultra fast? Do we want to focus on just sending text data to the GPU to render from an glyph atlas? How about layout? And intermixing right-aligned with left-aligned text? What if we just focus on code editing and not generic editing with rich font support. How about inserting widgets in-between rows? Do we want unlimited undo? How about crash recovery?

These are all questions that can inform the design of a text editor, and they are things I’m starting to think about.

To inform myself on the problem domain better, I started writing a piecetable implementation with various tweaks to outperform those I’ve seen previously. What I’ve come up with is a combination of a b+tree (bplus tree) and a piecetable. The neat thing about it is the high-density you get per-cacheline as compared to something in a RBTree or SplayTree (Atom recently did this, and AbiWord did it a decade ago). It’s at least as fast as those, but with much less malloc overhead because you need fewer, but densely packed allocations.

I prefer dense-cacheline packed structures over pointer chasing (dereferencing pointers) because the CPU can crunch through numbers in a cacheline much faster than it can load another cacheline (as it may not yet be in L1-3 caches). So while it looks like you’re doing more work, you may in fact be saving time.

On my 10 year old 1.2ghz ThinkPad, I can do 1.5 to 2 million random inserts per-second. So I think it’s “good enough” to move on to solving the next layer of problems.

One neat thing about using the linked-leaves from a b+tree is that you get a “next pointer” from each leaf to the next sequential leaf. So if you need to reconstruct the buffer, it’s a simple scan without tree traversal. This is a common problem in a text editor, because we’re sending out text data to diagnostic engines constantly and it needs to be optimized for.

Part of the piecetable design is that you have two buffers. The original data (file state at loading) and change data (append only buffer if each character typed by the user). The piece table is just pointing to ranges in each buffer set to reconstruct the final buffer.

If you log all of your operations to a log file, you can fairly quickly get yourself a crash recovery mechanism. Additionally, you can create unlimited undo.

One thing I don’t like about GtkTextBuffer today is that you cannot open “very large files” with it. It can certainly handle 100mb text files, but it has to load all of that data into memory. And that means that opening a 10gb SQL dump is going to be fairly difficult. But if we implemented on-demand, paginated data loading (reading from disk when that portion of the file is needed), we can get a fixed memory overhead for a file.

One downside to that approach is that if the file is modified behind the scenes, you are basically screwed. (A proper rename() would not affect things since the old FD would still be valid). One way to work around this is to copy the file before editing (a swap file). If your filesystem has reflink support, that copy is even “free”.

Some files are in encodings that require conversion to UTF-8. If a character encoding crossed the page/block boundary, we’d not be able to convert it without locating the neighboring page. So it seems somewhat at odds with this design. But if we just do the iconv (or similar) encoding conversion as part of the copy to our swap file, you can ensure you have valid UTF-8 to begin with. It’s also a convenient place to count new lines so that you can get relatively accurate document height from the start (otherwise you have to scan the data and count newlines which will jump the scrollbar around).

Another thing that might be an interesting trick is to keep around PangoLayouts for each of the cursor lines. It might allow us to mutate the layout immediately upon the key-press-event and render the content out of order (without going through layout cycles). This is somewhat similar to what other editors do to make things “feel” more interactive. It guarantees you render the key event on the next frame, even if slightly incorrect.

In short, writing a good, fast, modern text editor today is the combination of writing a database and a graphics engine. Btrees, low-cardinality indexes, page caches, write ahead logs, transaction replays, memory pooling, GPU dispatching, texture atlases, layout, and more.

October 13, 2017

NoCoffee: Visual Impairment Simulator

Four years ago, on a snowy February day, Aaron Leventhal huddled in his unheated home and created a Chrome extension called NoCoffee. This extension allows users to experience web content through different lenses of visual impairments*.

I recently ran across this extension again, and thought it is high-time we ported it to Firefox. Firefox’s support of WebExtension standards means that this should be trivial. It was! With Aaron’s permission, I posted the source to github and did some tweaking and cleanup.

You can now try out the extension in Firefox!

* Not medically or scientifically accurate.

New packages in Fedora: rtags, renderdoc

Just a very quick announcement. I created two new packages for Fedora:

  • rtags - A source code indexer for the C language family. I use it together with Emacs, to get IDE-like functionality. The package contains systemd (user) unit files that should socket-activate the rtags daemon on demand. Check via systemctl status --user rtags.socket.
  • renderdoc - a stand-alone graphics debugger; it is a wonderful tool to help developers when doing anything 3D (OpenGL, Vulkan) related.
    Thanks goes to @baldurk for a patch to install renderdoc's library into a private location.

Install on Fedora 26, 27 and rawhide via dnf install rtags renderdoc.

I also created my first flatpak:

  • NixView - as the name suggests a viewer for the free and open NIX scientific data format. It is available via flathub. Open data formats are the necessary basis for data sharing, which is one of the most important problems in science that is still not solved properly.

All this was done during the workshop & hackathon on open data formats and data sharing in neuroscience (in Japan!). Thanks goes to the German Neuroinformatics Node and Red Hat (my employer), that made it possible for me to attend it; and also to the Japanese Node for organizing it. I had a great and productive time in Japan.

Policy hacking

Last week I attended the first ever GNOME Foundation hackfest in Berlin, Germany. The hackfest was part of an effort to redefine how the GNOME Foundation operates and is perceived. There are a number of aspects to this process:

  1. Giving the Board of Directors a higher-level strategic oversight role.
  2. Empowering our staff to take more executive action.
  3. Decentralising the Foundation, so that authority and power is pushed out to the community.
  4. Engaging in strategic initiatives that benefit the GNOME project.

Until now, the board has largely operated in an executive mode: each meeting we decide on funding requests, trademark questions and whatever other miscellaneous issues come our way. While some of this decision-making responsibility is to be expected, it is also fair to say that the board spends too much time on small questions and not enough on bigger ones.

One of the reasons for last week’s hackfest was to try and shift the board from its executive role to a more legislative one. To do this, we wrote and approved spending policies, so that expenditure decisions don’t have to be made on a case-by-case basis. We also approved a budget for the financial year and specified budget holders for some lines of expenditure.

With these in place the board is now in a position to relinquish control over trivial spending decisions and to take up a high-level budget oversight role. Going forward the board will have have its eye on the big budget picture and not on the detail. Smaller spending decisions will be pushed out to our staff, to individual budget holders from the community and to committees.

It is hoped that these changes will allow us to play a more strategic role in the future. This transition will probably take some time yet, and there are some other areas that still need to be addressed. However, with the Berlin hackfest we have made a major step forward.

Huge thanks to the good people at Kinvolk for providing the venue for the event, and to the GNOME Foundation for sponsoring me to attend.

GLib tools rewrite

You can safely skip this article if you’re not building software using enumeration types and signal handlers; or if you’re already using Meson.

For more that 15 years, GLib has been shipping with two small utilities:

  • glib-mkenums, which scans a list of header files and generates GEnum and GFlags types out of them, for use in GObject properties and signals
  • glib-genmarshal, which reads a file containing a description of marshaller functions, and generates C code for you to use when declaring signals

If you update to GLib 2.54, released in September 2017, you may notice that the glib-mkenums and glib-genmarshal tools have become sligly more verbose and slightly more strict about their input.

During the 2.54 development cycle, both utilities have been rewritten in Python from a fairly ancient Perl, in the case of glib-mkenums; and from C, in the case of glib-genmarshal. This port was done to address the proliferation of build time dependencies on GLib; the cross-compilation hassle of having a small C utility being built and used during the build; and the move to Meson as the default (and hopefully only) build system for future versions of GLib. Plus, the port introduced colorised output, and we all know everything looks better with colors.

Sadly, none of the behaviours and expected input or output of both tools have ever been documented, specified, or tested in any way. Additionally, it turns out that lots of people either figured out how to exploit undefined behaviour, or simply cargo-culted the use of these tools into their own project. This is entirely on us, and I’m going to try and provide better documentation to both tools in the form of a decent man page, with examples of integration inside Autotools-based projects.

In the interest of keeping old projects building, both utilities will try to replicate the undefined behaviours as much as possible, but now you’ll get a warning instead of the silent treatment, and maybe you’ll get a chance at fixing your build.

If you are maintaining a project using those two utilities, these are the things to watch out for, and ideally to fix by strictly depending on GLib ≥ 2.54.


  • if you’re using glib-genmarshal --header --body to avoid the “missing prototypes” compiler warning when compiling the generated marshallers source file, please switch to using --prototypes --body. This will ensure you’ll get only the prototypes in the source file, instead of a whole copy of the header.

  • Similarly, if you’re doing something like the stanza below in order to include the header inside the body:

    foo-marshal.h: foo-marshal.list Makefile
            $(AM_V_GEN) \
              $(GLIB_GENMARSHAL) --header foo-marshal.list \
            > foo-marshal.h
    foo-marshal.c: foo-marshal.h
            $(AM_V_GEN) (
              echo '#include "foo-marshal.h"' ; \
              $(GLIB_GENMARSHAL) --body foo-marshal.list \
            ) > foo-marshal.c

    you can use the newly added --include-header command line argument, instead.

  • The stanza above has also been used to inject #define and #undef pre-processor directives; these can be replaced with the -D and -U newly added command line arguments, which work just like the GCC ones.

  • This is not something that came from the Python port, as it’s been true since the inclusion of glib-genmarshal in GLib, 17 years ago: the NONE and BOOL tokens are deprecated, and should not be used; use VOID and BOOLEAN, respectively. The new version of glib-genmarshal will now properly warn about this, instead of just silently converting them, and never letting you know you should fix your marshal.list file.

If you want to silence all messages outside of errors, you can now use the --quiet command line option; conversely, use --verbose if you want to get more messages.


The glib-mkenums port has been much more painful than the marshaller generator one; mostly, because there are many, many more ways to screw up code generation when you have command line options and file templates, and mostly because the original code base relied heavily on Perl behaviour and side effects. Cargo culting Autotools stanzas is also much more of a thing when it comes to enumerations than marshallers, apparently. Imagine what we could achieve if the tools that we use to build our code didn’t actively work against us.

  • First of all, try and avoid having mixed encoding inside source code files that are getting parsed; mixing Unicode and ISO-8859 encoding is not a great plan, and C does not have a way to specify the encoding to begin with. Yes, you may be doing that inside comments, so who cares? Well, a tool that parses comments might.

  • If you’re mixing template files with command line arguments for some poorly thought-out reason, like this:

    foo-enums.h: Makefile
            $(AM_V_GEN) $(GLIB_MKENUMS) \
              --fhead '#ifdef FOO_ENUMS_H' \
              --fhead '#defineFOO_ENUMS_H' \
              --template \
              --ftail '#endif /* FOO_ENUMS_H */' \
            > foo-enums.h

    the old version of glib-mkenums would basically build templates depending on the phase of the moon, as well as some internal detail of how Perl works. The new tool has a specified order:

    • the HEAD stanzas specified on the command line are always prepended to the template file
    • the PROD stanzas specified on the command line are always appended to the template file
    • the TAIL stanzas specified on the command line are always appended to the template file

Like with glib-genmarshal, the glib-mkenums tool also tries to be more verbose in what it expects.

Ideally, by this point, you should have switched to Meson, and you’re now using a sane build system that generates this stuff for you.

If you’re still stuck with Autotools, though, you may also want to consider dropping glib-genmarshal, and use the FFI-based generic marshaller in your signal definitions — which comes at a small performance cost, but if you’re putting signal emission inside a performance-critical path you should just be ashamed of yourself.

For enumerations, you could use something like this macro, which I tend to employ in all my projects with just few, small enumeration types, and where involving a whole separate pass at parsing C files is kind of overkill. Ideally, GLib would ship its own version, so maybe it’ll be replaced in a new version.

Many thanks to Jussi Pakkanen, Nirbheek Chauhan, Tim-Philipp Müller, and Christoph Reiter for the work on porting glib-mkenums, as well as fixing my awful Parseltongue.

Call for help: fund GIMP development and Libre animation

Too long, didn’t read? In a few words: our GIMP development + ZeMarmot production is currently funded barely above 400 € per month, this doesn’t pay the bills, my main computer broke today and Aryeom’s graphics tablet has been working badly for some time now. We are a bit bummed out.

So we call for your help!
You can fund GIMP development and ZeMarmot production on Patreon or Tipeee!

Read below for more.

If you read us regularly, you know that I am hacking GIMP a lot. We are just a handful of regular developers  in GIMP, I am one of them. My contributions go from regular bug fixes to bigger features, maintenance of several pieces of code as well as regular code review from contributed patches. I do this in the context of ZeMarmot project, with Aryeom Han, director and animator.  We draw on and hack GIMP because we believe in Free Software.
On the side, I also contribute to a lot of other Free Software.

Our absolutely-not-hidden goal is to be able, one day, to live from hacking Free Software and creating Libre Art. But clearly there is no denying that we are currently failing. With about 400€ a month for 2 people, association LILA can barely pay a few days a month (by the rules, which means a good part of the sum even goes to non-wage labour costs). These 400€ are not even the monthly rent we pay for our 1-room flat (31 m², in the far suburb of Paris); so you would assume well that we don’t live from it. We mostly live off savings and other things to pay the bills. These “other things” also use time we would rather spend on drawing and coding.

We would indeed enjoy working full-time on ZeMarmot, creating Free Software and Libre Art for everyone to enjoy. But we are very far from this point.

The main reason why we have not stopped the project already is that we promised we’d release the pilot. Funders are counting on us. Of course the other reason is that we still hope things will work out and that we will be able to live from what we love. Still the project is done at slow pace because we can’t afford to starve, right? So we are at times demoralized.

This is why I am doing this call. If you can afford it and believe that improving GIMP is important, then I would propose to fund ZeMarmot which supports paid development.
Similarly if you want to see more Libre Art, and in particular cool animation films, and maybe later other movies under Libre licenses in professional quality, then I again propose to support ZeMarmot.

» Patreon funding «
» Tipeee funding «

Our material is dying

And so why is this post released today? The situation has been hard for months now, but today it is dire: my laptop just broke. It just won’t turn on. All my data are safe since I do regular backups (and I think the hard drive is still ok anyway), but I don’t have a computer anymore to work on (I am writing this on a 8-year old 32-bit netbook which barely stands opening a few browser tabs!).

On her side, Aryeom’s graphics tablet has had issues for months. As you may remember, we partly dealt with them, but the tablet regularly shuts down for no reason, we have to remove and put back the battery or similar annoying “workarounds”. And we fear that we have to buy a new one soon.

So that’s what triggered this blog post because I realize how precarious is our situation. We barely get funding for living bills, we eat our savings and now we have (expensive) material issues. So we are calling you all who like Free Software and Libre Art. Do you believe ZeMarmot is a good thing? Do you believe our project has any meaning and that it should continue for years and years? We believe this, and have believed it for the last 2 years where we have been trying. If you do too, maybe help us a bit, relatively to your means. If you really can’t afford it, at least you can spread the word.

ZeMarmot is a wonderful experience for us, and we really don’t want it to have a bitter end (though we won’t regret a second of it).

Thanks for reading!

October 12, 2017

Going to GNOME.Asia 2017

I’m going to GNOME.Asia 2017, which is gonna be held in ChongQing, China during 14-16th Oct. 2017.

I’m at Terminal 3 of Beijing Capital International Airport, waiting to board the plane to ChongQing, China. To be honest, I’m really exited about this trip. I’ve never been to ChongQing before. So visiting the city and eat hotpot is will be a lot of fun. :-)

See you in ChongQing.

October 11, 2017

A scrolling primer

A few years ago, I wrote a post about scrolling in GTK+ 3. Time for another look!

The common case

The basic idea of the changes described back then is still the same. We expect touchpad (or track point) scrolling to be one of the most common forms of scrolling, and the scrollbar operates as a narrow indicator for this.

As you can see, we change the cursor to indicate scrolling, and it you can freely scroll in all directions. It is kinetic, too.

Classical scrolling

Of course, it is still possible to just click and drag the slider, for classical scrolling.

Another aspect of ‘classical’ scrolling is that you can click on the trough outside the slider, and either warp the position to where you clicked, or jump in page-size increments.

By default, a primary click warps, and Shift-primary click goes by pages. We just added back middle click as an alternative to Shift-primary click, since this is a common alternative that many people are used to. For mainly historical reasons, GTK+ has a setting, gtk-primary-button-warps-slider, which switches the roles of primary click and Shift-primary click for this.

The typical keyboard shortcuts (Page Up, Page Down, Home, End) let you control scrolling with the keyboard.

Smooth scrolling

There’s more to scrolling in GTK+ that you may not know about. One feature that we introduced long ago is a ‘zoom’ or ‘fine adjustment’ mode, which slows the scrolling down to allow pixel-precise positioning.

To trigger this mode you can either use a long press or shift-click in the slider. As you can see in the video, once you move the pointer below or above the scrollbar, it will keep scrolling at the same relaxed speed until you let go.

As a variation on this theme, more recently we added a variable speed variant of smooth scrolling.

To trigger it, you secondary click in the trough outside the slider. Once the scrolling starts, you can control the speed by moving the pointer closer or farther away from the scrollbar. This is pretty addictive, once you’ve discovered it!

Custom locations

As the last feature, applications can add a context menu that gets triggered by secondary click on the slider, and make it scroll to important positions.

Thats it, scroll on!

[Problem2 SPOJ:JNEXT] Next Lexicographical Permutation Algorithm; Ad-Hoc


I was solving SPOJ-JNEXT, and from the comments section I came across this page:

Nice explanation but there's a caveat to keep in mind to understand the working better-

It mentions:
If there is no such element – i.e. the entire sequence is non-decreasing – then this is already the last permutation.
It should rather be "non-increasing" instead of "non-decreasing".

[UPDATE: It has been corrected on the website The author personally wrote a thank you note to me confirming  the error and rectifying it.]

My approach to solving this problem was similar so I won't bore you with another copy of how the algorithm works here: you may use the explanation in link above (with the correction in mind).

Tested and accepted:,chandniverma/

Time Complexity of get_next_perm() in worst case: O(n) where n is the number of digits in each test case in the input.
Space complexity: O(n)

Endless Reddit AMA

Along with many colleagues from all across the company (and globe), I’m taking part in Endless’s first Reddit Ask Me Anything today. From my perspective in London it starts at 5pm on Wednesday 11th; check our website for a countdown or this helpful table of time conversions.

Have you been wanting to ask about our work and products in a public forum, but never found a socially-acceptable opportunity? Now’s your chance!

October 10, 2017

GNOME Foundation hackfest in Berlin

Last weekend we held a GNOME Foundation hackfest in Berlin, as planned in August. That means not just a hackfest organised by the foundation, but to improve the foundation itself. Many of the topics we had to cover are interconnected and it’s a challenge to untangle it all and sort it out. Being in the same room, with a projector and a whiteboard, helped a lot. Many thanks to Kinvolk who let us use their 🏢 meeting area for three days.

We are great at setting work for ourselves, less so at getting that work done. This doesn’t mean that we are not doing anything, rather that we have a hard time getting things done quickly enough that the list doesn’t keep growing. A goal was thus to remedy that situation.

To achieve that, one of the main areas we focused on was putting policies in place so that some things work themselves out and the board gets more time to work on other tasks. Indeed many questions required board intervention until now when it wasn’t actually necessary. As an example, our Director of Operations would have needed us to vote for her to be able to buy enveloppes to send letters as part of her day to day work. That was kind of silly and the newly approved policies will put an end to that. They will be propagated to the appropriate places (wiki pages, mailing lists…) in the coming weeks.

Our Executive Director was instrumental in making this happen. His feedback and proposals enabled us to have something concrete and clear that we could discuss, adjust, and agree on.

We reviewed the status of our ongoing action items and managed to do quite some clean up, as you’ll be able to tell if you read the minutes of the next meeting we’ll have. Although I’m disappointed that we didn’t get to spend more time on that part, we also managed to actually work on some action items and cross them off.

This event allowed us to make great progress. I wouldn’t go as far as calling it a clean slate, but it’s pretty close. I’m looking forward to the next time we meet in person and I’m happy that we plan on doing it more often than the traditional yearly Board meeting at GUADEC.

Stable GNOME Photos Flatpaks moved to Flathub

Starting from version 3.26, the stable GNOME Photos Flatpaks have been moved to Flathub. They are no longer available from GNOME’s Flatpak repository.

To migrate, first delete the old build:

$ flatpak uninstall org.gnome.Photos/x86_64/stable

Then install it from Flathub:

$ flatpak remote-add --from flathub
$ flatpak install flathub org.gnome.Photos

Note that this is only about the stable build. The nightly continues to be available from its existing location in GNOME’s repository. You can keep updating it with:

$ flatpak update --user org.gnome.Photos/x86_64/master

Fleet Commander: production ready!

It’s been awhile since I last wrote any updates about Fleet Commander, that’s not to say that there hasn’t been any progress since 0.8. In many senses we (Oliver and I) feel like we should present Fleet Commander as a shiny new project now as many changes have gone through and this is the first release we feel is robust enough to call it production ready.

What is Fleet Commander?

For those missing some background, let me introduce Fleet Commander for you, Fleet Commander is an integrated solution for large Linux desktop deployments that provides a configuration management interface that is controlled centrally and that covers desktop, applications and network configuration. For people familiar with Group Policy Objects in Active Directory in Windows, it is very similar.

Many people ask why not use other popular Linux configuration management tools like Ansible or Puppet, the answer is simple, those are designed for servers that run in a controlled environment like a data center or the cloud, it follows a push model where the configuration changes happen as a series of commands run in the server. If something goes wrong it is easy to audit and rollback if you have access to that server. However desktop machines in large corporate environments can run many times behind a NAT on a public WiFi, think a laptop owned by an on-site support engineer that roams from site to site. Fleet Commander pulls a bunch of configuration data and makes it available to apps without running intrusive shell scripts or walking in into users’ $HOME directory. Ansible and puppet did not solve the core problems of desktop session configuration management so we had to create something new.

At Red Hat we talk to many enterprise desktop customers with a mixed environment of Windows, Macs and Linux desktops and our interaction with them has helped us identify this gap in the GNOME ecosystem and motivated us to roll up our sleeves and try to come up with an answer.

How to build a profile

The way Fleet Commander works when building profiles is somewhat interesting compared to its competitors. We’ve inspired our solution on the good old Sabayon tool. On our admin web UI you get a VM desktop session where you run and configure your apps, Fleet Commander will record those changes and list them. The user will select them and the final selection will get bound together as part of the profile.

You can then apply the profile to individual users, groups, hosts or host groups.


Supported apps/settings

Right now we support anything dconf based (GSettings), GNOME Online Accounts, LibreOffice and NetworkManager. In the near future we plan to tackle our main problem which is giving support to browsers, we’re probably going to start just with bookmarks as it is the most demanded use case.

Cockpit integration


The Fleet Commander UI runs on top of the Cockpit admin UI. Cockpit has given us a lot of stuff for free (a basic UI framework, a web service, built-in websocket support for our SPICE javascript client, among many other things).

FreeIPA Integration

A desktop configuration management solution has to be tightly tied to an identity management solution, (like in Active Directory), FreeIPA is the best Free Software corporate identity management project out there and integrating with it allowed us to remove quite a bit of complexity from our code base while improving security. FreeIPA now stores the profile data and the assignments to users, groups and hosts.


SSSD is the client daemon that enrolls and authenticates a Linux machine in a FreeIPA or Active Directory domain, having fleet commander hooking into it was a perfect fit for us and also allowed us to remove a bunch of code from previous versions while having a much more robust implementation. SSSD now retrieves and stores the profile data from FreeIPA.

Our new website is live! We have updated introduction materials and documentation and jimmac has put together a great design and layout. Check it out!
I’d like to thank Alexander Bokovoy and Fabiano Fidencio for their invaluable help extending FreeIPA and SSSD to integrate with Fleet Commander and Jakub for his help on the website design. If you want to know more, join us on our IRC channel (#fleet-commander @ freenode) and our GitHub project page.

It is currently available in Fedora 26 and we are in the process of releasing EPEL packages for CentOS/RHEL.

October 09, 2017

IP Accounting and Access Lists with systemd

TL;DR: systemd now can do per-service IP traffic accounting, as well as access control for IP address ranges.

Last Friday we released systemd 235. I already blogged about its Dynamic User feature in detail, but there's one more piece of new functionality that I think deserves special attention: IP accounting and access control.

Before v235 systemd already provided per-unit resource management hooks for a number of different kinds of resources: consumed CPU time, disk I/O, memory usage and number of tasks. With v235 another kind of resource can be controlled per-unit with systemd: network traffic (specifically IP).

Three new unit file settings have been added in this context:

  1. IPAccounting= is a boolean setting. If enabled for a unit, all IP traffic sent and received by processes associated with it is counted both in terms of bytes and of packets.

  2. IPAddressDeny= takes an IP address prefix (that means: an IP address with a network mask). All traffic from and to this address will be prohibited for processes of the service.

  3. IPAddressAllow= is the matching positive counterpart to IPAddressDeny=. All traffic matching this IP address/network mask combination will be allowed, even if otherwise listed in IPAddressDeny=.

The three options are thin wrappers around kernel functionality introduced with Linux 4.11: the control group eBPF hooks. The actual work is done by the kernel, systemd just provides a number of new settings to configure this facet of it. Note that cgroup/eBPF is unrelated to classic Linux firewalling, i.e. NetFilter/iptables. It's up to you whether you use one or the other, or both in combination (or of course neither).

IP Accounting

Let's have a closer look at the IP accounting logic mentioned above. Let's write a simple unit /etc/systemd/system/ip-accounting-test.service:


This simple unit invokes the ping(8) command to send a series of ICMP/IP ping packets to the IP address (which is the Google DNS server IP; we use it for testing here, since it's easy to remember, reachable everywhere and known to react to ICMP pings; any other IP address responding to pings would be fine to use, too). The IPAccounting= option is used to turn on IP accounting for the unit.

Let's start this service after writing the file. Let's then have a look at the status output of systemctl:

# systemctl daemon-reload
# systemctl start ip-accounting-test
# systemctl status ip-accounting-test
● ip-accounting-test.service
   Loaded: loaded (/etc/systemd/system/ip-accounting-test.service; static; vendor preset: disabled)
   Active: active (running) since Mon 2017-10-09 18:05:47 CEST; 1s ago
 Main PID: 32152 (ping)
       IP: 168B in, 168B out
    Tasks: 1 (limit: 4915)
   CGroup: /system.slice/ip-accounting-test.service
           └─32152 /usr/bin/ping

Okt 09 18:05:47 sigma systemd[1]: Started ip-accounting-test.service.
Okt 09 18:05:47 sigma ping[32152]: PING ( 56(84) bytes of data.
Okt 09 18:05:47 sigma ping[32152]: 64 bytes from icmp_seq=1 ttl=59 time=29.2 ms
Okt 09 18:05:48 sigma ping[32152]: 64 bytes from icmp_seq=2 ttl=59 time=28.0 ms

This shows the ping command running — it's currently at its second ping cycle as we can see in the logs at the end of the output. More interesting however is the IP: line further up showing the current IP byte counters. It currently shows 168 bytes have been received, and 168 bytes have been sent. That the two counters are at the same value is not surprising: ICMP ping requests and responses are supposed to have the same size. Note that this line is shown only if IPAccounting= is turned on for the service, as only then this data is collected.

Let's wait a bit, and invoke systemctl status again:

# systemctl status ip-accounting-test
● ip-accounting-test.service
   Loaded: loaded (/etc/systemd/system/ip-accounting-test.service; static; vendor preset: disabled)
   Active: active (running) since Mon 2017-10-09 18:05:47 CEST; 4min 28s ago
 Main PID: 32152 (ping)
       IP: 22.2K in, 22.2K out
    Tasks: 1 (limit: 4915)
   CGroup: /system.slice/ip-accounting-test.service
           └─32152 /usr/bin/ping

Okt 09 18:10:07 sigma ping[32152]: 64 bytes from icmp_seq=260 ttl=59 time=27.7 ms
Okt 09 18:10:08 sigma ping[32152]: 64 bytes from icmp_seq=261 ttl=59 time=28.0 ms
Okt 09 18:10:09 sigma ping[32152]: 64 bytes from icmp_seq=262 ttl=59 time=33.8 ms
Okt 09 18:10:10 sigma ping[32152]: 64 bytes from icmp_seq=263 ttl=59 time=48.9 ms
Okt 09 18:10:11 sigma ping[32152]: 64 bytes from icmp_seq=264 ttl=59 time=27.2 ms
Okt 09 18:10:12 sigma ping[32152]: 64 bytes from icmp_seq=265 ttl=59 time=27.0 ms
Okt 09 18:10:13 sigma ping[32152]: 64 bytes from icmp_seq=266 ttl=59 time=26.8 ms
Okt 09 18:10:14 sigma ping[32152]: 64 bytes from icmp_seq=267 ttl=59 time=27.4 ms
Okt 09 18:10:15 sigma ping[32152]: 64 bytes from icmp_seq=268 ttl=59 time=29.7 ms
Okt 09 18:10:16 sigma ping[32152]: 64 bytes from icmp_seq=269 ttl=59 time=27.6 ms

As we can see, after 269 pings the counters are much higher: at 22K.

Note that while systemctl status shows only the byte counters, packet counters are kept as well. Use the low-level systemctl show command to query the current raw values of the in and out packet and byte counters:

# systemctl show ip-accounting-test -p IPIngressBytes -p IPIngressPackets -p IPEgressBytes -p IPEgressPackets

Of course, the same information is also available via the D-Bus APIs. If you want to process this data further consider talking proper D-Bus, rather than scraping the output of systemctl show.

Now, let's stop the service again:

# systemctl stop ip-accounting-test

When a service with such accounting turned on terminates, a log line about all its consumed resources is written to the logs. Let's check with journalctl:

# journalctl -u ip-accounting-test -n 5
-- Logs begin at Thu 2016-08-18 23:09:37 CEST, end at Mon 2017-10-09 18:17:02 CEST. --
Okt 09 18:15:50 sigma ping[32152]: 64 bytes from icmp_seq=603 ttl=59 time=26.9 ms
Okt 09 18:15:51 sigma ping[32152]: 64 bytes from icmp_seq=604 ttl=59 time=27.2 ms
Okt 09 18:15:52 sigma systemd[1]: Stopping ip-accounting-test.service...
Okt 09 18:15:52 sigma systemd[1]: Stopped ip-accounting-test.service.
Okt 09 18:15:52 sigma systemd[1]: ip-accounting-test.service: Received 49.5K IP traffic, sent 49.5K IP traffic

The last line shown is the interesting one, that shows the accounting data. It's actually a structured log message, and among its metadata fields it contains the more comprehensive raw data:

# journalctl -u ip-accounting-test -n 1 -o verbose
-- Logs begin at Thu 2016-08-18 23:09:37 CEST, end at Mon 2017-10-09 18:18:50 CEST. --
Mon 2017-10-09 18:15:52.649028 CEST [s=89a2cc877fdf4dafb2269a7631afedad;i=14d7;b=4c7e7adcba0c45b69d612857270716d3;m=137592e75e;t=55b1f81298605;x=c3c9b57b28c9490e]
    _CMDLINE=/usr/lib/systemd/systemd --switched-root --system --deserialize 25
    MESSAGE=ip-accounting-test.service: Received 49.6K IP traffic, sent 49.6K IP traffic

The interesting fields of this log message are of course IP_METRIC_INGRESS_BYTES=, IP_METRIC_INGRESS_PACKETS=, IP_METRIC_EGRESS_BYTES=, IP_METRIC_EGRESS_PACKETS= that show the consumed data.

The log message carries a message ID that may be used to quickly search for all such resource log messages (ae8f7b866b0347b9af31fe1c80b127c0). We can combine a search term for messages of this ID with journalctl's -u switch to quickly find out about the resource usage of any invocation of a specific service. Let's try:

# journalctl -u ip-accounting-test MESSAGE_ID=ae8f7b866b0347b9af31fe1c80b127c0
-- Logs begin at Thu 2016-08-18 23:09:37 CEST, end at Mon 2017-10-09 18:25:27 CEST. --
Okt 09 18:15:52 sigma systemd[1]: ip-accounting-test.service: Received 49.6K IP traffic, sent 49.6K IP traffic

Of course, the output above shows only one message at the moment, since we started the service only once, but a new one will appear every time you start and stop it again.

The IP accounting logic is also hooked up with systemd-run, which is useful for transiently running a command as systemd service with IP accounting turned on. Let's try it:

# systemd-run -p IPAccounting=yes --wait wget
Running as unit: run-u2761.service
Finished with result: success
Main processes terminated with: code=exited/status=0
Service runtime: 878ms
IP traffic received: 231.0K
IP traffic sent: 3.7K

This uses wget to download the PDF version of the 2nd day schedule of everybody's favorite Linux user-space conference All Systems Go! 2017 (BTW, have you already booked your ticket? We are very close to selling out, be quick!). The IP traffic this command generated was 231K ingress and 4K egress. In the systemd-run command line two parameters are important. First of all, we use -p IPAccounting=yes to turn on IP accounting for the transient service (as above). And secondly we use --wait to tell systemd-run to wait for the service to exit. If --wait is used, systemd-run will also show you various statistics about the service that just ran and terminated, including the IP statistics you are seeing if IP accounting has been turned on.

It's fun to combine this sort of IP accounting with interactive transient units. Let's try that:

# systemd-run -p IPAccounting=1 -t /bin/sh
Running as unit: run-u2779.service
Press ^] three times within 1s to disconnect TTY.
sh-4.4# dnf update
sh-4.4# dnf install firefox
sh-4.4# exit
Finished with result: success
Main processes terminated with: code=exited/status=0
Service runtime: 5.297s
IP traffic received: …B
IP traffic sent: …B

This uses systemd-run's --pty switch (or short: -t), which opens an interactive pseudo-TTY connection to the invoked service process, which is a bourne shell in this case. Doing this means we have a full, comprehensive shell with job control and everything. Since the shell is running as part of a service with IP accounting turned on, all IP traffic we generate or receive will be accounted for. And as soon as we exit the shell, we'll see what it consumed. (For the sake of brevity I actually didn't paste the whole output above, but truncated core parts. Try it out for yourself, if you want to see the output in full.)

Sometimes it might make sense to turn on IP accounting for a unit that is already running. For that, use systemctl set-property foobar.service IPAccounting=yes, which will instantly turn on accounting for it. Note that it won't count retroactively though: only the traffic sent/received after the point in time you turned it on will be collected. You may turn off accounting for the unit with the same command.

Of course, sometimes it's interesting to collect IP accounting data for all services, and turning on IPAccounting=yes in every single unit is cumbersome. To deal with that there's a global option DefaultIPAccounting= available which can be set in /etc/systemd/system.conf.

IP Access Lists

So much about IP accounting. Let's now have a look at IP access control with systemd 235. As mentioned above, the two new unit file settings, IPAddressAllow= and IPAddressDeny= maybe be used for that. They operate in the following way:

  1. If the source address of an incoming packet or the destination address of an outgoing packet matches one of the IP addresses/network masks in the relevant unit's IPAddressAllow= setting then it will be allowed to go through.

  2. Otherwise, if a packet matches an IPAddressDeny= entry configured for the service it is dropped.

  3. If the packet matches neither of the above it is allowed to go through.

Or in other words, IPAddressDeny= implements a blacklist, but IPAddressAllow= takes precedence.

Let's try that out. Let's modify our last example above in order to get a transient service running an interactive shell which has such an access list set:

# systemd-run -p IPAddressDeny=any -p IPAddressAllow= -p IPAddressAllow= -t /bin/sh
Running as unit: run-u2850.service
Press ^] three times within 1s to disconnect TTY.
sh-4.4# ping -c1
PING ( 56(84) bytes of data.
64 bytes from icmp_seq=1 ttl=59 time=27.9 ms

--- ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 27.957/27.957/27.957/0.000 ms
sh-4.4# ping -c1
PING ( 56(84) bytes of data.
ping: sendmsg: Operation not permitted
--- ping statistics ---
1 packets transmitted, 0 received, 100% packet loss, time 0ms
sh-4.4# ping -c1
PING ( 56(84) bytes of data.
64 bytes from icmp_seq=1 ttl=64 time=0.116 ms

--- ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.116/0.116/0.116/0.000 ms
sh-4.4# exit

The access list we set up uses IPAddressDeny=any in order to define an IP white-list: all traffic will be prohibited for the session, except for what is explicitly white-listed. In this command line, we white-listed two address prefixes: (with no explicit network mask, which means the mask with all bits turned on is implied, i.e. /32), and Thus, the service can communicate with Google's DNS server and everything on the local loop-back, but nothing else. The commands run in this interactive shell show this: First we try pinging which happily responds. Then, we try to ping (that's Google's other DNS server, but excluded from this white-list), and as we see it is immediately refused with an Operation not permitted error. As last step we ping (which is on the local loop-back), and we see it works fine again, as expected.

In the example above we used IPAddressDeny=any. The any identifier is a shortcut for writing ::/0, i.e. it's a shortcut for everything, on both IPv4 and IPv6. A number of other such shortcuts exist. For example, instead of spelling out we could also have used the more descriptive shortcut localhost which is expanded to ::1/128, i.e. everything on the local loopback device, on both IPv4 and IPv6.

Being able to configure IP access lists individually for each unit is pretty nice already. However, typically one wants to configure this comprehensively, not just for individual units, but for a set of units in one go or even the system as a whole. In systemd, that's possible by making use of .slice units (for those who don't know systemd that well, slice units are a concept for organizing services in hierarchical tree for the purpose of resource management): the IP access list in effect for a unit is the combination of the individual IP access lists configured for the unit itself and those of all slice units it is contained in.

By default, system services are assigned to system.slice, which in turn is a child of the root slice -.slice. Either of these two slice units are hence suitable for locking down all system services at once. If an access list is configured on system.slice it will only apply to system services, however, if configured on -.slice it will apply to all user processes of the system, including all user session processes (i.e. which are by default assigned to user.slice which is a child of -.slice) in addition to the system services.

Let's make use of this:

# systemctl set-property system.slice IPAddressDeny=any IPAddressAllow=localhost
# systemctl set-property apache.service IPAddressAllow=

The two commands above are a very powerful way to first turn off all IP communication for all system services (with the exception of loop-back traffic), followed by an explicit white-listing of (which could refer to the local company network, you get the idea) but only for the Apache service.


After playing around a bit with this, let's talk about use-cases. Here are a few ideas:

  1. The IP access list logic can in many ways provide a more modern replacement for the venerable TCP Wrapper, but unlike it it applies to all IP sockets of a service unconditionally, and requires no explicit support in any way in the service's code: no patching required. On the other hand, TCP wrappers have a number of features this scheme cannot cover, most importantly systemd's IP access lists operate solely on the level of IP addresses and network masks, there is no way to configure access by DNS name (though quite frankly, that is a very dubious feature anyway, as doing networking — unsecured networking even – in order to restrict networking sounds quite questionable, at least to me).

  2. It can also replace (or augment) some facets of IP firewalling, i.e. Linux NetFilter/iptables. Right now, systemd's access lists are of course a lot more minimal than NetFilter, but they have one major benefit: they understand the service concept, and thus are a lot more context-aware than NetFilter. Classic firewalls, such as NetFilter, derive most service context from the IP port number alone, but we live in a world where IP port numbers are a lot more dynamic than they used to be. As one example, a BitTorrent client or server may use any IP port it likes for its file transfer, and writing IP firewalling rules matching that precisely is hence hard. With the systemd IP access list implementing this is easy: just set the list for your BitTorrent service unit, and all is good.

    Let me stress though that you should be careful when comparing NetFilter with systemd's IP address list logic, it's really like comparing apples and oranges: to start with, the IP address list logic has a clearly local focus, it only knows what a local service is and manages access of it. NetFilter on the other hand may run on border gateways, at a point where the traffic flowing through is pure IP, carrying no information about a systemd unit concept or anything like that.

  3. It's a simple way to lock down distribution/vendor supplied system services by default. For example, if you ship a service that you know never needs to access the network, then simply set IPAddressDeny=any (possibly combined with IPAddressAllow=localhost) for it, and it will live in a very tight networking sand-box it cannot escape from. systemd itself makes use of this for a number of its services by default now. For example, the logging service systemd-journald.service, the login manager systemd-logind or the core-dump processing unit systemd-coredump@.service all have such a rule set out-of-the-box, because we know that neither of these services should be able to access the network, under any circumstances.

  4. Because the IP access list logic can be combined with transient units, it can be used to quickly and effectively sandbox arbitrary commands, and even include them in shell pipelines and such. For example, let's say we don't trust our curl implementation (maybe it got modified locally by a hacker, and phones home?), but want to use it anyway to download the the slides of my most recent casync talk in order to print it, but want to make sure it doesn't connect anywhere except where we tell it to (and to make this even more fun, let's minimize privileges further, by setting DynamicUser=yes):

    # systemd-resolve
    -- Information acquired via protocol DNS in 2.8ms.
    -- Data is authenticated: no
    # systemd-run --pipe -p IPAddressDeny=any \
                         -p IPAddressAllow= \
                         -p IPAddressAllow=2a01:238:43ed:c300:10c3:bcf3:3266:da74 \
                         -p DynamicUser=yes \
                         curl | lp

So much about use-cases. This is by no means a comprehensive list of what you can do with it, after all both IP accounting and IP access lists are very generic concepts. But I do hope the above inspires your fantasy.

What does that mean for packagers?

IP accounting and IP access control are primarily concepts for the local administrator. However, As suggested above, it's a very good idea to ship services that by design have no network-facing functionality with an access list of IPAddressDeny=any (and possibly IPAddressAllow=localhost), in order to improve the out-of-the-box security of our systems.

An option for security-minded distributions might be a more radical approach: ship the system with -.slice or system.slice configured to IPAddressDeny=any by default, and ask the administrator to punch holes into that for each network facing service with systemctl set-property … IPAddressAllow=…. But of course, that's only an option for distributions willing to break compatibility with what was before.


A couple of additional notes:

  1. IP accounting and access lists may be mixed with socket activation. In this case, it's a good idea to configure access lists and accounting for both the socket unit that activates and the service unit that is activated, as both units maintain fully separate settings. Note that IP accounting and access lists configured on the socket unit applies to all sockets created on behalf of that unit, and even if these sockets are passed on to the activated services, they will still remain in effect and belong to the socket unit. This also means that IP traffic done on such sockets will be accounted to the socket unit, not the service unit. The fact that IP access lists are maintained separately for the kernel sockets created on behalf of the socket unit and for the kernel sockets created by the service code itself enables some interesting uses. For example, it's possible to set a relatively open access list on the socket unit, but a very restrictive access list on the service unit, thus making the sockets configured through the socket unit the only way in and out of the service.

  2. systemd's IP accounting and access lists apply to IP sockets only, not to sockets of any other address families. That also means that AF_PACKET (i.e. raw) sockets are not covered. This means it's a good idea to combine IP access lists with RestrictAddressFamilies=AF_UNIX AF_INET AF_INET6 in order to lock this down.

  3. You may wonder if the per-unit resource log message and systemd-run --wait may also show you details about other types or resources consumed by a service. The answer is yes: if you turn on CPUAccounting= for a service, you'll also see a summary of consumed CPU time in the log message and the command output. And we are planning to hook-up IOAccounting= the same way too, soon.

  4. Note that IP accounting and access lists aren't entirely free. systemd inserts an eBPF program into the IP pipeline to make this functionality work. However, eBPF execution has been optimized for speed in the last kernel versions already, and given that it currently is in the focus of interest to many I'd expect to be optimized even further, so that the cost for enabling these features will be negligible, if it isn't already.

  5. IP accounting is currently not recursive. That means you cannot use a slice unit to join the accounting of multiple units into one. This is something we definitely want to add, but requires some more kernel work first.

  6. You might wonder how the PrivateNetwork= setting relates to IPAccessDeny=any. Superficially they have similar effects: they make the network unavailable to services. However, looking more closely there are a number of differences. PrivateNetwork= is implemented using Linux network name-spaces. As such it entirely detaches all networking of a service from the host, including non-IP networking. It does so by creating a private little environment the service lives in where communication with itself is still allowed though. In addition using the JoinsNamespaceOf= dependency additional services may be added to the same environment, thus permitting communication with each other but not with anything outside of this group. IPAddressAllow= and IPAddressDeny= are much less invasive. First of all they apply to IP networking only, and can match against specific IP addresses. A service running with PrivateNetwork= turned off but IPAddressDeny=any turned on, may enumerate the network interfaces and their IP configured even though it cannot actually do any IP communication. On the other hand if you turn on PrivateNetwork= all network interfaces besides lo disappear. Long story short: depending on your use-case one, the other, both or neither might be suitable for sand-boxing of your service. If possible I'd always turn on both, for best security, and that's what we do for all of systemd's own long-running services.

And that's all for now. Have fun with per-unit IP accounting and access lists!

fwupd hits 1.0.0

Today I released fwupd version 1.0.0, a version number most Open Source projects seldom reach. Unusually it bumps the soname so any applications that link against libfwupd will need to be rebuilt. The reason for bumping is that we removed a lot of the cruft we’ve picked up over the couple of years since we started the project, and also took the opportunity to rename some public interfaces that are now used differently to how they were envisaged. Since we started the project, we’ve basically re-architected the way the daemon works, re-imagined how the metadata is downloaded and managed, and changed core ways we’ve done the upgrades themselves. It’s no surprise that removing all that crufty code makes the core easier to understand and maintain. I’m intending to support the 0_9_X branch for a long time, as that’s what’s going to stay in Fedora 26 and the upcoming Fedora 27.

Since we’ve started we now support 72 different kinds of hardware, with support for another dozen-or-so currently being worked on. Lots of vendors are now either using the LVFS to distribute firmware, or are testing with one or two devices in secret. Although we have 10 (!) different ways of applying firmware already, vendors are slowly either switching to a more standard mechanism for new products (UpdateCapsule/DFU/Redfish) or building custom plugins for fwupd to update existing hardware.

Every month 165,000+ devices get updated using fwupd using the firmware on the LVFS; possibly more as people using corporate mirrors and caching servers don’t show up in the stats. Since we started this project there are now at least 600,000 items of hardware with new firmware. Many people have updated firmware, fixing bugs and solving security issues without having to understand all the horrible details involved.

I guess I should say thanks; to all the people both uploading firmware, and the people using, testing, and reporting bugs. Dell have been a huge supporter since the very early days, and now smaller companies and giants like Logitech are also supporting the project. Red Hat have given me the time and resources that I need to build something as complicated and political as shared infrastructure like this. There is literally no other company on the planet that I would rather work for.

So, go build fwupd 1.0.0 in your distro development branch and report any problems. 1.0.1 will follow soon with fixes I’m sure, and hopefully we can make some more vendor announcements in the near future. There are a few big vendors working on things in secret that I’m sure you’ll all know :)