October 22, 2019

Join GNOME in our fight against a patent troll

A photo of a large group of people sitting and standing on steps, with GNOME balloons and signs in the background.

A month ago, GNOME was hit by a patent troll for developing the Shotwell image management application. It’s the first time a free software project has been targeted in this way, but we worry it won’t be the last. Rothschild Patent Imaging, LLC offered to let us settle for a high five figure amount, for which they would drop the case and give us a license to carry on developing Shotwell. This would have been simple to do so; it would have caused less work, cost less money, and provided the Foundation a lot less stress. But it also would be wrong. Agreeing to this would leave this patent live, and allow this to be used as a weapon against countless others. We will stand firm against this baseless attack, not just for GNOME and Shotwell, but for all free and open source software projects.

For these reasons, GNOME Foundation Executive Director Neil McGovern instructed our legal counsel at Shearman & Sterling to file three papers with the court in California.

First, a motion to dismiss the case outright. We don’t believe that this is a valid patent, or that software can or should be able to be patented in this way. We want to make sure that this patent isn’t used against anyone else, ever.

Second, our answer to the claim. We don’t believe that there is a case GNOME needs to answer to. We want to show that the use of Shotwell, and free software in general, isn’t affected by this patent.

Thirdly, our counterclaim. We want to make sure that this isn’t just dropped when Rothschild realizes we’re going to fight this.

We want to send a message to all software patent trolls out there — we will fight your suit, we will win, and we will have your patent invalidated. To do this, we need your help. Please help support the GNOME Foundation in sending a message that patent trolls should never target free software by making a donation to the GNOME Patent Troll Defense Fund. If you’re interested in keeping up with the news on this case, subscribe to our Legal Updates email list.

GUADEC 2017 Group Photo” by Jonathan Kang licensed CC-BY-SA

GNOME, and Free Software Is Under Attack

A month ago, GNOME was hit by a patent troll. We’re fighting, but need some money to fund the legal defense, and counterclaim. I just donated, and if you use or develop free software you should too.

October 20, 2019

Five-or-More Modernisation: It's a Wrap

As probably most of you already know, or recently found out, at the beginning of this week the GSoC coding period officially ended, and it is time for us, GSoC students, to submit our final evaluations and the results we achieved thus far. This blog post, as you can probably tell from the title, will be a summary of all of the work I put into modernising Five or More throughout the summer months.

My main task was rewriting Five or More in Vala since this simple and fun game did not find its way to the list of those included in the Games Modernisation Initiative. This fun, strategy game consists of aligning, as often as possible, five or more objects of the same shape and color, to make them disappear and score points.

Besides the Vala rewrite, there were also some other tasks included, such as migrating to Meson and dropping autotools, as well as keeping the view and logic separated and updating the UI to make this game more relatable for the public and more fresh-looking. However, after thoroughly discussing the details with my mentor, Robert Roth (IRC: evfool), more emphasis was placed upon rewriting the code to Vala, since the GSoC program is specifically designed for software development. However, slight UI modifications were integrated as to match the visual layout guidelines.

Some of the tasks, namely porting to gettext, porting to Meson and dropping autotools happened earlier on, during the pre-GSoC time frame, in the attempt to familiarise myself with the project and the tasks that would come with it.

Afterward, I started with the easier tasks and advanced towards the more complex ones. At first, I ported the application window and the preferences menu and added callbacks for each of the preferences buttons. I then continued with porting the preview widget displaying the next objects to be rendered on the game board.

Next, it was time to channel my attention towards porting the game board, handling board colour changes and resizing the board alongside the window resize, by using the grid frame functionality inside the libgnome-games-support library.

The following target was implementing the actual gameplay logic, which consisted of a pathfinding algorithm based on A*, erasing all objects of the same shape and colour aligned in a row, column or diagonal from the board, if there were either five or more than five, and adding to the score whenever that happened. I also made the object movement possible with both clicking and keyboard keys, for more ease of use.

Finishing touches included adding the high scores tables via the libgnome-games-support library, displaying an option to change the theme of the game, adding a grid to be able to make out easier the actual cells in which different shaped and coloured objects should reside, as well as updating some information contained by the help pages.

Some features, however, could not be done during the GSoC period. These included handling raster game themes, making significant UI updates, as well as some other extra-features I wanted to add, which were not part of the original project description such as gamepad support or sound effects. The fist feature mentioned in this list, handling raster images was decided upon skipping as a suggestion from my mentor, as the existing raster themes were low-resolution and did not scale well to large size and HiDPI displays.

For easier reading, I decided to also include this list of the tasks successfully done during GSoC:

  • Ported from autotools to Meson
  • Ported application window and preferences menu
  • Added callbacks for all buttons
  • Ported widget of next objects to be rendered on the game board
  • Handled the resize and colour change of the game board
  • Implemented the gameplay logic (pathfinding, matching objects and calculating scores)
  • Made gameplay accessible for both mouse clicks and keyboard controls
  • Added high scores via the libgnome-games-support library
  • Handling vector image theme changing
  • Made slight modifications to the UI to serve as a template and changed the spacing as recommended in the visual layout guidelines.
  • Updated documentation.

All code is available by accessing this merge request link.

Five-or-More Modernisation: Now You Can Properly Play It

As Google Summer of Code is officially drawing to an end, all of my attention was focused towards making the Five or More Vala version feature-complete. As you probably already know from my previous blog post, the game was somehow playable at that time, but it was missing some of the key features included in the old version.

So what’s new this time? First and foremost, you can surely notice the game board now sports a grid, which wasn’t there until now. On the same note, there are also animations used for clicking a piece on the board, for an improved gaming experience. For further accessibility, some header bar hints are available at different stages in the game: at the start of any new game, at the end of each game, as well as whenever there is no clear path between the initial position and the cell indicated by the user for the current move.


Overall final game look


By using libgnome-games-support, I was able to implement a high scores table, which gets updated each time the player gets a score ranging in the top 10 highest scores for the chosen board size. The high scores for each of all of the three categories can also be viewed as showed in the screencast below. Also, you can see I have done quite my fair share of testing the functionalities and assuring everything worked as planned 😄.


High scores


Further changes include theme changing, although this momentarily only works for the vector themes available in Five or More, namely the ball and shape themes, as implementation for raster images is not fully functional as of this moment.


Changing theme


Also, now window size is being saved in between runs, so each time the game starts it will take into consideration the last window size settings, including wether the window was full screened or not.

As for the most exciting change revealed in this blog post, it concerns playing Five or More using keyboard controls. Basically, the user can play the game by navigating with the keyboard arrows, the home, end, page up, page down and space, enter or return keys, as described in the wikipage section for using the keyboard.


Playing Five-or-More usig keyboard controls


If you have been following my journey with GSoC up closely, you probablly remember me mentioning something about extra-features in my first blog post, such as adding gamepad support, sounds, or making some design changes. I need to say I feel optimistic about getting some of these features done in the weeks to come, post GSoC. I feel the ground has been already laid down somehow for gamepad support by adding keyboard controls. Also, Five or More has already undergone some slight design changes, such as widget spacing and alignment.

Disney+ streaming uses draconian DRM, avoid

First of all, as always my opinions are my own, not those of my employer.

Since I have 2 children I was happy to learn that the Netherlands would be one of the first countries to get Disney+ streaming.

So I subscribed for the testing period, problem all devices in my home run Fedora. I started up Firefox and was greeted with an "Error Code 83", next I tried Chrome, same thing.

So I mailed the Disney helpdesk about this, explaining how Linux works fine with Netflix, AmazonPrime video and even the web-app from my local cable provider. They promised to get back to me in 24 hours, the eventually got back to me in about a week. They wrote: "We are familiar with Error 83. This often happens if you want to play Disney + via the web browser or certain devices. Our IT department working hard to solve this. In the meantime, I want to advise you to watch Disney + via the app on a phone or tablet. If this error code still occurs in a few days, you can check the help center ..." this was on September 23th.

So I thought, ok they are working on this lets give them a few days. It is almost a month later now and nothing has changed. Their so called help-center does not even know about "Error Code 83" even though the internet is full of people experiencing this. Note that this error also happens a lot on other platforms, it is not just Linux.

Someone on tweakers.net has done some digging and this is a Widevine error: "the response is: {"errors":[{"code":"platform-verification-failed","description":"Platform verification status incompatible with security level"}]}". Widevine has 3 security levels and many devices, including desktop Linux and many Android devices only support the lowest security setting (software encryption only). In this case e.g. Netflix will not offer full HD or 4k resolutions, but otherwise everything works fine, which is a balance between DRM and usability which I can accept. Disney+ OTOH seems to have the drm features kranked up to maximum draconian settings and simply will not work on a lot of android devices, nor on Chromebooks, nor on desktop Linux.

So if you care about Linux in any way, please do not subscribe to Disney+, instead send them a message letting them know that you are boycotting them until they get their Linux support in order.

October 18, 2019

GNOME Shell Hackfest 2019

This week, I have attended the GNOME Shell Hackfest 2019 held in Leidschendam, The Netherlands. It was a fantastic event, in a fantastic city! The list of attendees was composed of key members of the community, so we managed to get a lot done — a high amount of achievements for only three days of hackfest, in fact.

Mutter

My personal interests in this hackfest floated around Mutter more than GNOME Shell. In fact, we managed to either review, merge, or discuss a few things interesting points:

  • Multiple frame clocks for multiple monitors; Jonas and Carlos talked about event delivery and painting routines, and my small contribution to the discussion was about the requirements for moving actors between monitors and their corresponding frame clocks. Looks like we managed to get a pretty solid picture of what should happen next!
  • Carlos opened merge requests for his ClutterSeat work, and it looks promising.
  • Landed a couple of improvements to geometric picking.
  • Carlos, Jonas and I are now officially maintainers of Mutter, in addition to Florian!

Some of my own work also had a share of love:

  • My graphene branch was merged! I’ll continue the CoglMatrix → graphene_matrix_t transition, but that’s a huge step already. Not only graphene allows us to offload a lot from Cogl, but in the future it will probably play an important role in reducing CPU usage.
  • Another cool merge request making CoglJournal more useful was also merged. Though in the future, we might as well drop the journal entirely due to paint nodes.
  • During the flight back, I managed to plot the full transition to paint nodes in Clutter, and actually write about a third of the code for it. It went all the way from splitting pick and paint code paths, to introducing new types of paint nodes, to post-transition plans such as paint node caching, diffing, etc. It will be a massive change, but the possibilities it will bring are well worth it. For GNOME 3.36, I expect to land at the very least half of what’s necessary for the paint node transition to happen.

GNOME Shell

During the past few months, I found myself involved in a couple of very big changes in GNOME Shell: the new lock screen, and a bigger design review of the overview.

What started as an attempt to upstream part of Endless OS features (Drag n’ Drop of icons in custom positions) triggered a bigger change in how other elements of the Shell are places, and how they behave. More on this GNOME Shell front will be shared later; right now, it is too much of a moving target, and any exposition of what’s going on would only be misleading.

One small nicety that sparked during the hackfest was the wiggling effect of failed password attempts. It was a 30-minute hack that turned into a very nice effect, designers enjoyed and polished a few parameters, it was merged and will be part of 3.36. Nice!

Conclusion

This hackfest was very productive and I believe it had a big impact, both in short and long term plans. I’d like to appreciate Carlos Garnacho for all his work on organizing the event, and Hans de Goede for being our host. The hackfest was absolutely smooth, and we could focus on what matters because of your work!

Letting Birds scooters fly free

(Note: These issues were disclosed to Bird, and they tell me that fixes have rolled out. I haven't independently verified)

Bird produce a range of rental scooters that are available in multiple markets. With the exception of the Bird Zero[1], all their scooters share a common control board described in FCC filings. The board contains three primary components - a Nordic NRF52 Bluetooth controller, an STM32 SoC and a Quectel EC21-V modem. The Bluetooth and modem are both attached to the STM32 over serial and have no direct control over the rest of the scooter. The STM32 is tied to the scooter's engine control unit and lights, and also receives input from the throttle (and, on some scooters, the brakes).

The pads labeled TP7-TP11 near the underside of the STM32 and the pads labeled TP1-TP5 near the underside of the NRF52 provide Serial Wire Debug, although confusingly the data and clock pins are the opposite way around between the STM and the NRF. Hooking this up via an STLink and using OpenOCD allows dumping of the firmware from both chips, which is where the fun begins. Running strings over the firmware from the STM32 revealed "Set mode to Free Drive Mode". Challenge accepted.

Working back from the code that printed that, it was clear that commands could be delivered to the STM from the Bluetooth controller. The Nordic NRF52 parts are an interesting design - like the STM, they have an ARM Cortex-M microcontroller core. Their firmware is split into two halves, one the low level Bluetooth code and the other application code. They provide an SDK for writing the application code, and working through Ghidra made it clear that the majority of the application firmware on this chip was just SDK code. That made it easier to find the actual functionality, which was just listening for writes to a specific BLE attribute and then hitting a switch statement depending on what was sent. Most of these commands just got passed over the wire to the STM, so it seemed simple enough to just send the "Free drive mode" command to the Bluetooth controller, have it pass that on to the STM and win. Obviously, though, things weren't so easy.

It turned out that passing most of the interesting commands on to the STM was conditional on a variable being set, and the code path that hit that variable had some impressively complicated looking code. Fortunately, I got lucky - the code referenced a bunch of data, and searching for some of the values in that data revealed that they were the AES S-box values. Enabling the full set of commands required you to send an encrypted command to the scooter, which would then decrypt it and verify that the cleartext contained a specific value. Implementing this would be straightforward as long as I knew the key.

Most AES keys are 128 bits, or 16 bytes. Digging through the code revealed 8 bytes worth of key fairly quickly, but the other 8 bytes were less obvious. I finally figured out that 4 more bytes were the value of another Bluetooth variable which could be simply read out by a client. The final 4 bytes were more confusing, because all the evidence made no sense. It looked like it came from passing the scooter serial number to atoi(), which converts an ASCII representation of a number to an integer. But this seemed wrong, because atoi() stops at the first non-numeric value and the scooter serial numbers all started with a letter[2]. It turned out that I was overthinking it and for the vast majority of scooters in the fleet, this section of the key was always "0".

At that point I had everything I need to write a simple app to unlock the scooters, and it worked! For about 2 minutes, at which point the network would notice that the scooter was unlocked when it should be locked and sent a lock command to force disable the scooter again. Ah well.

So, what else could I do? The next thing I tried was just modifying some STM firmware and flashing it onto a board. It still booted, indicating that there was no sort of verified boot process. Remember what I mentioned about the throttle being hooked through the STM32's analogue to digital converters[3]? A bit of hacking later and I had a board that would appear to work normally, but about a minute after starting the ride would cut the throttle. Alternative options are left as an exercise for the reader.

Finally, there was the component I hadn't really looked at yet. The Quectel modem actually contains its own application processor that runs Linux, making it significantly more powerful than any of the chips actually running the scooter application[4]. The STM communicates with the modem over serial, sending it an AT command asking it to make an SSL connection to a remote endpoint. It then uses further AT commands to send data over this SSL connection, allowing it to talk to the internet without having any sort of IP stack. Figuring out just what was going over this connection was made slightly difficult by virtue of all the debug functionality having been ripped out of the STM's firmware, so in the end I took a more brute force approach - I identified the address of the function that sends data to the modem, hooked up OpenOCD to the SWD pins on the STM, ran OpenOCD's gdb stub, attached gdb, set a breakpoint for that function and then dumped the arguments being passed to that function. A couple of minutes later and I had a full transaction between the scooter and the remote.

The scooter authenticates against the remote endpoint by sending its serial number and IMEI. You need to send both, but the IMEI didn't seem to need to be associated with the serial number at all. New connections seemed to take precedence over existing connections, so it would be simple to just pretend to be every scooter and hijack all the connections, resulting in scooter unlock commands being sent to you rather than to the scooter or allowing someone to send fake GPS data and make it impossible for users to find scooters.

In summary: Secrets that are stored on hardware that attackers can run arbitrary code on probably aren't secret, not having verified boot on safety critical components isn't ideal, devices should have meaningful cryptographic identity when authenticating against a remote endpoint.

Bird responded quickly to my reports, accepted my 90 day disclosure period and didn't threaten to sue me at any point in the process, so good work Bird.

(Hey scooter companies I will absolutely accept gifts of interesting hardware in return for a cursory security audit)

[1] And some very early M365 scooters
[2] The M365 scooters that Bird originally deployed did have numeric serial numbers, but they were 6 characters of type code followed by a / followed by the actual serial number - the number of type codes was very constrained and atoi() would terminate at the / so this was still not a large keyspace
[3] Interestingly, Lime made a different design choice here and plumb the controls directly through to the engine control unit without the application processor having any involvement
[4] Lime run their entire software stack on the modem's application processor, but because of [3] they don't have any realtime requirements so this is more straightforward

comment count unavailable comments

g_array_steal() and g_ptr_array_steal() in GLib 2.63.1

Another set of new APIs in the upcoming GLib 2.63.1 release allow you to steal all the contents of a GArray, GPtrArray or GByteArray, and continue using the array container to add more contents to in future.

This is work by Paolo Bonzini and Emmanuel Fleury, and will be available in the soon-to-be-released 2.63.1 release.

Here’s a quick example using GPtrArray — usage is similar in GArray and GByteArray:

g_autoptr(GPtrArray) chunk_buffer = g_ptr_array_new_with_free_func (g_bytes_unref);

/* Some part of your application appends a number of chunks to the pointer array. */
g_ptr_array_add (chunk_buffer, g_bytes_new_static ("hello", 5));
g_ptr_array_add (chunk_buffer, g_bytes_new_static ("world", 5));

…

/* Periodically, the chunks need to be sent as an array-and-length to some other part of the program. */
GBytes **chunks;
gsize n_chunks;

chunks = g_ptr_array_steal (chunk_buffer, &n_chunks);
for (gsize i = 0; i < n_chunks; i++)
  {
    /* Do something with each chunk here, and then free them, since g_ptr_array_steal() transfers ownership of all the elements and the array to the caller. */
    …

    g_bytes_unref (chunks[i]);
  }

g_free (chunks);

/* After calling g_ptr_array_steal(), the pointer array can be reused for the next set of chunks. */
g_assert (chunk_buffer->len == 0);

October 17, 2019

libinput and tablet pad keys

Upcoming in libinput 1.15 is a small feature to support Wacom tablets a tiny bit better. If you look at the higher-end devices in Wacom's range, e.g. the Cintiq 27QHD you'll notice that at the top right of the device are three hardware-buttons with icons. Those buttons are intended to open the config panel, the on-screen display or the virtual keyboard. They've been around for a few years and supported in the kernel for a few releases. But in userspace, they events from those keys were ignored, casted out in the wild before eventually running out of electrons and succumbing to misery. Well, that's all changing now with a new interface being added to libinput to forward those events.

Step back a second and let's look at the tablet interfaces. We have one for tablet tools (styli) and one for tablet pads. In the latter, we have events for rings, strips and buttons. The latter are simply numerically ordered, so button 1 is simply button 1 with no special meaning. Anything more specific needs to be handled by the compositor/client side which is responsible for assigning e.g. keyboard shortcuts to those buttons.

The special keys however are different, they have a specific function indicated by the icon on the key itself. So libinput 1.15 adds a new event type for tablet pad keys. The events look quite similar to the button events but they have a linux/input-event-codes.h specific button code that indicates what they are. So the compositor can start the OSD, or control panel, or whatever directly without any further configuration required.

This interface hasn't been merged yet, it's waiting for the linux kernel 5.4 release which has a few kernel-level fixes for those keys.

libinput and button scrolling locks

For a few years now, libinput has provided button scrolling. Holding a designated button down and moving the device up/down or left/right creates the matching scroll events. We enable this behaviour by default on some devices (e.g. trackpoints) but it's available on mice and some other devices. Users can change the button that triggers it, e.g. assign it to the right button. There are of course a couple of special corner cases to make sure you can still click that button normally but as I said, all this has been available for quite some time now.

New in libinput 1.15 is the button lock feature. The button lock removes the need to hold the button down while scrolling. When the button lock is enabled, a single button click (i.e. press and release) of that button holds that button logically down for scrolling and any subsequent movement by the device is translated to scroll events. A second button click releases that button lock and the device goes back to normal movement. That's basically it, though there are some extra checks to make sure the button can still be used for normal clicking (you will need to double-click for a single logical click now though).

This is primarily an accessibility feature and is likely to find it's way into the GUI tools under the accessibility headers.

Riddle me this

Found this today while playing around, thought people might enjoy this riddle.

$> echo test.c
typedef int foo;
int main()
{
  foo foo = 1;
  return (foo) +0;
}
$> gcc -Wall -o test test.c && ./test && echo $?

What does this print?

  1. 0
  2. 1
  3. Some compilation warnings, then 0.
  4. Some compilation warnings, then 1.
  5. It doesn’t compile.

I’ll put an answer in the comments.

Gnome-shell Hackfest 2019 – Day 3

As promised, some late notes on the 3rd and last day of the gnome-shell hackfest, so yesterday!

Some highlights from my partial view:

  • We had a mind blowing in depth discussion about the per-crtc frame clocks idea that’s been floating around for a while. What started as “light” before-bedtime conversation the previous night continued the day after straining our neurons in front of a whiteboard. We came out wiser nonetheless, and have a much more concrete idea about how should it work.
  • Georges updated his merge request to replace Cogl structs with graphene ones. This now passes CI and was merged \o/
  • Much patch review happened in place, and some other pretty notable refactors and cleanups were merged.
  • The evening was more rushed than usual, with some people leaving already. The general feeling seemed good!
  • In my personal opinion the outcome was pretty good too. There’s been progress at multiple levels and new ideas sparked, you should look forward to posts from others :). It was also great to put a face to some IRC nicks, and meet again all the familiar ones.

Kudos to the RevSpace members and especially Hans, without them this hackfest couldn’t have happened.

October 16, 2019

Gnome-shell Hackfest 2019 – Day 2

Well, we are starting the 3rd and last day of this hackfest… I’ll write about yesterday, which probably means tomorrow I’ll blog about today :).

Some highlights of what I was able to participate/witness:

  • Roman Gilg of KDE fame came to the hackfest, it was a nice opportunity to discuss mixed DPI densities for X11/Xwayland clients. We first thought about having one server per pixel density, but later on we realized we might not be that far from actually isolating all X11 clients from each other, so why stop there.
  • The conversation drifted into other topics relevant to desktop interoperation. We did discuss about window activation and focus stealing prevention, this is a topic “fixed” in Gnome but in a private protocol. I had already a protocol draft around which was sent today to wayland-devel ML.
  • A plan was devised for what is left of Xwayland-on-demand, and an implementation is in progress.
  • The designers have been doing some exploration and research on how we interact with windows, the overview and the applications menu, and thinking about alternatives. At the end of the day they’ve demoed to us the direction they think we should take.

    I am very much not a designer and I don’t want to spoil their fine work here, so stay tuned for updates from them :).

  • As the social event, we had a very nice BBQ with some hackerspace members. Again kindly organized by Revspace.

Τι κάνεις FOSSCOMM 2019

Thanks to the sponsorship of Fedora, I was able to travel to Lamia, Greece from October 10 to October 14 to attend at FOSSCOMM (Free and Open Source Software Communities Meeting), the pan-Hellenic conference of free and open source software communities.

Things I did in the event:

1.- Set up a Fedora booth

I arranged the booth during the first hours when I arrived Lamia. The event registration started at 4:00 p.m. and thanks to the help of enthusiastic volunteers and Alex Angelo (I met him in GUADEC 2019), the booth was all ready to go since the first day of the event.

The Fedora project sent swags directly to the University of Central Greece, and I created my own handmade decoration. I used Fedora and GNOME ballons to have a nice booth 🙂 Thanks to the tools provided by the university I was able to finish what I had in mind:

2.- Spread up the Fedora word

When the students visited our Fedora booth, they were excited to take some Fedora gifts, especially the tattoo sticker. I was asking how many of them used Fedora, and most of them were using Ubuntu, Linux Mint, Kali Linux and Elementary OS. It was an opportunity to share the Fedora 30 edition and give the beginner’s guide that the Fedora community wrote in a little book. Most of them enjoyed taking photos with the Linux frame I did in Edinburgh 💙  Alex shared also his Linux knowledge in our Fedora booth.

3.- Do a keynote about Linux on Supercomputers

I was invited to the conference to do a talk about Linux in supercomputers. Only 9 out of 42 attendees were non-Linux users. However, I am so glad that they attended to know what is going on in the supercomputer world that uses Linux. Then, I started by asking questions about Linux in general, and some linuxers were able to answer part of the questions but not all of them. I have been told by professor Thanos that Greece has a supercomputer called Aris, as well as the students were aware about GPUs technologies. When I asked a question about GPUs, a female student answered correctly about the use of GPUs and that is why she won the t-shirt of the event I offered as a prize to the audience. You might see my entire talk in the live streaming video. 

4.- Do a workshop of GTK on C

I was planning to teach the use of the GTK library with C, Python, and Vala. However, because of the time and the preference of the attendees, we only worked with C. The workshop was supported by Alex Angelo who also traduced some of my expressions in Greek. I was flexible in using different Operating Systems such as Linux Mint, Ubuntu, Kubuntu among other distros. There were only two users that used Fedora. Almost half of the audience did not bring a laptop, and then I grouped in groups to work together. I enjoyed to see young students eager to learn, they took their own notes, and asked questions. You might see the video of the workshop that was recorded by the organizers.

My feelings about the event:

The agenda of the event was so interesting, I was quite sad to not attend because I had to take care of the booth, and most of the talks were done in Greek. As you can see in the pictures, there were a variety of technical talks in charge of women. I was impressed by Greek ladies because they are well prepared, most of them were self-taught in Linux and in trending technologies such as IoT, security, programming, Linux, and in bio-science.

Authorities supported this kind of Linux events and I think that was an important factor to have a successful event. Miss Catherine and Mister Thanos were pictured with minorities, women and kids were very excited to be part of FOSSCOMM 2019. Additionally, its local government also supported this event. Here a post in the magazine.

Greek people are warm and happy.  Thank you so much to everyone for the kindness!

Food for everyone

I was surprised by time and schedules, they started the journey every day at 8:00 am and the talks finished at 8p.m. The lunch break was set at 2:30p.m. and a local guy told me that just for breakfast they usually take a cup of coffee. We had a very delicious and consistent dinner on the first day of the event with the professors of the Informatics and Biology department of the University Central Greece. Free lunch and coffee breaks were served carefully to all. I enjoyed Greece food, we had a variety of salads and sweeties. 

Turistic places I visited

I only had a few hours before leaving Lamia, I had time to visit the castle and the museum where I learned more about the different ancient eras and legends of Greece.

Special Thanks

Thanks to Alex for being my local guide during the whole event! Thanks to Iris for the welcoming, to Argiris for the invitation and the t-shirt he promised me, and to Kath for being so nice in the thousand pictures we took and for the touristic guide and her help.

Thanks to Stathis who encouraged me to apply to FOSSCOMM, to each volunteer for the help they gave me and all the effort they did, I know that most of them live an hour and a half far from the university. Thanks again to Fedora for the travel sponsorship!

Libosinfo (Part I)

This is the first blog post of a series which will cover Libosinfo, what it is, who uses it, how it is used, how to manage it, and, finally, how to contribute to it.

A quick overview

Libosinfo is the operating system information database. As a project, it consists of three different parts, with the goal to provide a single place containing all the required information about an operating system in order to provision and manage it in a virtualized environment.

The project allows management applications to:

  • Automatically identify for which operating system an ISO image or an installation tree is intended to;

  • Find the download location of installable ISOs and LiveCDs images;

  • Find the location of installation trees;

  • Query the minimum, recommended, and maximum CPU / memory / disk resources for an operating system;

  • Query the hardware supported by an operating system;

  • Generate scripts suitable for automating “Server” and “Workstation” installations;

The library (libosinfo)

The library API is written in C, taking advantage of GLib and GObject. Thanks to GObject Introspection, the API is automatically available in all dynamic programming languages with bindings for GObject (JavaScript, Perl, Python, and Ruby). Auto-generated bindings for Vala are also provided.

As part of libosinfo, three tools are provided:

  • osinfo-detect: Used to detect an Operating System from a given ISO or installation tree.

  • osinfo-install-script: Used to generate a “Server” or “Workstation” install-script to perform automated installation of an Operating System;

  • osinfo-query: Used to query information from the database;

The database (osinfo-db)

The database is written in XML and it can either be consumed via libosinfo APIs or directly via management applications’ own code.

It contains information about the operating systems, devices, installation scripts, platforms, and datamaps (keyboard and language mappings for Windows and Linux OSes).

The database tools (osinfo-db-tools)

These are tools that can be used to manage the database, which is distributed as a tarball archive.

  • osinfo-db-import: Used to import an osinfo database archive;

  • osinfo-db-export: Used to export an osinfo database archive;

  • osinfo-db-validate: Used to validate the XML files in one of the osinfo database locations for compliance with the RNG schema.

  • osinfo-db-path: Used to report the paths associated with the standard database locations;

The consumers …

Libosinfo and osinfo-db have management applications as their target audience. Currently the libosinfo project is consumed by big players in the virtual machine management environment such as OpenStack Nova, virt-manager, GNOME Boxes, and Cockpit Machines.

… a little bit about them …

  • OpenStack Nova: An OpenStack project that provides a way to provision virtual machines, baremetal servers, and (limited supported for) system containers.

  • virt-manager: An application for managing virtual machines through libvirt.

  • GNOME Boxes: A simple application to view, access, and manage remote and virtual systems.

  • Cockpit Machines: A Cockpit extension to manage virtual machines running on the host.

… and why they use it

  • Download ISOs: As libosinfo provides the ISO URLs, management applications can offer the user the option to download a specific operating system;

  • Automatically detect the ISO being used: As libosinfo can detect the operating system of an ISO, management applications can use this info to set reasonable default values for resources, to select the hardware supported, and to perform unattended installations.

  • Start tree installation: As libosinfo provides the tree installation URLs, management applications can use it to start a network-based installation without having to download the whole operating system ISO;

  • Set reasonable default values for RAM, CPU, and disk resources: As libosinfo knows the values that are recommended by the operating system’s vendors, management applications can rely on that when setting the default resources for an installation.

  • Automatically set the hardware supported: As libosinfo provides the list of hardware supported by an operating system, management applications can choose the best defaults based on this information, without taking the risk of ending up with a non-bootable guest.

  • Unattended install: as libosinfo provides unattended installations scripts for CentOS, Debian, Fedora, Fedora Silverblue, Microsoft Windows, OpenSUSE, Red Hat Enterprise Linux, and Ubuntu, management applications can perform unattended installations for both “Workstation” and “Server” profiles.

What’s next?

The next blog post will provide a “demo” of an unattended installation using both GNOME Boxes and virt-install and, based on that, explain how libosinfo is internally used by these projects.

By doing that, we’ll both cover how libosinfo can be used and also demonstrate how it can ease the usage of those management applications.

October 15, 2019

openSUSE.Asia Summit 2019, Bali, Indonesia

I was at openSUSE.Asia summit 5-6 Oct 2019 at Udayana University, Bali, Indonesia . It was a great experience to be there. I got the opportunity to meet Douglas DeMaio openSUSE project Manager, Dr. Gerald Pfeifer, Chairman openSUSE. It was really great to meet Neil McGovern and Rosanna Yuen, from GNOME Foundation. Rosanna shared GNOME stickers that I will share with my students back in India. I would like to congratulate openSUSE.Asia summit local organizing team. They had really done a great job. It was a nice combination of quality talks and keynotes. I also presented my proposal at the conference for openSUSE.Asia 2020 to happen in India next year. It is been more than six year that I am an open source practitioner and promoter. It has been memorable Journey till date..

Following are the few of the glimpses form the event.   

 

On Recent Controversial Events

The last 33 days have been unprecedentedly difficult for the software freedom community and for me personally. Folks have been emailing, phoning, texting, tagging me on social media (— the last of which has been funny, because all my social media accounts are placeholder accounts). But, just about everyone has urged me to comment on the serious issues that the software freedom community now faces. Until now, I have stayed silent regarding all these current topics: from Richard M. Stallman (RMS)'s public statements, to his resignation from the Free Software Foundation (FSF), to the Epstein scandal and its connection to MIT. I've also avoided generally commenting on software freedom organizational governance during this period. I did this for good reason, which is explained below. However, in this blog post, I now share my primary comments on the matters that seem to currently be of the utmost attention of the Open Source and Free Software communities.

I have been silent the last month because, until two days ago, I was an at-large member of FSF's Board of Directors, and a Voting Member of the FSF. As a member of FSF's two leadership bodies, I was abiding by a reasonable request from the FSF management and my duty to the organization. Specifically, the FSF asked that all communication during the crisis come directly from FSF officers and not from at-large directors and/or Voting Members. Furthermore, the FSF management asked all Directors and Voting Members to remain silent on this entire matter — even on issues only tangentially related to the current situation, and even when speaking in our own capacity (e.g., on our own blogs like this one). The FSF is an important organization, and I take any request from the FSF seriously — so I abided fully with their request.

The situation was further complicated because folks at my employer, Software Freedom Conservancy (where I also serve on the Board of Directors) had strong opinions about this matter as well. Fortunately, the FSF and Conservancy both had already created clear protocols for what I should do if ever there was a disagreement or divergence of views between Conservancy and FSF. I therefore was recused fully from the planning, drafting, and timing of Conservancy's statement on this matter. I thank my colleagues at the Conservancy for working so carefully to keep me entirely outside the loop on their statement and to diligently assure that it was straight-forward for me to manage any potential organizational disagreements. I also thank those at the FSF who outlined clear protocols (ahead of time, back in March 2019) in case a situation like this ever came up. I also know my colleagues at Conservancy care deeply, as I do, about the health and welfare of the FSF and its mission of fighting for universal software freedom for all. None of us want, nor have, any substantive disagreement over software freedom issues.

I take very seriously my duty to the various organizations where I have (or have had) affiliations. More generally, I champion non-profit organizational transparency. Unfortunately, the current crisis left me in a quandary between the overarching goal of community transparency and abiding by FSF management's directives. Now that I've left the FSF Board of Directors, FSF's Voting Membership, and all my FSF volunteer roles (which ends my 22-year uninterrupted affiliation with the FSF), I can now comment on the substantive issues that face not just the FSF, but the Free Software community as a whole, while continuing to adhere to my past duty of acting in FSF's best interest. In other words, my affiliation with the FSF has come to an end for many good and useful reasons. The end to this affiliation allows me to speak directly about the core issues at the heart of the community's current crisis.

Firstly, all these events — from RMS' public comments on the MIT mailing list, to RMS' resignation from the FSF to RMS' discussions about the next steps for the GNU project — seem to many to have happened ridiculously quickly. But it wasn't actually fast at all. In fact, these events were culmination of issues that were slowly growing in concern to many people, including me.

For the last two years, I had been a loud internal voice in the FSF leadership regarding RMS' Free-Software-unrelated public statements; I felt strongly that it was in the best interest of the FSF to actively seek to limit such statements, and that it was my duty to FSF to speak out about this within the organization. Those who only learned of this story in the last month (understandably) believed Selam G.'s Medium post raised an entirely new issue. In fact, RMS' views and statements posted on stallman.org about sexual morality escalated for the worse over the last few years. When the escalation started, I still considered RMS both a friend and colleague, and I attempted to argue with him at length to convince him that some of his positions were harmful to sexual assault survivors and those who are sex trafficked, and to the people who devote their lives in service to such individuals. More importantly to the FSF, I attempted to persuade RMS that launching a controversial campaign on sexual behavior and morality was counter to his and FSF's mission to advance software freedom, and told RMS that my duty as an FSF Director was to assure the best outcome for the FSF, which IMO didn't include having a leader who made such statements. Not only is human sexual behavior not a topic on which RMS has adequate academic expertise, but also his positions appear to ignore significant research and widely available information on the subject. Many of his comments, while occasionally politically intriguing, lack empathy for people who experienced trauma.

IMO, this is not and has never been a Free Speech issue. I do believe freedom of speech links directly to software freedom: indeed, I see the freedom to publish software under Free licenses as almost a corollary to the freedom of speech. However, we do not need to follow leadership from those whose views we fundamentally disagree. Moreover, organizations need not and should not elevate spokespeople and leaders who speak regularly on unrelated issues that organizations find do not advance their mission, and/or that alienate important constituents. I, like many other software freedom leaders, curtail my public comments on issues not related to FOSS. (Indeed, I would not even be commenting on this issue if it had not become a central issue of concern to the software freedom community.) Leaders have power, and they must exercise the power of their words with restraint, not with impunity.

RMS has consistently argued that there was a campaign of “prudish intimidation” — seeking to keep him quiet about his views on sexuality. After years of conversing with RMS about how his non-software-freedom views were a distraction, an indulgence, and downright problematic, his general response was to make even more public comments of this nature. The issue is not about RMS' right to say what he believes, nor is it even about whether or not you agree or disagree with RMS' statements. The question is whether an organization should have a designated leader who is on a sustained, public campaign advocating about an unrelated issue that many consider controversial. It really doesn't matter what your view about the controversial issue is; a leader who refuses to stop talking loudly about unrelated issues eventually creates an untenable distraction from the radical activism you're actively trying to advance. The message of universal software freedom is a radical cause; it's basically impossible for one individual to effectively push forward two unrelated controversial agendas at once. In short, the radical message of software freedom became overshadowed by RMS' radical views about sexual morality.

And here is where I say the thing that may infuriate many but it's what I believe: I think RMS took a useful step by resigning some of his leadership roles at the FSF. I thank RMS for taking that step, and I wish the FSF Directors well in their efforts to assure that the FSF becomes a welcoming organization to all who care about universal software freedom. The FSF's mission is essential to our technological future, and we should all support that mission. I care deeply about that mission myself and have worked and will continue to work in our community in the best interest of the mission.

I'm admittedly struggling to find a way to work again with RMS, given his views on sexual morality and his behaviors stemming from those views. I explicitly do not agree with this “(re-)definition” of sexual assault. Furthermore, I believe uninformed statements about sexual assault are irresponsible and cause harm to victims. #MeToo is not a “frenzy”; it is a global movement by individuals who have been harmed seeking to hold both bad actors and society-at-large accountable for ignoring systemic wrongs. Nevertheless, I still am proud of the essay that I co-wrote with RMS and still find many of RMS' other essays compelling, important, and relevant.

I want the FSF to succeed in its mission and enter a new era of accomplishments. I've spent the last 22 years, without a break, dedicating substantial time, effort, care and loyalty to the various FSF roles that I've had: including employee, volunteer, at-large Director, and Voting Member. Even though my duties to the FSF are done, and my relationship with the FSF is no longer formal, I still think the FSF is a valuable institution worth helping and saving, specifically because the FSF was founded for a mission that I deeply support. And we should also realize that RMS — a human being (who is flawed like the rest of us) — invented that mission.

As culture change becomes more rapid, I hope we can find reasonable nuance and moderation on our complex analysis about people and their disparate views, while we also hold individuals fully accountable for their actions. That's the difficulty we face in the post-post-modern culture of the early twenty-first century. Most importantly, I believe we must find a way to stand firm for software freedom while also making a safe environment for victims of sexual assault, sexual abuse, gaslighting, and other deplorable actions.

Hardening Cockpit with systemd (socket activation)³

Background A major future goal for Cockpit is support for client-side TLS authentication, primarily with smart cards. I created a Proof of Concept and a demo long ago, but before this can be called production-ready, we first need to harden Cockpit’s web server cockpit-ws to be much more tamper-proof than it is today. This heavily uses systemd’s socket activation. I believe we are now using this in quite a unique and interesting way that helped us to achieve our goal rather elegantly and robustly.

October 13, 2019

Apple of 2019 is the Linux of 2000

Last week the laptop I use for macOS development said that there is an XCode update available. I tried to install it but it said that there is not enough free space available to run the installer. So I deleted a bunch of files and tried again. Still the same complaint. Then I deleted some unused VM images. Those would free a few dozen gigabytes, so it should make things work. I even emptied the trash can to make sure nothing lingered around. But even this did not help, I still got the same complaint.

At this point it was time to get serious and launch the terminal. And, true enough, according to df the disk had only 8 gigabytes of free space even though I had just deleted over 40 gigabytes of files from it (using rm, not the GUI, so things really should have been gone). A lot of googling and poking later I discovered that all the deleted files had gone to "reserved space" on the file system. There was no way to access those files or delete them. According to documentation the operating system would delete those files "on demand as more space is needed". This was not very comforting because the system most definitely was not doing that and you'd think that Apple's own software would get this right.

After a ton more googling I managed to find a chat buried somewhere deep in Reddit which listed the magical indentation that purges reserved space. It consisted of running tmutil from the command line and giving it a bunch of command line arguments that did not seem to make sense or have any correlation to the thing that I wanted to do. But it did work and eventually I got XCode updated.

After my blood pressure dropped to healthier levels I got the strangest feeling of déjà vu. This felt exactly like using Linux in the early 2000s. Things break at random for reasons you can't understand and the only way to fix it is to find terminal commands from discussion forums, type them in and hope for the best. Then it hit me.

This was not an isolated incidence. The parallels are everywhere. Observe:

External monitors

Linux 2000: plugging an external monitor will most likely not work. Fanboys are very vocal that this is the fault of monitor manufacturers for not providing modeline info.

Apple 2019: plugging an external projector will most likely not work. Fanboys are very vocal that this is the fault of projector manufacturers for not ensuring that their HW works with every Apple model.

Software installation

Linux 2000: There is only One True Way of installing software: using distro packages. If you do anything else you are bad and you should feel bad.

Apple 2019: There is only True Way of installing software: using the Apple store. If you do anything else you are bad and you should feel bad.

Hardware compatibility

Linux 2000: only a limited number of hardware works out of the box, even for popular devices like 3D graphics cards. Things either don't work at all, have reduced functionality, or kinda work but fail spuriously every now and then for no discernible reason.

Apple 2019: only a limited number of hardware works out of the box, even for popular devices like Android phones. Things either don't work at all, have reduced functionality, or kinda work but fail spuriously every now and then for no discernible reason.

Technical support

Linux 2000: if your problem is not google-trivial, there's nothing you can do. Asking friends for assistance does not help, because they will just type your problem description into Google and read the first hit.

Apple 2019: if your problem is not google-trivial, there's nothing you can do. Calling Apple's tech support line does not help, because they will just type your problem description into Google and read the first hit.

Laptop features

Linux 2000: it is very difficult to find a laptop with more than two USB ports.

Apple 2019: it is very difficult to find a laptop with more than two USB ports.

Advocate behaviour

Linux 2000: fanboys will let you know in no uncertain terms that their system is the best and will take over all desktop computer usage. Said fanboys are condescending elitist computer nerds.

Apple 2019: fanboys will let you know in no uncertain terms that their system is the best and will take over all desktop computer usage. Said fanboys are condescending elitist hipster latte web site designers.

October 12, 2019

Getting a stack trace out of a Flatpak

So, the flatpak application you use just crashed

How do you report it? If you file a bug just saying it crashed, the developers will probably ask for some stack trace. On Fedora 30, for example, abrt (the crash reporting system) doesn't provide any useful information. Let's see if we can extract that information.

We are gonna have to use the terminal to use some command line tools. Flatpak has a tool flatpak-coredumpctl to use the core dump in the flatpak sandbox. The core dump is an image of the program memory when it crashed that will contain a lot about the crash. But by default the tool will not be able to provide much useful info. There is some initial setup need to be able to have a better output.

First you must make sure that you have the right Debug package for the right version of the Flatpak runtime. Well, actually, for the corresponding SDK.

To check the runtime and version:

$ flatpak info org.gnome.MyApp

Check the line that starts with Sdk:

Sdk: org.gnome.Sdk/x86_64/3.34

What is after Sdk; could be a different value, and that's what is important here. This is what we are looking for

You can use the command flatpak install --user org.gnome.Sdk.Debug (in our example, just put the one found above) to install it. WARNING: it's likely big. For exampe the debug package for Gnome 3.34 is 3.4GB here. I recommend installing it for the user as unless you have a lot of space on your system filesystem (if separate) it will fail.

Then you need to install the debug info for the app. It is the app ID suffixed with .Debug. In that case org.gnome.MyApp.Debug.

Both will provide the debug info that is necessary to see where in the code things crashed.

$ flatpak-coredumpctl org.gnome.MyApp

This is gdb being launched inside the flatpak. It will take a while to process, and use quite a good chunk of memory. What follows is mostly for people that are not familiar with `gdb` ; if you are, just go ahead, you know what to do.

When it is ready, something like this will be printed in the terminal:

Program terminated with signal SIGSEGV, Segmentation fault.

And then at the end, the prompt: (gdb)

You can type where and press "Enter". This is the stack trace: lines starting with #NNN where NNN is an increasing number. You can copy that output to provide it to the developers in the bug report.

Then you can type quit followed by "Enter" when you are done.

Hope that's useful to you.

Music, Flathub and Qt

I have started reading recently about music theory and such, with the purpose to try to learn music (again). This lead me to look at music software, and what we have on Linux.

I found a tutorial by Ted Felix on Linux and MIDI

I quickly realised that trying these apps on my Dell XPS 13 was really an adventure, mostly because of HiDPI (the high DPI screen that the PS 13 has). Lot of the applications found on Fedora, by default, don't support high DPI and a thus quasi impossible to use out of the box. Some of it is fixable easily, some of it with a bit more effort and some, we need to try harder.

Almost all the apps I have tried used Qt. With Qt5 the fix is easy, albeit not necessarily user friendly. Just set the QT_AUTO_SCREEN_SCALE_FACTOR environment variable to 1 as specified in Qt HiDPI support documentation. There is also an API to set the attribute on the QCoreApplication object. There must be a good reason why this opt-in and not opt-out.

This is the case of Muse3 (aka muse sequence not to be confused with MuseScore which work out of the box, at east from Flathub), and Rosegarden.

LMMS and Hydrogen on the other hand are using Qt4 on Fedora 29. The good news? They both have been ported to Qt5, so it is just a matter of building these newer versions. After doing so, they still need the workaround described above.

This is where Flathub comes into play: make them available on Flathub, where we can set that environment variable for the runtime.

In the end, I have Hydrogen available on Flathub, the three others in queue for Flathub, and all have had patches submitted (with Muse3 and Rosegarden already merged upstream).

Now what other cool Free Software music oriented application exist?

October 11, 2019

g_warning_once() in GLib 2.63.1

GLib 2.63.1 will be released in the next few weeks, and will contain a fun new API to slightly simplify emitting a warning once, and then shutting up to avoid emitting loads of log spam.

It is g_warning_once(), and works just like g_warning().

Thanks to Jonas Ã…dahl for putting it together!

October 10, 2019

Next conferences

Just to say I’m going to a pair of conferences here in Spain:

At WMES 2019 I will lead a Wikidata workshop about adding historical heritage data, basically repeating the one at esLibre.

At LAS 2019 I plan to attend to the Flatpak workshops and to call for a BoF for people involved in opensource conference organizations to share experiences and reuse tools.

Lots of thanks for the Wikimedia España association and GNOME Foundation for their travel sponsorship. Without their help I could not attend both.

See you in Pamplona and Barcelona.

2019-10-10 Thursday.

  • Mail chew, projections, planning, patch porting; performance win for idle JSON conversion of sidebars.
  • Intensely frustrated to (minor) update openSUSE 15.1, and discover it jumped thunderbird from 60.8 to 68, breaking all of the extensions needed to be productive - (eg. making it possible to move E-mail between folders without getting RSI from mouse-use), horrible; downgraded with trepidation.
  • TDF is making good progress on a carbon policy, but I was curious as to how writing more efficient software might help here. Crunching some numbers with a 50/50 desktop/laptop mix and a 65% increase from CPU use gives ~10^-5 kg of CO2 per CPU second you're not idle. Multiplying that by the number of LibreOffice users 2x10^8 (pace Sir David MacKay) to get a big number, that's around 2 tons of CO2 saved for every second of CPU busy time that we can save our user-base; go optimizers!

Free software at 40°C

It’s that time of year again, time for a belated reflection on the GUADEC conference!

In August I traveled to Thessaloniki, Greece, to attend first the annual GNOME Foundation board handover day, then the advisory board meeting, then the GUADEC conference and associated unconference days.

The board discussion focused quite a lot on the strategic goals for the GNOME Foundation which you can hear more about in executive director Neil McGovern’s talk. Nuritzi has also blogged about the process of putting together these strategic goals.

One of the goals that’s most important to me personally (and in fact, in a slightly different form it was part of my original candidacy statement the first time I was elected to the GNOME Foundation board) is:

Evaluate and adopt new technologies to stay competitive with proprietary desktops.

In the evening after the board handover I got a chance to talk about this with some GNOME people over gyros and why I think it’s important — but that’s a topic that could fill an entire blog post! And I intend to publish one at some point.

I’d like to describe some of the talks that I thought were highlights during the conference days.

Setting a Positive Voice For GNOME, Britt Yazel (video) — One of a surprisingly large number of talks this year about how a large free software project is made up of humans, something which people tend to forget.

Hack: Embedding Learning Tools in the OS, Meg Ford / Manuel Quiñones (video) — This is a great talk about some of the cool stuff in a project which is quite familiar to me; as I had worked on it for a year. I had not really gotten a chance to take a step back and see how cool it actually is, and this talk brought that home for me.

Environmentally Friendly GNOME, Philip Withnall (video) — Fantastic use of the relatively little data we have, to do quite a lot of order-of-magnitude calculations about GNOME’s environmental footprint, as well as suggestions on how to reduce it.

About Maintainers and Contributors, Georges Basile Stavracas Neto (video) — Another talk about humans, and one of the reasons why I think this conference will be regarded as a turning point in GNOME’s history. Georges talks honestly about the struggles that many of us face when we become the public face of a well-known project.

Free Software / Utopia, Deb Nicholson (video) — A strong closer for the conference, and a strong statement of values for a community that is — sorry if I sound like a broken record — made up of humans. I fully support the idea that we shouldn’t just try to equal proprietary software, but to do better, not only technically, but in our treatment of humans!

There were a lot of difficult decisions to make as to which conference track to follow and I ended up switching back and forth a lot, and in addition I was rehearsing my own talk. This meant I still had to miss some things that I would have liked to attend, like (unfortunately) the workshops on Saturday.

On Saturday there was also the annual meeting of the GNOME Foundation, but I think plenty of other people have blogged about that already.

On Sunday I delievered my talk (video, slides) and started to run out of time near the end, so the video shows how I gradually started talking faster and faster… Happily, a few people expressed interest in taking on some of the projects for GJS that I suggested in the talk, and I hope that when the next round of Outreachy and Google Summer of Code internships come around I will be able to be a mentor again.

During the unconference days I did not actually stick very much to the published schedule; I was mostly occupied with hallway conversations. I spent one day with the travel committee talking about what we could do to improve the travel sponsorship process, and taking advantage of the fact that most of the committee was in the same place to review the open tickets.

I also spent some time hacking on GJS, both by myself and with others: Meg Ford and Marco Trevisan started fixing some GJS bugs.

The last day was filled with a whirlwind tour of the many museums of Thessaloniki — anyone who knows me personally knows that I love museums and could have stayed for many more hours in each museum.

Thanks to the GNOME Foundation for sponsoring my attendance.

October 09, 2019

2019-10-09 Wednesday.

  • Testing, project planning, etc. Got the Oculus Quest update making it possible to play 'go' games, very impressed with Daedalus' flying thing. Band practice at church, worked late.

October 08, 2019

thoughts on rms and gnu

Yesterday, a collective of GNU maintainers publicly posted a statement advocating collective decision-making in the GNU project. I would like to expand on what that statement means to me and why I signed on.

For many years now, I have not considered Richard Stallman (RMS) to be the head of the GNU project. Yes, he created GNU, speaking it into existence via prophetic narrative and via code; yes, he inspired many people, myself included, to make the vision of a GNU system into a reality; and yes, he should be recognized for these things. But accomplishing difficult and important tasks for GNU in the past does not grant RMS perpetual sovereignty over GNU in the future.

ontological considerations

More on the motivations for the non serviam in a minute. But first, a meta-point: the GNU project does not exist, at least not in the sense that many people think it does. It is not a legal entity. It is not a charity. You cannot give money to the GNU project. Besides the manifesto, GNU has no by-laws or constitution or founding document.

One could describe GNU as a set of software packages that have been designated by RMS as forming part, in some way, of GNU. But this artifact-centered description does not capture movement: software does not, by itself, change the world; it lacks agency. It is the people that maintain, grow, adapt, and build the software that are the heart of the GNU project -- the maintainers of and contributors to the GNU packages. They are the GNU of whom I speak and of whom I form a part.

wasted youth

Richard Stallman describes himself as the leader of the GNU project -- the "chief GNUisance", he calls it -- but this position only exists in any real sense by consent of the people that make GNU. So what is he doing with this role? Does he deserve it? Should we consent?

To me it has been clear for many years that to a first approximation, the answer is that RMS does nothing for GNU. RMS does not write software. He does not design software, or systems. He does hold a role of accepting new projects into GNU; there, his primary criteria is not "does this make a better GNU system"; it is, rather, "does the new project meet the minimum requirements".

By itself, this seems to me to be a failure of leadership for a software project like GNU. But unfortunately when RMS's role in GNU isn't neglect, more often as not it's negative. RMS's interventions are generally conservative -- to assert authority over the workings of the GNU project, to preserve ways of operating that he sees as important. See for example the whole glibc abortion joke debacle as an example of how RMS acts, when he chooses to do so.

Which, fair enough, right? I can hear you saying it. RMS started GNU so RMS decides what it is and what it can be. But I don't accept that. GNU is about practical software freedom, not about RMS. GNU has long outgrown any individual contributor. I don't think RMS has the legitimacy to tell this group of largely volunteers what we should build or how we should organize ourselves. Or rather, he can say what he thinks, but he has no dominion over GNU; he does not have majority sweat equity in the project. If RMS actually wants the project to outlive him -- something that by his actions is not clear -- the best thing that he could do for GNU is to stop pretending to run things, to instead declare victory and retire to an emeritus role.

Note, however, that my personal perspective here is not a consensus position of the GNU project. There are many (most?) GNU developers that still consider RMS to be GNU's rightful leader. I think they are mistaken, but I do not repudiate them for this reason; we can work together while differing on this and other matters. I simply state that I, personally, do not serve RMS.

selective attrition

Though the "voluntary servitude" questions are at the heart of the recent joint statement, I think we all recognize that attempts at self-organization in GNU face a grave difficulty, even if RMS decided to retire tomorrow, in the way that GNU maintainers have selected themselves.

The great tragedy of RMS's tenure in the supposedly universalist FSF and GNU projects is that he behaves in a way that is particularly alienating to women. It doesn't take a genius to conclude that if you're personally driving away potential collaborators, that's a bad thing for the organization, and actively harmful to the organization's goals: software freedom is a cause that is explicitly for everyone.

We already know that software development in people's free time skews towards privilege: not everyone has the ability to devote many hours per week to what is for many people a hobby, and it follows of course that those that have more privilege in society will be more able to establish a position in the movement. And then on top of these limitations on contributors coming in, we additionally have this negative effect of a toxic culture pushing people out.

The result, sadly, is that a significant proportion of those that have stuck with GNU don't see any problems with RMS. The cause of software freedom has always run against the grain of capitalism so GNU people are used to being a bit contrarian, but it has also had the unfortunate effect of creating a cult of personality and a with-us-or-against-us mentality. For some, only a traitor would criticise the GNU project. It's laughable but it's a thing; I prefer to ignore these perspectives.

Finally, it must be said that there are a few GNU people for whom it's important to check if the microphone is on before making a joke about rape culture. (Incidentally, RMS had nothing to say on that issue; how useless.)

So I honestly am not sure if GNU as a whole effectively has the demos to make good decisions. Neglect and selective attrition have gravely weakened the project. But I stand by the principles and practice of software freedom, and by my fellow GNU maintainers who are unwilling to accept the status quo, and I consider attempts to reduce GNU to founder-loyalty to be mistaken and without legitimacy.

where we're at

Given this divided state regarding RMS, the only conclusion I can make is that for the foreseeable future, GNU is not likely to have a formal leadership. There will be affinity groups working in different ways. It's not ideal, but the differences are real and cannot be papered over. Perhaps in the medium term, GNU maintainers can reach enough consensus to establish a formal collective decision-making process; here's hoping.

In the meantime, as always, happy hacking, and: no gods! No masters! No chief!!!

October 07, 2019

GStreamer Conference 2019 (including GStreamer and PipeWire hackfests)

GStreamer Conference 2019 banner

GStreamer Conference 2019 in Lyon France


So the GStreamer Conference 2019 is approaching being held in Lyon, France between 31st October and 1st November 2019. This year is special as it marks the GStreamer projects 20th year of existence. I still remember seeing the announcement of GStreamer 0.0.9 which Erik Walthinsen sent to the GNOME announe mailing list. Back then I felt that multimedia support where one of the big gaps around the Linux operating system that needed filling (no, XAnim was nice for its time, but it was not a long term solution :) and GStreamer seemed like the perfect project to fill it. So I joined the GStreamer IRC channel determined to try to help the project succeed however I could. A little over a year later we all met for the first time at GUADEC in Copenhagen, even posing for this exciting team photo.

GStreamer Team at GUADEC Copenhagen in 2001 (we all looked slightly younger and fresher back then.)


Anyway, 20 years later there will be a talk and presentation by GStreamer co-founder Wim Taymans (wearing blue shirt and black pants in picture above) at the GStreamer Conference commemorating 20 years of GStreamer. Detailing taking the project from idealistic spare time effort to the multimedia industry juggernaut it is today.

Of course the conference is not going to be focused on the past, as there is a long line up of great talks talking about modern streaming with DASH, HDR support in GStreamer, latest developments around GStreamer and Rust, Virtual reality, Vulkan and more. Actually on the ‘and more’ topic, Wim Taymans will also do a presentation on PipeWire, the next generation audio and video server, at the GStreamer Conference this year, hopefully demoing some of the great improvements in things like our pro-audio Jack emulation support.
So if you haven’t already, make your way to the GStreamer Conference 2019 website and register for the 10th annual GStreamer Conference!

For those going be aware that there will also be a joint GStreamer fall hackfest and PipeWire hackfest in the two days following the GStreamer Conference. So be sure to sign up for those if interested. They will be co-located with participants flowing freely between the two events.

GNOME 3.34 Release Party in Brno, Czech Republic

In September 25th we had once more a local meetup in Brno to celebrate another fantastic GNOME Release!

GNOME “Thessaloniki” 3.34 is out now and will be reaching distros in the following months. This version is the result of the work of approximately 777 contributors in the last six months. For more details, check out the release notes.

Our Brno celebrations this cycle were held in Schrott, a place with a wide variety of beers and a neat industrial decor. Dominika Vagnerova arranged delicious GNOME themed cupcakes with eatable app icons that went along pretty well with the drinks.

This was an excelent opportunity for us to sit down, relax, and chat about GNOME, Free Software, and all things that bring us together.

More photos of the event are available in our shared album, including ~exclusive~ pictures of application maintainers eating their apps’ cupcakes. :-)

Thanks everyone that showed up, special thanks to Dominika for organizing the event and Rishi for the photos. Stay tuned for 3.36!

October 06, 2019

Talking at ARES 2019 in Canterbury, UK

It’s conference season and I attended the International Conference on Availability, Reliability, and Security (ARES) in Canterbury, UK. (note that in the future, the link might change to something more sustainable)

A representative of the Kent University opened the event. It is the UK’s European University, he said, with 20000 students, many of them being from other countries. He attributed that to the proximity to mainland Europe. Indeed it’s only an hour away (if you don’t have to go back to London to catch a direct Eurostar rather than one that stops in, say, Ashford). The conference was fairly international, too, with 230 participants from 33 countries. As an academic conference, they care about the “acceptance rate” which, in this case, was at 20.75%. Of course, he could have mentioned any number, because it’s impossible to verify.

The opening keynote was given by Alistair MacWilson from Bletchley Park. Yeah, the same Bletchley Park which Alan Turing worked at. He talked about the importance of academia in closing the cybersecurity talent gap. He said that the deficit of people knowing anything about cybersecurity skills is 3.3M with 380k alone in Europe, but APAC being desperately short of 2.1M professionals. All that is good news for us youngsters in the business, but not so good, he said, if you rely on the security of your IT infrastructure… It’s not getting any better, he said, considering that the number of connected devices and the complexity of our infrastructure is rising. You might think, he said, that highly technical skills are required to perform cybersecurity tasks. But he mentioned that 88% of the security problems that the global 5000 companies have stem from human factors. Inadequate and unfocussed training paired with insufficient resources contribute to that problem, he said. So if you don’t get continuous training then you will fall behind with your skill-set.

There were many remarkable talks and the papers can be found online; albeit behind a paywall. But I expect SciHub to have copies and authors to be willing to share their work if you ask. Anyway, one talk I remember was about delivering Value Added Services to electric vehicle charging. They said that it is currently not very attractive for commercial operators to provide charging stations, because the margin is low. Hence, additional monetisation in form of Value Added Services (VAS) could be added. They were thinking of updating the software of the vehicle while it is charging. I am not convinced that updating the car’s firmware makes a good VAS but I’m not an economist and what do I know about the world of electric vehicles. Anyway, their proposal to add VAS to the communication protocol might be justified, but their scenario of delivering software updates over that channel seems like a lost opportunity to me. Software updates are currently the most successful approach to protecting users, so it seems warranted to have an update protocol rather than a VAS protocol for electric vehicles.

My own talk was about using the context and provenance of USB-borne events (illegal public copy) to mitigate attacks via that channel. So general idea, known to readers of my blog, is to take the state of the session into account when dealing with events stemming from USB devices. More precisely, when your session is locked, don’t automatically load drivers for a new USB device. Your session is locked, after all. You’re not using your machine and cannot insert a new device. Hence, the likelihood of someone else maliciously inserting a device is higher than when your session is unlocked. Of course, that’s only a heuristic and some will argue that they frequently plug devices into their machine when it’s locked. Fair enough. I argue that we need to be sensitive and change as little as possible to the user’s way of working with the machine to get high acceptance rates. Hence, we need to be careful when devices like keyboards are inserted. Another scenario is the new network card that has been attached via USB. It should be more suspicious to accept that nameserver that came from the new network card’s DHCP server when the system has a perfectly working network configuration (and the DHCP response did not contain a default gateway). Turns out, that those attacks are mounted right now in real-life and we have yet to find defences that we can deploy on a large scale.

It’s been a nice event, even though the sandwiches for lunch got boring after a few days ;-) I am happy to have met researchers from other areas and I hope to stay in touch.

Hardening Flatpak Permissions Over Time

One of the main goals of Flatpak is to sandbox applications but a common complaint is that many packages add a lot of insecure permissions which is entirely valid. I’ll be showing an example of how over time many permissions now have secure alternatives.

For this example I’ll be using Pithos which is an application I maintain.
It is an online radio player for Pandora.com.

These are the Flatpak permissions used for the application 2 years ago:

  "finish-args": [
    "--share=ipc",
    "--share=network",
    "--socket=x11",
    "--socket=wayland",
    "--socket=pulseaudio",

    "--env=DCONF_USER_CONFIG_DIR=.config/dconf",
    "--filesystem=xdg-run/dconf",
    "--filesystem=~/.config/dconf:ro",
    "--talk-name=ca.desrt.dconf",

    "--talk-name=org.freedesktop.secrets",

    "--talk-name=org.freedesktop.Notifications",

    "--talk-name=org.gnome.SettingsDaemon",
    "--talk-name=org.mate.SettingsDaemon",

    "--talk-name=org.gnome.ScreenSaver",
    "--talk-name=org.cinnamon.ScreenSaver",
    "--talk-name=org.freedesktop.ScreenSaver",
    "--talk-name=com.canonical.Unity.Session",

    "--talk-name=org.kde.StatusNotifierWatcher",

    "--own-name=org.mpris.MediaPlayer2.pithos"
  ]

These are the possible* permissions today:

  "finish-args": [
    "--share=ipc",
    "--share=network",
    "--socket=fallback-x11",
    "--socket=wayland",
    "--socket=pulseaudio",

    "--talk-name=org.gnome.SettingsDaemon.MediaKeys",
    "--talk-name=org.mate.SettingsDaemon",

    "--talk-name=org.kde.StatusNotifierWatcher"

So lets go over each permission from the old manifest.

    "--share=ipc", "--share=network", "--socket=x11", "--socket=wayland", "--socket=pulseaudio"

These are simply required for the application to function. The application will always require network access and wayland access. We did replace x11 with fallback-x11 which means on Wayland X11 is blocked. Eventually (hopefully within a year?) pulseaudio will have a similar solution for pipewire which will be able to block things like being able to record audio.

    "--filesystem=xdg-run/dconf", "--filesystem=~/.config/dconf:ro", "--talk-name=ca.desrt.dconf"

As of GLib 2.60.6+ this permission is simply removed and settings are stored locally. This prevents applications from reading and writing settings for all software on the system.

    "--talk-name=org.freedesktop.secrets"

In an upcoming libsecret release this will also store secrets locally preventing the application from being able to read and write all stored secrets in the system keyring.

* This is not in a stable release yet.

    "--talk-name=org.freedesktop.Notifications"

A portal for notifications have been supported for a long time and the application simply uses the GNotification API and just works.

    "--talk-name=org.gnome.SettingsDaemon", "--talk-name=org.mate.SettingsDaemon"

These permissions are required for GNOME (and forks of GNOME like MATE) to bind media keys. Granting talk access to everything the settings daemon can do is not ideal and lets the application mess with system wide settings. Years ago gnome-settings-daemon improved this by splitting the features into their own namespaces so you only need to expose org.gnome.SettingsDaemon.MediaKeys which should be relatively safe to talk to. Hopefully this goes away entirely eventually with a host side solution using MPRIS.

    "--talk-name=org.gnome.ScreenSaver", "--talk-name=org.cinnamon.ScreenSaver", "--talk-name=org.freedesktop.ScreenSaver", "--talk-name=com.canonical.Unity.Session"

These were used to track the state of the screensaver to pause playback. This isn’t the most dangerous thing in the world but it does allow applications to activate the screensaver. GTK now supports the GtkApplication::screensaver-active property which lets an application monitor the state without having extra permissions.

    "--own-name=org.mpris.MediaPlayer2.pithos"

This was never a dangerous permission but as of Flatpak 1.0.4 you implicitly have permission to own org.mpris.MediaPlayer2.$app_id and subnames so that is now used for our name.

    "--talk-name=org.kde.StatusNotifierWatcher"

Lastly we do not have an alternative to this permission which still isn’t ideal and could be used to spoof other applications or exploit the host service which was likely never designed to handle untrusted data.


In conclusion there is certainly progress being made and this isn’t the only application slowly tightening permissions. There are of course still big holes where no alternative exists (most usage of --filesystem) but already today relatively secure but still well integrated flatpak’d applications can and do exist.

October 05, 2019

Incremental present in GTK4

When working with graphical applications, there are multiple constraints and techniques applied in order to reduce the number of pixels that are being uploaded to the GPU, swapped on screen, or being manipulated. Even with highly optimized GPUs, the massive number of pixels we have to deal with (a 1080p monitor, for example, has 2 million pixels!) forces everyone to have some level of scrutiny.

When it comes to Linux compositors and clients, a widely adopted technique is regional rendering. GTK tracks which parts of the window actually changed and only redraws that part; then sends this information to the compositor so that the compositor itself can redraw only the new contents of the window.

Fortunately, the entire graphics stack is well optimized for doing that! When using EGL, we can use eglSwapBuffersWithDamageEXT(), which receives a list of rectangles representing the parts of the window that changed. Mutter also uses a similar API after compositing the desktop.

With GTK4, we have not only GL-based renderers, but Vulkan renderers as well! A few years back, GTK4 received support for Vulkan on Wayland. While developing GNOME To Do (which already uses GTK4), I recently tried the Vulkan renderer, but the result was disappointing:

It looks terribly broken!

After a lot of research, asking around various places, and trying to understand Vulkan again, finally it was fixed! I must say, it was extremely hard to figure it out. Vulkan is very interesting, but the number of details is absolutely overwhelming. However, having it rendering correctly wasn’t enough.

The Vulkan renderer in GTK4 always submits the entire window to the compositor. Sure, we’re still rendering everything in the GPU, so pixels are not being uploaded, but it’s still suboptimal. Fortunately for us, we have VK_KHR_incremental_present, which allows us to tell Vulkan which parts of the window actually changed — like eglSwapBuffersWithDamageEXT() — and optimize the rendering.

Much better, isn’t it?

This was already merged and will be part of GTK 4.

October 04, 2019

Investigating the security of Lime scooters

(Note: to be clear, this vulnerability does not exist in the current version of the software on these scooters. Also, this is not the topic of my Kawaiicon talk.)

I've been looking at the security of the Lime escooters. These caught my attention because:
(1) There's a whole bunch of them outside my building, and
(2) I can see them via Bluetooth from my sofa
which, given that I'm extremely lazy, made them more attractive targets than something that would actually require me to leave my home. I did some digging. Limes run Linux and have a single running app that's responsible for scooter management. They have an internal debug port that exposes USB and which, until this happened, ran adb (as root!) over this USB. As a result, there's a fair amount of information available in various places, which made it easier to start figuring out how they work.

The obvious attack surface is Bluetooth (Limes have wifi, but only appear to use it to upload lists of nearby wifi networks, presumably for geolocation if they can't get a GPS fix). Each Lime broadcasts its name as Lime-12345678 where 12345678 is 8 digits of hex. They implement Bluetooth Low Energy and expose a custom service with various attributes. One of these attributes (0x35 on at least some of them) sends Bluetooth traffic to the application processor, which then parses it. This is where things get a little more interesting. The app has a core event loop that can take commands from multiple sources and then makes a decision about which component to dispatch them to. Each command is of the following form:

AT+type,password,time,sequence,data$

where type is one of either ATH, QRY, CMD or DBG. The password is a TOTP derived from the IMEI of the scooter, the time is simply the current date and time of day, the sequence is a monotonically increasing counter and the data is a blob of JSON. The command is terminated with a $ sign. The code is fairly agnostic about where the command came from, which means that you can send the same commands over Bluetooth as you can over the cellular network that the Limes are connected to. Since locking and unlocking is triggered by one of these commands being sent over the network, it ought to be possible to do the same by pushing a command over Bluetooth.

Unfortunately for nefarious individuals, all commands sent over Bluetooth are ignored until an authentication step is performed. The code I looked at had two ways of performing authentication - you could send an authentication token that was derived from the scooter's IMEI and the current time and some other stuff, or you could send a token that was just an HMAC of the IMEI and a static secret. Doing the latter was more appealing, both because it's simpler and because doing so flipped the scooter into manufacturing mode at which point all other command validation was also disabled (bye bye having to generate a TOTP). But how do we get the IMEI? There's actually two approaches:

1) Read it off the sticker that's on the side of the scooter (obvious, uninteresting)
2) Take advantage of how the scooter's Bluetooth name is generated

Remember the 8 digits of hex I mentioned earlier? They're generated by taking the IMEI, encrypting it using DES and a static key (0x11, 0x22, 0x33, 0x44, 0x55, 0x66, 0x77, 0x88), discarding the first 4 bytes of the output and turning the last 4 bytes into 8 digits of hex. Since we're discarding information, there's no way to immediately reverse the process - but IMEIs for a given manufacturer are all allocated from the same range, so we can just take the entire possible IMEI space for the modem chipset Lime use, encrypt all of them and end up with a mapping of name to IMEI (it turns out this doesn't guarantee that the mapping is unique - for around 0.01%, the same name maps to two different IMEIs). So we now have enough information to generate an authentication token that we can send over Bluetooth, which disables all further authentication and enables us to send further commands to disconnect the scooter from the network (so we can't be tracked) and then unlock and enable the scooter.

(Note: these are actual crimes)

This all seemed very exciting, but then a shock twist occurred - earlier this year, Lime updated their authentication method and now there's actual asymmetric cryptography involved and you'd need to engage in rather more actual crimes to obtain the key material necessary to authenticate over Bluetooth, and all of this research becomes much less interesting other than as an example of how other companies probably shouldn't do it.

In any case, congratulations to Lime on actually implementing security!

comment count unavailable comments

October 03, 2019

Meet Alyssa Rosenzweig and Panfrost

Hi, I’m Gaurav Agrawal, a member of the GNOME Engagement Team. I recently had the chance to interview Alyssa Rosenzweig, who is a lead developer at Panfrost project which is a free and open source driver Mali Midgard and Bifrost GPUs. Alyssa spent her summer as an intern at Collabora working on improving Panfrost’s OpenGL ES 2.0 userspace, which helps GNOME Shell work fluidly on supported Mali Hardware.

A screenshot of panfrost in action, with four open images of a Debian terminal, a logo, a jellyfish, and a computer generated landscape.

How about we kick off with a little bit of background on Panfrost?

Panfrost is a free, open-source graphics stack for Arm Mali GPUs, focused on the popular Midgard series. While these chips are popular among Android devices, they have been historical thorns in Linux’s side, due to the closed nature of the official drivers. Panfrost aims to change that, bringing the benefits of open-source to the Mali world.

What started out as a small community reverse-engineering effort has now matured into a reliable OpenGL ES 2.0 driver. Since May, I’ve been using Panfrost as my daily driver to program Panfrost. And yes, I’m answering these questions from a machine with Panfrost!

How did you get involved with the Project’s team/founded the team.

I’m passionate about spreading free software across the entire stack. To me, it’s not enough to have a free kernel; we also need a free desktop environment like GNOME. Yet it’s not enough to have just a free kernel and free desktop environment — we need free drivers and free firmware. Researching the state of free firmware for x86 systems, I realized that for long-term success, free software needs to win on Arm platforms, where free firmware at the lowest levels is still an option on systems like the Rockchip RK3399. These Rockchip systems have gained considerable mainline support, including support for the on-board video processing unit, thanks to past Collabora contributions. The future looked bright for Linux on Arm.

Unfortunately, despite these freedom wins, these Arm boards featured Mali GPUs, whose proprietary drivers prevented free software from truly taking off here. Frustrated with the GPU serving as the sole obstacle to a modern fully open source laptop, two years ago I purchased a development machine with a Mali — and the rest is history.

We will love to know, what were the issues with existing proprietary Arm drivers, which users were facing?

The issue with proprietary drivers is both practical and philosophical, and the proprietary Mali drivers are no exception. 3D acceleration is a de facto requirement of the modern system; even if a user is not interested in video games, they still need acceleration to run software using OpenGL like GNOME with full performance. Thus, philosophically, the requirement of the proprietary drivers for OpenGL support prevents normal usage of systems with Mali with free software.

Practically, the proprietary drivers pose a number of challenges for Linux users. Arm’s userspace drivers require the use of Arm’s kernel drivers. While these kernel drivers are technically open source, they are tightly coupled with the proprietary stack, which prevents their integration with the upstream “mainline” kernel. Today, most users never have to think about installing a kernel; the kernel for their hardware is part of their distribution, and distributions can easily maintain support for any hardware supported upstream. But a Linux user that needs a Mali chip cannot rely on their distribution for the kernel; the driver is maintained out-of-tree and requires a complex porting process to work against a normal upstream kernel. Far too often, users will resort to use outdated, buggy, insecure, downstream kernels, simply because they cannot use the mainline kernel if they need graphics.

Panfrost changes that. Our kernel module is designed for open-source and is included in the mainline kernel. Likewise, our userspace implementation is open-source and part of the upstream Mesa project, shared with the open-source Linux drivers for Intel, AMD, and Broadcom GPUs. Thus, with Panfrost, Linux users can install the distribution of their choice, using a modern, secure upstream kernel, while 3D graphics works out of the box.

Your project focuses on improving Panfrost’s OpenGL ES 2.0 userspace, we will like to know what this is about, and how it will benefit others?

OpenGL ES 2.0 is the core API for graphics on Arm platforms. Although newer versions of OpenGL ES exist, most software a user will encounter day-to-day can run on OpenGL ES 2.0. By focusing on this API, Panfrost is able to provide a smooth user experience where it counts.

Panfrost uses the open source Mesa implementation of OpenGL ES 2.0 to provide this experience to users. Mesa provides the OpenGL frontend via the common open-source “Gallium” API. Panfrost is a Gallium driver, thus enabling OpenGL ES 2.0 apps to run atop Mali with no proprietary components.

But Panfrost goes further! OpenGL ES is the “embedded subset” of OpenGL, the API used more commonly on Linux. The proprietary userspace drivers only support OpenGL ES, with no support for desktop OpenGL, leaving Linux users forced to specially compile software or use fickle translation layers. Fortunately, Panfrost provides a solution!

Leveraging the power of a strong open-source community via Mesa and Gallium, Panfrost is able to support OpenGL 2.1, a “common denominator” API prevalent on Linux. Other drivers have contributed to the desktop OpenGL support in Mesa and Gallium, and via this shared open-source framework, this work is shared and everyone benefits — including Panfrost users.

In practice, this support means a user running a distribution like Debian can install desktops like GNOME and have acceleration work out of the box. Whereas the proprietary userspace would leave a would-be GNOME user to fend for herself, Panfrost provides a smooth, Linux-first experience.

A young woman, against a blue background, wearing a red shirt. She has long, dark brown hair.We really want to know how are you so creative with commit messages ;) (… , :^, ./test-clear works, woo, I think I got it ?, Fix textures \0/, 🤔 , I tried…, Hmm )

Programming is mentally draining for some and physically draining for others. For me, I think programming is _emotionally_ draining. By the time my code works, sometimes you just have to let out all that emotion into the nearest text box. Sometimes that’s IRC, and sometimes that’s the commit message :-)

You went on a bug fixing adventure with GNOME, and we are excited to know what treasures you got ;)

Sometimes debugging feels like chasing my tail. But that’s not a problem — I’m not going in circles; I’m spiraling out and learning so much along the way. Sometimes that knowledge doesn’t help fix the bug, but it’s always a treasure!

GNOME offered no shortage of treasures. I installed a standard GNOME system from my distribution, which was built with OpenGL 2.1, rather than OpenGL ES 2.0. While OpenGL 2.1 has been tested with Panfrost, at the time, we had not tested it as extensively as ES 2.0, so there were all sorts of little gotcha’s I discovered. For instance, desktop OpenGL uses a slightly different texture specification mechanism, which challenged our previous texture implementation and demenaded a refactor — something I never would have noticed if I weren’t bringing up an app like GNOME.

_The_ bug, as it were, was unrelated to my research into complex topics like textures and tilers. No, in fact, it was a trivial piece of code related to the viewport descriptor. Panfrost’s implementation was correct for OpenGL ES 2.0, but again, OpenGL 2.1 offers more flexibility, so our implementation did not work there. After an agonizing bug search, a little bit of robustness improvements to the viewport code made all the difference in the world, and a moment later, I was running GNOME.

It will be really interesting to know how you all got nearer to the “Rasterization Discard” with the work “Scoreboard Implementation” on Mali’s Tiled Architecture, and we are curious to know simple explanation of these terms.

In graphics with OpenGL, the fundamental unit is a “draw”. Each draw has a pair of shaders, small programs running on the GPU. The first shader is the vertex shader, which determines where on the screen the GPU should render. The second shader is the fragment shader, which determines what colours the GPU should render. For an application like GNOME, these shaders are simple, copying the images of windows onto the screen. For a game, these shaders can be arbitrarily complex to implement fancy rendering algorithms.

Mali’s architecture subdivides draws into two parts, a vertex job (running a vertex shader) and a tiler job (setting up a fragment shader). If a simple app submits 100 draws, the driver will generate 100 vertex jobs and 100 tiler jobs. But these jobs have to run in a specific order: while all of the vertex jobs could run simultaneously, each tiler job (fragment shader) can only run once the corresponding vertex job (vertex shader) has run. Mali lets each job “depend” on other jobs, so one job can only run after its dependencies have run, much like a package in a package manager only installing once its dependencies were installed.

Originally, we hardcoded these dependencies, but this proved inflexible. This hardcoding was replaced by a high-level description of each job’s dependencies, so an automatic algorithm can compute in which order jobs need to be specified. This algorithm is knowing as “scoreboarding”.

The benefit of this automatic approach is seen in “rasterization discard”, an OpenGL feature that runs vertex shaders but does not draw anything to the screen. On Mali, that means we generate a vertex job, but we _don’t_ generate a tiler job. When we hardcoded jobs, this asymmetry was a problem, but once we implemented an automatic algorithm, this is as simple as just… not generating a tiler job. In negative lines of code, we can implement rasterization discard!

What are some popular devices that you believe can adopt your work, and through them it will benefit lots of people ?

A number of Arm-based Chromebooks use GPUs supported by Panfrost, including my personal development laptop, the Samsung Chromebook Plus. Collabora has contributed to the open-source mainline support for ChromeOS on these Chromebooks, and as a result of our open-first approach, Linux users of these Chromebooks benefit from a well-supported mainline stack. With Panfrost in the upstream kernel, these Chromebooks work on mainline _without_ sacrificing critical hardware support!

Beyond Chromebooks, based on the same high-performance RK3399 chipset, the Linux community’s own Pinebook Pro will support Panfrost. On a smaller scale, Mali chips are ubiquitous in phones and tablets; Panfrost will help the postmarketOS project achieve one of their stated goals, running the mainline kernel on phones for Linux for long-term support.

Looking back so far, what did you folks enjoyed the most with working around FOSS projects and communities?

The people! No matter where I go in the FOSS world, there’s always a friendly face. In real life, I can sometimes be timid, but online in the open source community, I can always hop into an IRC channel and strike up a chat with a developer or a fellow user. That sense of community, that despite coming from a myriad of countries, timezones, and identity backgrounds, we’re all united by a common purpose — that is a breath of fresh air from societies so focused on individual competition.

What are some inspirational lessons which you want to share with us, which will inspire newcomers contributing to FOSS ?

You can make a difference in the world of free software. It’s easy to be jaded and feel that nothing we do matters, that the tides of the world are set in stone at the whim of someone more powerful. Sometimes that can be true, but in the free software community, everyone has a chance to make a difference. If you can code, find an interesting open-source project to contribute to. If you’re multilingual, the community is always looking for translators. And even if you’re just an end-user, testing counts — if something doesn’t look right or doesn’t seem right, file a bug report and let the developers know! Or, if you’re a little extroverted and knowledgable on some software (even as a user!), try hanging out on your favourite project’s IRC channel and helping other users with the software — you never know whose day you could be improving with some patience and a little kindness. Little changes add up to making free software the beautiful place it is today, and you can help.

How can someone become involved?

Try Panfrost! Panfrost is shipping with Linux 5.2 and Mesa 19.2, arriving in popular distributions shortly. If you have a board with a compatible Mali GPU, grab the open-source stack and start testing! Maybe try your favourite desktop environment, or grab an open-source video game compatible with OpenGL 2.1, like SuperTuxKart. Give it a spin!

Edited for content and grammar. Images provided by Alyssa Rosenzweig, licensed CC-BY-SA 4.0.

Some Flatpak updates

Flatpak development is not standing still. Here is a quick summary of recent and coming changes.

Better extensions

In 1.4.2, Flatpak gained the ability to use extra-data for extensions. This mechanism has been around for applications for a long time, but it is a new feature for extensions.

The 19.08 version of the freedesktop runtime uses it for its new org.freedesktop.Platform.openh264 extension, which uses the Cisco openh264 builds.

Since we are taking the ‘run everywhere’ aspect of Flatpak seriously, we’ve backported this feature from the 1.4 branch to older stable branches and released 1.2.4 and 1.0.9, so even users on very stable distributions can enjoy this new feature.

Future plans

We’ve quietly started to work on Flatpak 1.6, which should be out before the end of the year.

On the roadmap for the this release, we have

  • Support for masking updates and pinning apps.  This gives users more control about what updates Flatpak installs, without having to answer questions every time.
  • Parental controls. This optional feature uses libmalcontent to implement policies about what applications users can install and run, based on OARS content ratings.
  • Disk space checks. This is an ongoing effort to improve the accuracy of our disk- and download-size handling and to handle low disk space situations more gracefully.
  • Infrastructure for purchases/donations. This is still a bit of a research topic.

You can follow the discussion around these features, the flatpak roadmap and general flatpak topics on the flatpak mailing list.

Coming soon to portals

Things are happening on the portal side too. Some of these have already landed, and will appear in a release soon.

Secrets

We have a secrets portal now.  It works by providing a master secret to the sandboxed app, which is then used to store the applications secrets in an encrypted file inside the sandbox . The master secret is stored in the session keyring.

This is nice in that applications don’t leave their secrets behind in the keyring when they are uninstalled, and the application secrets are safe from others.

The backend for this portal will be provided by gnome-keyring and libsecret will automatically use it inside a sandbox. Backend implementations for other environments are more than welcome.

The secret portal is the work of Daiki Ueno, who gave a talk about it at Guadec.

Self-updates

The Flatpak commandline and tools like Discover or the Elementary app store do a fine job of handling updates for Flatpak apps and runtimes.

But the reality is that self-updating is a popular feature for applications, so we added an update portal that lets them do this in a clean way, with proper integration in the Flatpak machinery.

Backgrounds 1

The background portal monitors applications that are running in the background (without open windows). It gives apps a way to request permission to run in the background, and it notifies users when apps are trying to do so sneakily without permission. The portal also lets applications request to be started automatically when the user logs in.

To implement this, the portal needs information from the compositor about open windows, and which applications they belong to. Currently, this is implemented for gnome-shell, other backends are more than welcome.

Window sharing

The screencast portal now lets you select individual windows, in addition to screens, if the application asks for this.

For now, the portal identifies windows by the application icon and window title. We are looking to improve this by using thumbnails.

Backgrounds 2

We will add a small bit of desktop integration with a portal for setting desktop wallpapers.

A portal library

In the ideal case, portal functionality is used transparently by existing desktop libraries without the need for apps to do anything special. Examples for this are GtkFileChooserNative using the file chooser portal, or libsecret using the new secret portal.

But for some portals, there is no natural library api, and in these cases, doing the portal interaction with D-Bus calls can be a bit cumbersome.

Therefore, we are working on a libportal library that will provide GIO-style async apis for portal requests.

Open for contribution

If you want to get involved with Flatpak development, or are just curious, check out the flatpak project on github, chime in on the Flatpak mailing list, or find us on IRC in #flatpak on freenode.

GUADEC 2019 | Part 1: Passing the Baton

This year, GUADEC was held in Thessaloniki, Greece from August 23rd – 28th. I had a great time at the conference and took some time to travel after, so I was able to see some of Northern Greece, in addition to hanging out with some of the best people I know while at GUADEC.

Since there’s a lot of talk about, I’ll be doing two separate posts, one about the Board meeting (in this post), and one about the conference itself (next post). 

Board Handover Meeting

I arrived in Thessaloniki a few days prior to GUADEC for the Board handover meeting. I really enjoyed my time as President and Chair of the Board, so passing the baton to the new Board was a bittersweet moment for me.

For those of you wondering, I didn’t run for the Board again this year because I won’t have as much time to dedicate to GNOME in the upcoming months. I pride myself on being a really active and proactive member of the Board, so having enough time to spend working on GNOME-related things is important to me. 

Don’t worry though, I’ll still be contributing to GNOME. For example, I’m one of the lead organizers of the Linux App Summit, and am helping with some other big initiatives, such as our Diversity and Inclusion work, and measuring impact at GNOME

Setting Strategic Goals

Helping architect the strategy that the GNOME Foundation is following is one of my proudest accomplishments because I think it’s the most far-reaching thing that I’m leaving behind. 

When I first started on the Board in 2016, I began questioning a lot of our budgetary categories and pushing for the Board to consider goals for the Foundation and how to use the budget as an instrument of achieving those goals. I established an annual hackfest in order to start working more strategically as a Board and to do a deep-dive on the budget. 

Flash forward, and we now have goals for the Foundation! We started these at last year’s Foundation Hackfest, and Philip Chimento and I spent time earlier this year refining them.

During the Board meeting at GUADEC, we gave our list of goals as input to our Executive Director, Neil McGovern, and he helped us turn them into something that could be adopted by the Foundation and presented during the Annual General Meeting (AGM).

Here are the Foundation’s long-term goals:

1. Sustainable Project & Foundation 

  • Sustain and increase funding levels
  • Increase number of contributors
  • Create and sustain infrastructure for Foundation Staff

2. Increased User Base

  • Foster a vibrant Linux desktop 
  • Uphold reputation as the most accessible desktop
  • Support improving the basic function of a desktop for everyone

3. Wider Awareness Through Leadership 

  • Develop better marketing and outreach tactics
  • Become an exemplary FOSS community
  • Evaluate and adopt new technologies to stay competitive with proprietary desktops

If you’d like to learn more about what these goals actually mean for the Foundation, check out Neil’s talk about GNOME’s Growth

It’s important to state that this is just the first step. The Foundation still needs to create KPIs, or something similar, in order to track and measure the Foundation’s progress towards the goals. 

Having publicly announced the Foundation’s goals is a move towards being a more data-driven organization and for us to be able to measure the GNOME Foundation’s impact. It marks a new stage of maturity for the GNOME Foundation, and I’m glad to have been a part of it.

Defining Board Roles

Since this GUADEC was later than other years, we held officer elections a few weeks prior to meeting in-person and had most of the transition stuff already out of the way. 

For those of you who missed it, these are the new Directors and relevant officer roles: 

  • Rob McQueen – President
  • Allan Day – Chair, Vice President
  • Carlos Soriano – Treasurer
  • Philip Chimento – Secretary
  • Federico Mena Quintero – Vice Secretary
  • Tristan Van Berkom
  • Britt Yazel

This year, the Board decided to split the Chair and President roles and adopt the gender neutral term “Chairâ€� instead of “Chairman.â€� The distinction here is that the Chair helps to run meetings, while the President has some differentiating management roles and special abilities, like signing power for the Foundation. The President is also usually the one who speaks at conferences on behalf of the Foundation, in addition to the Executive Director, although, really, any Director can do so. 

Rob and Allan will work as a team to manage Neil since we found that having a duo approach worked well last year when Rob and I comprised the management team. 

If you’re interested in putting names to faces, check out this year’s Annual General Meeting (AGM), where the Foundation Directors and staff were presented to the community.

Approving Committees

One of the things that the Board needs to do each year is to re-approve committees. We found this out while I was on the Board, when we were re-evaluating the Foundation’s structure and doing a deep dive into the Board’s delegated powers. So, now, we approve committees and members each year. 

In order to make sure that committees are functioning well and that they have the support they need from the Board, we decided to create liaisons to the committees last year. 

These liaisons are supposed to meet with the committee members they represent at least twice a year and try to understand any pain points that the committees may have, as well as to relate the Foundation’s goals and make sure that the committees think through ways to support those goals. 

Here’s a list of the new committee liaisons for the upcoming year:

  • Engagement – Britt Yazel
  • Membership – Tristan 
  • Code of Conduct – Federico 
  • Travel – Philip Chimento
  • Sponsorship – dissolved since we now have a member of the Foundation staff to work on fundraising for the Foundation

During the meeting, we also talked about re-evaluating the committee’s membership in order to make sure that only active members of committees have access to sensitive data. This is something that the new Board will be following up on this year. 

Handing Over Tasks

At the same time that the rest of the GNOME Project moved to using GitLab, the Board created a GitLab project in order to keep track of our open issues. This allowed us to prioritize initiatives and work more effectively on complex issues. 

Luckily, I had completed most of the tasks assigned to me, so there wasn’t much for me to hand over to new board members. 🙂

—–

That’s it for this post. Now onto the actual conference and BoF days.

October 02, 2019

VDA 0.90 Beta 1 Released

Vala Data Access library has reached a 0.90 Beta 1 release.

VDA provides a set of interfaces to wraps database connection, execution of SQL commands and access to returned values of the queries. Read the previous introduction post.

This version supports:

  • First Beta version of VDA
  • Native implementation for PostgreSQL
  • GDA implementation with support for SQLite and PosgreSQL
  • Basic SqlValue implementation and conversion between values
  • Basic SQL commands parsing support for SELECT, INSERT, DELETE, UPDATE, using parameters in order to provide the values required to execute it
  • Object Oriented API for SQL commands: SELECT, INSERT, DELETE, UPDATE
  • Support for Parameters in queries using GDA declaration syntax
  • SQL command parsing use GCalc from GNOME Calculator
  • Async queries execution
  • Support to map Row’s values to GObject based classes’ properties using DataObject interface
  • Support for row modification/insert using DataObject interface
  • DataObject supports Vda.SqlValue properties, both generic or specific
  • DataObject support conversion between Vda.SqlValue values and between basic properties’ types like string, int, double, float, bool

No pre-releases was made, because the API was changed while implementing database providers and included most of the interfaces to implement for better more powerful features.

VDA return Vda.SqlValue objects when you access a Table’s row’s column’s value; the provider is responsible to create it when is requested, so no overhead is present for SQL queries execution.

GUADEC 2019

Meeting my fellow GNOMies is something I look forward to every year. For eight years now I have traveled to participate in GUADEC and returned home with my head thinking of next year’s edition of the conference.

This year, I was busy with lots of activities, but still, I managed to chill with the friends I work with online throughout the whole year.  Putting faces into new names is also something very pleasant in these opportunities.

In the pre-registration party, I hosted a “Newcomers dinner“. Not many people could attend because of their personal travel plans, but those that participated were excited about being at the conference and getting to know so many cool people.

Besides that, it was the first GUADEC that we had a trained Code of Conduct Incident Response Team. We did an extensive training workshop with Otter Tech. Highly recommended!

Right at the first talks day, I hosted the interns’ lightning talks, that thanks to the amazing local team, are recorded and available online. The audience (and myself) were enthusiastic about hearing from the interns. After a few years of organizing these activities, I can still remember myself being an intern and giving my lightning talk back in 2012. Time flies! :-)

The quality of talks is always outstanding, so I listed below the ones I attended and recommend watching online:

  • Desktop Secrets Management for the Future by Daiki Ueno: I have lately been heavily interested in application sandboxing, so it was great catching up with Daiki’s work and ideas for our keyring story.
  • Managing GNOME Sessions with Systemd by Benjamin Berg and Iain Lane: It is being great and educative to follow their progress throughout the years on this task. The cherry on the cake is seeing this work landing and explained in Benjamin’s blog post.
  • Designing Multi-Process Application Security by Christian Hergert: Watching Christian talk is always exciting and educational. We are lucky to have such skillful developer in our project, and I definitely learned valuable lessons on application security.
  • Portals – Principles and Practice by Matthias Clasen: As I mentioned above, I have been lately interested in application sandboxing, so I couldn’t miss Matthias’ talk on Portals. It is so nice to see our application ecosystem evolving with Flatpak and its technologies.
  • GNU HEALTH: The Fight for our Rights in the Public Health System by Luis  Falcón: I personally care a lot about such social issues especially for being myself originally from the developing world, where people often don’t enjoy the same rights people in the developed world take for granted. The keynote was very well chosen.
  • Environmentally Friendly GNOME by Philip Withnall: IMPORTANT! We are running out of time to stop climate change, and I think every segment of society needs to discuss the issue. I hope to see the ideas discussed in this talk brought forward in our community.
  • Simple is Hard – Creating Beautiful App Icons by Jakub Steiner: Jimmac is so creative and talented that I can’t ever miss his talks. It is great to work with the design team on a daily basis, and this was a good opportunity to better understand their creative processes.
  • Accessibility Features for Mutter/GNOME Shell on Wayland by Oliver Fourdan: This work is very important. Oliver has made significant progress in shrinking the accessibility gap we currently have. Thanks for that!
  • Designing GNOME Mobile Apps by Tobias Bernard: Exciting work! It was great to see their progress on making GNOME apps adaptative. I hope this can make our platform even more attractive to vendors interested in building mobile OSes.
  • The Growth of GNOME by Neil McGovern: It is very reassuring listening to Neil describe the plans of growth for the GNOME Foundation.
  • Lightning Talks: It is always fun to see fellow GNOMies delivering their talk considering the lightning talks’ time constraint. :-D

During the BoF days, I conducted the Newcomers workshop, where we had various participants learning hands-on how to make their first code contribution to the GNOME project. Thanks everyone that showed up to participate and to help newcomers. I hope we can improve and repeat the workshop all over the world. GNOME.Asia will have its edition of the Newcomers workshop, so if you will be around in Gresik, don’t miss it!

In the Boxes BoF we discussed a roadmap to land some highly anticipated features such as UEFI support, Import/Export VMs, etc… Stay tuned here and also in the @BoxesGNOME Twitter account, where I have been doing outreach for our project by interacting with a part of our user base [wherever they are].

The social events were a blast. We had delicious food, great music, and passionate conversations at the Gala Dinner. The Picnic Day was fun and relaxing. The Museum BoF was enjoyable and nerdy (how I like it ;-)).

Checkout the photos!

Thanks to my employer, Red Hat, for sponsoring my trip and accommodation in the beautiful Thessaloniki!

October 01, 2019

A new time and life next steps

the opensource symbol

Since the beginning of my career in 1998 I’ve been related with Linux and opensource in me or other way. From sysadmin I grow to distro making, hardware certification and finally consulting, plus some other added skills. Parallel I developed a personal career in libre software communities and got the privilege to give lots of talks particularly in Spain and Ibero-America. That was a big time. All this stopped in 2011 with the combination of the big economic crisis in Spain and a personal psychological situation. All lead me to go back from Madrid to my home city, Almería, to look for health recovering. Now, after several years here I’m ready to take a new step and reboot my career.

Not all this time has been wasted. I dedicated lots of hours to a new project which in several senses has been the inverse of the typical practices in opensource communities. Indeed, I’ve tried to apply most of them but instead in the world-wide Internet now with a 100% hyper-local focus. This mean working in the context of a medium-small city (less than 200k inhabitants) with intensive in-person meetings and Internet communications support. Not all the results has been as successful as I pretended, probably because I kept very big expectations; as Antonio Gramsci said «I’m a pessimist because of intelligence, but an optimist because of will» :-) The effort was developed in what we named HackLab Almería and some time ago I wrote a recap about my experience. To me was both an experiment and a recovering therapy.

That time worked to recover ambitions, a la Gramsci, and to bring relevant important and itinerant events to our nice city, always related with opensource. Retaking the experience of the good-old HispaLinux conferences we were able of hosting a set of extraordinary great technological conferences: from PyConES 2016 to Akademy 2017, GUADEC 2018 and LibreOffice Conference 2019. For some time I thought Almería was the first city to host these three… after I realized Brno did it before! The icing of the cake was the first conference on secure programming in Spain: SuperSEC. I consider all of this a great personal success.

Forgot to mention I enrolled in a university course too, more as a excuse to work in an area for which I have never found time: information and software methodology modeling. This materializes in my degree project, in advanced development state but not yet finished, around the ISO/IEC 29110 norm and the EPF Composer. I’m giving it a final push in the coming months.

Now I’m closing this stage to start a new one, with different priorities and goals. First one is to reboot my professional career, so I’m looking for a new job and started a B2 English certification course. I’m resuming my participation in opensource communities —I’ll attend LAS 2019 next November— and hope to contribute with small but not trivial collaborations to several communities. After all I think the most I’ve been doing all these years has been just shepherding the digital commons.

See you in your recruitment process! ;-)

PS: this is an Spanish version of this post.

GNOME 3.34 is now managed using systemd

If you are already using GNOME 3.34, then most likely your session is managed using systemd right now. For a long time now we were already running a systemd instance for every user, which is used to launch DBus and for DBus activated applications. So, with GNOME 3.34, we finally took the next step and moved the rest of the session over to run using systemd.

From a user’s perspective nothing should have changed and at this point I believe that most regressions have been dealt with. Neither will this change affect application developers for the time being as XDG autostart files continue to be supported and are prefered at least for the time being.

The great thing is, that this enables further improvements. There has been a lot of work to allow Xwayland to be started on demand and systemd plays a small part in that feature. Similar, we will also be able to shut down services that are only needed if specific hardware is present (e.g. smartcards). Also, using systemd it is now easy to sandbox all components which will give you an extra bit of security.

That said, there are a few changes, concepts and general information that is worth mentioning.

Slices, scopes and user sessions

On a systemd managed system each user is assigned a user-X.slice (systemd.slice(5)) and the user’s session will be run in a session-Y.scope (systemd.scope(5)). You can also see a few other user specific units on the host, including user@X.service, which is the user’s systemd instance. This is a separate systemd process for the user and it will shut down again if the user is not logged in anymore.

With the systemd move, not only DBus activated applications and services, but rather your entire session is now launched using your user’s systemd instance. This has a few side effects that may seem odd at first. For example, the previously mentioned session-Y.scope which used to contain over 200 processes is now down to a mere 4 processes. Another side effect is that it got harder to understand which session a process belongs to (this is relevant for a number of services) or that ps will not show a tty anymore.

But, we have addressed these side effects and hopefully there are no regressions at this point. Your GNOME session is still invariably bound to the session-Y.scope (e.g. using loginctl kill-session continues to work reliably). And services have been updated to understand the new regime and pick the correct session in all cases. To handle all this, new API was also added to the systemd DBus interface and further improvements may happen in this area.

Trying it out

So, if you have GNOME 3.34, you can now apply all the neat tools that systemd has manage your user’s session. Just remember to add the --user option, and things should work as expected. A good candidate for trying all this out is gsd-media-keys.

If we look at systemctl --user, we’ll find two entries:

  • gsd-media-keys.target
  • gsd-media-keys.service

Note that failed units will not show up in the list, so it is advisable to always check the log if you suspect a service failure. Unfortunately this is needed so that you can reliably log in again after a session failure.

We can control gsd-media-keys.target (not gsd-media-keys.service), so you can try stopping and starting it and you will notice that most global keybindings will stop and start working.

  • systemctl --user stop gsd-media-keys.target
  • systemctl --user start gsd-media-keys.target

We can also pull up the log messages for the service from journalctl.

  • journalctl --user -u gsd-media-keys.service

But, unfortunately, it will not log much information by default. But, knowing systemd and GLib environment variables we can run:

  • systemctl --user --runtime edit gsd-media-keys.service

and write:

  • [Service]
    Environment=G_MESSAGES_DEBUG=all

This enables debug messages when the service is restarted next. The configuration will not persist as we passed --runtime. If you now restart gsd-media-keys.target and inspect its log again, you will notice that it contains a lot more information.

Developing GNOME

If you have a development version of GNOME installed somewhere outside of your normal path (e.g. jhbuild) and use this to log in, then you may need to update your setup. In jhbuild there is an example jhbuild-session script that ensures that the correct unit files will be used. The relevant lines copy them into the user’s $XDG_RUNTIME_DIRECTORY and reload the systemd daemon.

On a final note, I would like to thank everyone who has worked on this in the past. As far as I know, the first experimentations were done by Canonical, in particular Iain Lane did a lot of work and submitted the first patches. I picked up this work and made plenty of improvements to get it over the finishing line.

If you don’t have GNOME 3.34 yet, then try it out by installing Fedora 31 beta or your favourite distribution that includes GNOME 3.34 already.