December 08, 2022

GNOME Asia 2022 Experience

Hai semua

In this blog, I'm pumped to share my experience attending GNOME .Asia 2022 Summit held in Kuala Lumpur, Malaysia from 2nd December - 4th December.

Let's start :)

During the summit I presented on two topics - Topic 1: "Starting the open-source journey, and sharing my GSoC experience" which was of 30 Minutes and Topic 2: "Pitivi - GTK4 port" which was of approx 5 min.

The conference was filled with amazing experience, but the journey to the stage was quite hard.

The tough times began with the unavailability of appointments for my passport, which delayed it to November, which got again delayed due to issues with address verification as I just started living in hostel.

But, fortunately, due to the constant support of my family, and my undying dedication to attending the conference, we got everything under control, then comes the visa. In a pleasant surprise, unlike others I got it in under 2 hours, the thing I was most scared of got to be the easiest lol.

Then what, with the help of the GNOME Foundation, I got my tickets confirmed (Thanks Melissa), and I got ready to fly, btw, this was my first ever flight, I never had the chance to even fly domestically, so exploring the airport, immigration, customs, the security check was all an adventure for me.

Flight

The touchdown

touchdown

On reaching Kuala Lumpur we had a cab waiting for us, which was a really great gesture by the Organizing committee. Also, thank you Syazwan for waiting with me there and organizing the pickup.

There I got to unite with my fellow GSoCers - Pooja Patel, Asmit Malakannawar, Aman, and Anupam. They were a delight to meet, the trip became 1000x more fun due to them.

group photo

We then reached the hotel, freshened up, and did some sightseeing. And let me tell you, Kuala Lumpur is just beautiful.

The first day of summit

We got to meet and interact with so many new people. It was just awesome. The DEI workshop by the one and only Jonna was really cool, it was something we don't see much in conferences but is a much-needed activity. I also wrote some thank you packets as an activity :D

Thank you packets

Then the newcomer's workshop by Felipe was great, I along with other attendees missed him at the conference, but even virtually, his talk was really nice.

After the summit, we again went on to explore the city, unfortunately, it rained and delayed our plans, thus we weren't able to visit every place. But it was still awesome. We visited a fountain show, and it was the most amazing thing.

Fountain Petronas

Second day of summit

On the second day, my talk was the first in line, I presented on the topic - "Starting the open-source journey, and sharing my GSoC experience" unfortunately I got a bit late due to timezone issues, but nonetheless, it was my fault, but everyone was so nice about it and didn't let me feel down. Again, the Open Source community is the besttttttt.

The talk can be accessed at - GNOME Youtube

Talk

After receiving some really nice compliments, we proceeded to attend other talks, the talks by Anisa Kuci, Mantoh Kuma, and Matthias Clasen were just amazing. After which we got to listen to the amazing - Kristi Progri and Sammy fung in their lightning talks. By the way, they have opened the bidding for the next GNOME.Asia, so if anyone wants to host it in their country, be sure to reach out to them :)

After this, we again went for some more sightseeing and shopping :D

If you ever visit KL, be sure to visit Petronas and or KL tower, the view from there is heavenly

View

Last day of summit

All things eventually come to an end, and the same happened with this trip. On the last day, we had our intern lightning talks, where I and my fellow GSoCers / Friends gave talks on our projects during our GSoC internship at GNOME Foundation. It was an astonishing experience, I actually didn't have my talk prepared as I was going to cancel it first due to some reasons, but everything got sorted hence I decided to present, thus it wasn't my best presentation, but I'm proud of it :)

The talk can be accessed at - GNOME Youtube

Thanks

In the end, I would just like to thank the whole organizing team - especially - Kristi, Fenris, Syazwan, Sammy Fung for all their amazing work, the event was a blast.

group pic 1 group pic 2

I would also like to thank GNOME Foundation for Sponsoring my trip, it wouldn't have been possible for me to attend without their support.

At last, I also want to thank my college KIET Group of Institutions, Ghaziabad for co-sponsoring the trip and taking care of miscellaneous charges.

Oh, btw, if you are vegetarian, you are up for an adventure in Malaysia, it is super hard and rare to distinguish veg and non-veg, and well, even finding veg options is tough, special thanks to my friends who well, accompanied me every time during this crazy adventure.

I would also like to say, I'm in love with the goodies I got at the event, they are super cool!

goodies

End

I hope to meet all these amazing people soon again. It hasn't been many days, but I have started to miss them. Open Source and GNOME have a special place in my heart. Hopefully, we will meet again at GNOME Asia or GUADEC or some other event:D

December 07, 2022

Making an Orbic Speed RC400L autoboot when USB power is attached

As I mentioned a couple of weeks ago, I've been trying to hack an Orbic Speed RC400L mobile hotspot so it'll automatically boot when power is attached. When plugged in it would flash a "Welcome" screen and then switch to a display showing the battery charging - it wouldn't show up on USB, and didn't turn on any networking. So, my initial assumption was that the bootloader was making a policy decision not to boot Linux. After getting root (as described in the previous post), I was able to cat /proc/mtd and see that partition 7 was titled "aboot". Aboot is a commonly used Android bootloader, based on Little Kernel - LK provides the hardware interface, aboot is simply an app that runs on top of it. I was able to find the source code for Quectel's aboot, which is intended to run on the same SoC that's in this hotspot, so it was relatively easy to line up a bunch of the Ghidra decompilation with actual source (top tip: find interesting strings in your decompilation and paste them into github search, and see whether you get a repo back).

Unfortunately looking through this showed various cases where bootloader policy decisions were made, but all of them seemed to result in Linux booting. Patching them and flashing the patched loader back to the hotspot didn't change the behaviour. So now I was confused: it seemed like Linux was loading, but there wasn't an obvious point in the boot scripts where it then decided not to do stuff. No boot logs were retained between boots, which made things even more annoying. But then I realised that, well, I have root - I can just do my own logging. I hacked in an additional init script to dump dmesg to /var, powered it down, and then plugged in a USB cable. It booted to the charging screen. I hit the power button and it booted fully, appearing on USB. I adb shelled in, checked the logs, and saw that it had booted twice. So, we were definitely entering Linux before showing the charging screen. But what was the difference?

Diffing the dmesg showed that there was a major distinction on the kernel command line. The kernel command line is data populated by the bootloader and then passed to the kernel - it's how you provide arguments that alter kernel behaviour without having to recompile it, but it's also exposed to userland by the running kernel so it also serves as a way for the bootloader to pass information to the running userland. The boot that resulted in the charging screen had a androidboot.poweronreason=USB argument, the one that booted fully had androidboot.poweronreason=PWRKEY. Searching the filesystem for androidboot.poweronreason showed that the script that configures USB did not enable USB if poweronreason was USB, and the same string also showed up in a bunch of other applications. The bootloader was always booting Linux, but it was telling Linux why it had booted, and if that reason was "USB" then Linux was choosing not to enable USB and not starting the networking stack.

One approach would be to modify every application that parsed this data and make it work even if the power on reason was "USB". That would have been tedious. It seemed easier to modify the bootloader. Looking for that string in Ghidra showed that it was reading a register from the power management controller and then interpreting that to determine the reason it had booted. In effect, it was doing something like:
boot_reason = read_pmic_boot_reason();
switch(boot_reason) {
case 0x10:
  bootparam = strdup("androidboot.poweronreason=PWRKEY");
  break;
case 0x20:
  bootparam = strdup("androidboot.poweronreason=USB");
  break;
default:
  bootparam = strdup("androidboot.poweronreason=Hard_Reset");
}
Changing the 0x20 to 0xff meant that the USB case would never be detected, and it would fall through to the default. All the userland code was happy to accept "Hard_Reset" as a legitimate reason to boot, and now plugging in USB results in the modem booting to a functional state. Woo.

If you want to do this yourself, dump the aboot partition from your device, and search for the hex sequence "03 02 00 0a 20"". Change the final 0x20 to 0xff, copy it back to the device, and write it to mtdblock7. If it doesn't work, feel free to curse me, but I'm almost certainly going to be no use to you whatsoever. Also, please, do not just attempt to mechanically apply this to other hotspots. It's not going to end well.

comment count unavailable comments

December 06, 2022

Where Fedora Linux Could Improve

Disclaimers

  1. I am not employed by Red Hat (yes, I have to say that).
  2. I may sound rude, so I sincerely apologize for that, as I am really critical in some of these topics.
  3. “Fedora Project” is referred to the organization. “Fedora Linux” is referred to Fedora operating systems, like Fedora Workstation, Fedora KDE Plasma Desktop Edition, Fedora Silverblue, and others. “Fedora Linux community” is referred to the Fedora Project/Linux community, such as developers, contributors and users.

Introduction

The Fedora Project is a great organization to gain experience no matter the team you are in. I am currently a part of the Fedora Websites & Apps team improving my technical writing, communication and design skills.

With all the things the Fedora Project does well, there are several places that, in my opinion, need to be improved. I’d like to go over some key areas where we could improve Fedora Linux from a user perspective without breaking the Fedora Project’s core philosophies.

Unification and Uniformity

In my opinion, Fedora Linux suffers from severe inconsistencies, such as website design and distribution naming scheme, which can be quite daunting to learn from an end user perspective.

Standard Styles and Layouts

At the moment, many Fedora Linux pages look inconsistent from one another. Fedora Workstation, Fedora Server and Fedora IoT follow a common style and layout, whereas Fedora Silverblue, Fedora Kinoite and Fedora CoreOS don’t. This means there are a total of four entirely different styles and layouts. This gives the impression that Fedora Silverblue, Fedora Kinoite and Fedora CoreOS are external projects and are barely related to the Fedora Project, rather than official projects endorsed by the Fedora Project.

I believe that we should focus on standardizing by unifying the styles and layouts in a single framework, instead of reinventing the wheel with no standardization in mind. Luckily, Fedora Kinoite and Fedora CoreOS website maintainers are in favor of unifying; hopefully everyone else is on board and we can provide a platform for internal projects.

Naming Convention

Similar to Fedora Linux pages, I believe Fedora Linux lacks a naming convention. For example, Fedora Workstation uses GNOME and uses the traditional Linux desktop paradigm for managing packages. On the other hand, Fedora Silverblue is visually the same as Fedora Workstation, with the main difference that it reinvents the Linux desktop paradigm to solve fundamental issues with the Linux desktop. The name “Fedora Workstation” conveys what it is intended for, whereas “Fedora Silverblue” doesn’t. In reality, “Fedora Silverblue” doesn’t have a clear meaning, and doesn’t even give a hint to users about what it is trying to do or be.

This lack of a naming convention expands with Fedora KDE Plasma Desktop Edition (or Fedora KDE for short) and Fedora Kinoite for Plasma, Fedora Server and CoreOS for server, and others.

Furthermore, with Fedora Silverblue/Kinoite/CoreOS, there is no concept of spins and editions. Despite being fundamentally the same, each is treated as an entirely separate project. For example, Fedora KDE is Fedora Workstation with KDE Plasma instead of GNOME, whereas on the immutable end, Fedora Kinoite is Fedora Silverblue with KDE Plasma. Despite this equivalency, Fedora KDE is a spin, whereas Fedora Kinoite is a whole separate distribution, when it could be a spin instead.

To address this, a category that englobes all immutable distributions would not only improve the naming scheme, it would also be easier for users to reference all immutable distributions without needing to enumerate through them. For example, instead of Fedora Silverblue, it could be Fedora Immutable Workstation, or Fedora Atomic Workstation. For Plasma, it could be Fedora Immutable KDE. To reference all of them, we could call them “Fedora Immutable” or “Fedora Atomic”. This could also help us reintroduce spins and editions.

Only Ship Unfiltered Flathub by Default

Flatpak is pre-installed on Fedora Linux. It comes with a heavily filtered variant of Flathub and comes with Fedora Flatpaks, Fedora’s Flatpak remote.

Typically, distributions that pre-install Flatpak also pre-add an unfiltered Flathub, as it is the de-facto and most popular remote. I am personally completely against pushing Fedora Flatpaks and filtering Flathub, as it is a hostile approach in pushing Flatpak, and is unsustainable.

The Problems With Fedora Flatpaks

Since the very beginning of Fedora Flatpaks’s existence, the Fedora Project pushes it unconditionally, all without putting the effort to convince users and developers for any practical benefits. This is really harmful to Flatpak adoption and to Fedora Linux in general, as users (and even contributors) don’t have useful information regarding Fedora Flatpaks and its benefits. This leaves a distasteful experience with Fedora Flatpaks to users, while believing that this is a problem with Flatpak itself (like myself when I first tried them).

Fedora Flatpaks lacks many contributors and support, as many more developers and organizations are interested in Flathub. Furthermore, there is no support room/channel related for Fedora Flatpaks. In contrast, the Flatpak Matrix room is a shared room for Flatpak and Flathub. So, not only is there no proper support for Fedora Flatpaks, there aren’t enough contributors and developers to sustain Fedora Flatpaks.

10 months ago, I wrote an article at the Fedora Magazine to compare and contrast Fedora Flatpaks and Flathub. In the “Number of applications” section, I checked the amount of applications available in Fedora Flatpaks and Flathub. At that time, Fedora Flatpaks had 86 applications available, whereas Flathub had 1518. Let’s compare the numbers as of writing this article:

$ flatpak remote-ls --app fedora | wc -l
87
$ flatpak remote-ls --app flathub | wc -l
1988

We notice that, 10 months later, Fedora Flatpaks only published one application, whereas the amount of applications in Flathub went from just above 1500 to almost 2000. Fedora Flatpaks was first introduced in late 2018. In the span of 4 years, Fedora Flatpaks still hasn’t reached 100 applications.

Many developers and users do not consider Fedora Flatpaks appealing, myself included. This is dangerous for Flatpak’s future, as Fedora Linux is meant to be appealing for developers. However, developers side with Flathub instead, including those who develop free and open source software. These developers will only publish their applications on Flathub thanks to Flathub’s much wider coverage in support and discoverability.

Personally, I believe that Fedora Flatpaks is very anti-collaborative and hostile on the Fedora Project’s end, especially when the Fedora Project has a long history of collaborating with upstream projects to advance the Linux desktop. With Flathub, we notice that GNOME, KDE, elementary, Endless Foundation, and many other free and open source organizations collaborate for Flathub development and adoption, whereas the Fedora Project is one of the exceptions.

Why Only Flathub Should Be Pushed

We’re not only noticing usability paradigm shifts, we’re also noticing a community paradigm shift, where major communities with different backgrounds and goals agree, develop and adopt the same standard, instead of implementing everything internally and risking worsening fragmentation. Having the Fedora Project involved in Flathub development and adoption and dropping support for Fedora Flatpaks would seriously improve the Linux desktop (and mobile) in the long run.

We have noticed a consistent positive outcome when many communities come together and work on a single standard, notably systemd, freedesktop.org standards or even the infamous Linux kernel. When we all adopt the same standards, the improvements become exponentially larger.

From a legal perspective, Flatpak introduces a method to work around proprietary licenses and codecs, as explained in this video. Everything Flathub develops is entirely free and open source, so this does not go against the philosophy that the Fedora Project preaches. Furthermore, store front-ends, such as GNOME Software and KDE Discover, even have a dedicate section that explains the risks of proprietary software.

Adwaita-Qt

When I first used Qt applications on Fedora Linux on GNOME, many of them were literally unusable, due to styling issues. This is because Fedora Linux comes with Adwaita-Qt, “[a] style to bend Qt applications to look like they belong into GNOME Shell”. Here are some examples of usability issues:

Dolphin Kate Konsole

Initially, I had no good knowledge with Qt, so I thought these were application issues. Since the Fedora Linux community often claims that Fedora Linux provides a “vanilla” experience, the last thing I would expect is it modifying the interface and behavior of applications. Unfortunately, I was wrong – Fedora Linux does not provide a vanilla experience, as it modifies Qt applications on GNOME.

After 30 minutes of fiddling with styling, I finally found the solution: to use Breeze, the default style in Plasma. In my case, I uninstalled Adwaita-Qt from my systems.

The main problem with styling is that it is literally impossible to be as reliable as the most common style (in this case, Breeze). Most developers test Qt applications with Breeze, without testing with other styles. Breeze itself isn’t perfect, but it has the narrowest scope of breakage, but any custom style will always broaden the scope, and thus will increase the chance of breaking user experience. Shipping custom styles by default may make applications appear broken, and may even give the impression to the user that the application is at fault, which initially happened to me, as explained above.

I strongly urge the Fedora Project to stop shipping Adwaita-Qt by default. While I have no problem with custom styles existing, breaking applications by default is harmful to the relationship between application developers, distribution maintainers and users. The best we can do is ship however the developer tested their application, and let users break their system themselves.

Conclusion

Fedora Linux is my favorite Linux distribution, because it does many things right, especially the amount of effort the community is putting to advocate for free and open source software and community involvement. However, it is not perfect. There are areas I certainly disagree with, like pushing Fedora Flatpaks and Adwaita-Qt.

I personally believe that we should push big organizations into collaborating with other organizations and the rest of the community, instead of potentially creating an unsustainable alternative. I also believe that we should protect the user and developer experience by not modifying the visuals of the program, to avoid conflicts and guarantee the best experience to the user.


  • Edit 1: Update Flathub information (Credit to Cassidy James)
  • Edit 2: Correct typos and improve sentences

December 04, 2022

Color management, this time with PDF

In previous posts the topic of color management in Cairo was examined. Since then people have told me a few things about the issue. According to them (and who am I do to proper background research and fact checking, I'm just someone writing on the Internet) there are a few fundamental problems with Cairo. The main one is that Cairo's imaging model is difficult to implement in GPU shaders. It also is (again, according to Internet rumors) pretty much impossible to make work with wide gamut and HDR content.

Dealing with all that and printing (which is what I was originally interested in) seems like a too large a mouthful to swallow. One thing lead to another and thus in the spirit of Bender, I wrote my own color managed PDF generator library. It does not try to do any imaging or such, just exposes the functionality that is in the PDF image and document model directly. This turned out to take surprisingly little work because this is a serialization/deserialization problem rather than an image processing one. You just dump the draw commands and pixels to a file and let the PDF viewer take care of showing them. Within a few days I had this:

This is a CMYK PDF that is fully color managed. The image on the second page was originally an RGB PNG image with an alpha channel  that was converted to CMYK automatically. The red square is not part of the image, it is there to demonstrate that transparency compositing works. All drawing commands use the /DeviceCMYK color space. When creating the PDF you can select whether the output should be in RGB, grayscale or CMYK and the library automatically generates the corresponding PDF commands. All of the heavy lifting is done by LittleCMS, there are no unmanaged color conversions in the code base.

Since everything was so straightforward, I went even further. That screenshow is not actually showing a CMYK PDF. The yellow text on the purple background is a spot color that uses a special gold ink. Thus the PDF has five color channels instead of four. Those are typically used only in high quality print shops for special cases like printing with specific Pantone inks or specifying which parts of the print should be em/debossed, varnished or the like.

What would it take to use this for realsies?

There does seem to be some sort of a need for a library that produces color managed PDFs. It could be used at least by Inkscape and Gimp, possibly others as well. In theory Cairo could also use it for PDF generation so it could delete its own PDF backend code (and possibly also the PostScript one) and concentrate only on pushing and compositing pixels. Then again, maybe its backwards compatibility requirements are too strict for that.

In any case the work needed splits neatly into two parts. The first one is exposing the remaining functionality in PDF in the API. Most of it is adding functions like "draw a bezier with values ..." and writing out the equivalent PDF command. As the library itself does not even try to have its own imaging model, it just passes things directly on. This takes elbow grease, but is fairly simple.

The other part is text. PDF's text model predates Unicode so it is interesting to say the least. The current code only supports (a subset of) PDF builtin fonts and really only ASCII. To make things actually work you'd probably need to reimplement Cairo's glyph drawing API. Basically you should be able to take PangoCairo, change it a bit and point it to the new library and have things work just as before. I have not looked into how much work that would actually be.

There are currently zero tests and all validation has been done with the tried and true method of "try the output on different programs and fix issues until they stop complaining". For real testing you'd need access to a professional level printer or Adobe Acrobat Pro and I have neither.

December 02, 2022

Automated testing of GNOME Shell

Automated testing is important to ensure software continues to behave as it is intended and it’s part of more or less all modern software projects, including GNOME Shell and many of the dependencies it builds upon. However, as with most testing, we can always do better to get more complete testing. In this post, we’ll dive into how we recently improved testing in GNOME Shell, and what this unlocks in terms of future testability.

Already existing testing

GNOME Shell already performs testing as part of its continuous integration pipeline (CI), but tests have been limited to unit testing, meaning testing selected components in isolation ensuring they behave as expected, but due to of the nature of the functionalities that Shell implements, the amount of testing one can do as unit testing is rather limiting. Primarily, in something like GNOME Shell, it is just as important to test how things behave when used in their natural environment, i.e. instead of testing specific functionality in isolation, the whole Shell instance needs to be executed with all bits and pieces running as a whole, as if it was a real session.

In other words, what we need is being able running all of GNOME Shell as if it was installed and logged in into on a real system.

Test Everything

As discussed, to actually test enough things, we need to run all of GNOME Shell with all its features, as if it was a real session. What this also means is that we don’t necessarily have the ability to set up actual test cases filled with asserts as one does with unit testing; instead we need mechanisms to verify the state of the compositor in a way that looks more like regular usage. Enter “perf tests“.

Since many years back, GNOME Shell has had automated performance tests, that would measure how well the Shell performed doing various tasks. Each test is a tiny JavaScript function that performs a few operations, while making sure all the performed operations actually happened, and when it finishes, the Shell instance is terminated. For example, a “perf test” could look like

  1. Open overview
  2. Open notifications
  3. Close notifications
  4. Leave overview

As is it turns out, this infrastructure fits rather neatly with the kind of testing we want to add here – tests that that perform various tasks that exercise user facing functionality.

There are, however, more ways to  verify that things behave as expected other than triggering these operations and ensuring that they executed correctly. The most immediate next step is to ensure that there were no warnings logged during the whole test run. This is useful in part due to the fact that GNOME Shell is largely written in JavaScript, as this means the APIs provided by lower level components such as Mutter and GLib tend to have runtime input validation in introspected API entry points. Consequently, if an API is misused by some JavaScript code, it tends to result in warnings being logged. We can be more confident that a particular change won’t introduce regressions when it runs GNOME Shell completely without warnings.

This, however, is easier said than done, for two main reasons: we’ll be running in a container, and the complications that comes with mixing memory management models of different programming languages.

Running GNOME Shell in a container

For tests to be useful, they need to run in CI. Running in CI means running in a container, and that is not all that straightforward when it comes to compositors. The containerized environment is rather different than running on a regularly installed and setup Linux distribution; it lack many services that are expected to be running, and provide important functionality needed to build a desktop environment, like service and session management (e.g. logging out), system management (e.g. rebooting), dealing with network connectivity, and so on.

Running with most of these services missing is possible, but results in many warnings, and a partially broken session. To get any useful testing done, we need to eliminate all of these warnings, without just silencing them. Enter service mocking.

Mocked D-Bus Services

In the world of testing, “mocking” involves creating an implementation of an API, without the actual real world API implementation sitting behind it. Often these mocked services provide a limited pre-defined subset of functionality, for example hard coding results of API operations given a pre-defined set of possible input arguments. Sometimes, mocked APIs can simply only be there to pretend a service available, and nothing more is needed unless the functionality it provides needs to be actively triggered.

As part of CI testing in Mutter, the basic building blocks for mocking services needed to run a display server in CI have been implemented, but GNOME Shell needs many more compared to plain Mutter. As of this writing, in addition to the few APIs Mutter relies on, GNOME Shell also needs the following:

  • org.freedesktop.Accounts (accountsservice) – For e.g. the lock screen
  • org.freedesktop.UPower (upower) – E.g. battery status
  • org.freedesktop.NetworkManager (NetworkManager) – Manage internet
  • org.freedesktop.PolicyKit1 (polkit) – Act as a PolKit agent
  • net.hadess.PowerProfiles (power-profiles-daemon) – Power profiles management
  • org.gnome.DisplayManage (gdm) – Registering with GDM
  • org.freedesktop.impl.portal.PermissionStore (xdg-permission-store) – Permission checking
  • org.gnome.SessionManager (gnome-session) – Log out / Reboot / …
  • org.freedesktop.GeoClue2 (GeoClue) – Geolocation control
  • org.gnome.Shell.CalendarServer (gnome-shell-calendar-server) – Calendar integration

The mock services used by Mutter are implemented using python-dbusmock, and Mutter conveniently installs its own service mocking implementations. Building on top of this, we can easily continue mocking API after API until all the needed ones are provided.

As of now, either upstream python-dbusmock or GNOME Shell have mock implementations of all the mentioned services. All but one, org.freedesktop.Accounts, either existed or needed a trivial implementation. In the future, for further testing that involves interacting with the system, e.g. configuring Wi-Fi, we will need expand what these mocked API implementations can do, but for what we’re doing initially, it’s good enough.

Terminating GNOME Shell

Mixing JavaScript, a garbage collected language, and C, with all its manual memory management, has its caveats, and this is especially true during tear down. In the past the Mutter context was terminated, later followed by the JavaScript context. Terminating the JavaScript context last prevented Clutter and Mutter objects from being destroyed, as JavaScript may still have references to these objects. If you ever wondered why there tends to be warnings in journal when logging out, this is why. All of these warnings and potential crashes mean any tests that rely on zero warnings would fail. We can’t have that!

To improve this situation, we have to shuffle things around a bit. In rough terms, we now terminate the JavaScript context first, ensuring there are no references held by JavaScript, before tearing down the backend and the Mutter context. To make this possible without introducing even more issues, this meant tearing down the whole UI tree on shut-down, making sure the actual JavaScript context disposal more or less only involves cleaning up defunct JavaScript objects.

In the past, this has been complicated too, since not all components can easily handle bits and pieces of the Shell getting destroyed in a rather arbitrary order, as it means signals get emitted when they were not expected to, e.g. when parts of the shell that was expected to still exist has already been cleaned up. A while ago, a new door was opened making it possible to handle rather conveniently: enter the signal tracker, a helper that makes it possible to write code using signal handlers that automatically disconnects signal handlers on shutdown.

With the signal tracker in place and in use, a few smaller final fixes here, and the aforementioned reversed order we tear down the JavaScript context and the Mutter bits, we can now terminate without any warnings being logged.

And as a result, the tests pass!

Enabled in CI

Right now we’re running the “basic” perf test on each merge request in GNOME Shell. It performs some basic operations, including opening the quick settings menu, handles an incoming notification, opens the overview and application grid. A screen recording of what it does can be seen below.

What’s Next

More Tests

Testing more functionality than basic.js. There are some more existing “perf tests” that could potentially be used, but tests that aim for testing specific functionality, for example window management, or configuring the Wi-Fi, that isn’t related to performance don’t really exist yet. This will become easier after the port to standard JavaScript modules, when tests no longer have to be included in the gnome-shell binary itself.

Input Events

So far, widgets are triggered programmatically. Using input events via virtual input devices means we get more fully tested code paths. Better test infrastructure for things related to input is being worked on for Mutter, and can hopefully be reused in GNOME Shell’s tests.

Running tests from Mutter’s CI

GNOME Shell provides a decent sanity test for Clutter, Mutter’s compositing library, so ensuring that it runs successfully and without warnings is useful to make sure changes there doesn’t introduce regressions.

Screenshot-based Tests

Using so called reference screenshots, test will be able to ensure there were no actual visual changes unless so was intended. The basic infrastructure exist in and can be exposed by Mutter, but for something like GNOME Shell, we probably need a way other than in-tree reference images for storage as is done in Mutter, in order to not make the gnome-shell git repository grow out of hand.

Multi-monitor

Currently the tests use a single fixed resolution virtual monitor, but this should be expanded to involve multi monitor and hotplugging. Mutter has ways to create virtual monitors, but does not yet export this via by GNOME Shell consumable API.

GNOME Shell Extensions

Not only GNOME Shell itself needs testing, running tests specifically for extensions, or running GNOME Shell’s own tests as part of testing extensions would have benefits as well.

On PyGObject

Okay, I can’t believe I have to do this again.

This time, let’s leave the Hellsite out of it, and let’s try to be nuanced from the start. I’d like to avoid getting grief online.

The current state of the Python bindings for GObject-based libraries is making it really hard to recommend using Python as a language for developing GTK and GNOME applications.

PyGObject is currently undermaintained, even after the heroic efforts of Christoph Reiter to keep the fires burning through the long night. The Python community needs more people to work on the bindings, if we want Python to be a first class citizen of the ecosystem.

There’s a lot to do, and not nearly enough people left to do it.

Case study: typed instances

Yes, thou shall use GObject should be the law of the land; but there are legitimate reasons to use typed instances, and GTK 4 has a few of them:

At this very moment, it is impossible to use the types above from Python. PyGObject will literally error out if you try to do so. There are technical reasons why that was a reasonable choice 15+ years ago, but most language bindings written since then can handle typed instances just fine. In fact, PyGObject does handle them, since GParamSpec is a GTypeInstance; of course, that’s because PyGObject has some ad hoc code for them.

Dealing with events and render nodes is not so important in GTK4; but not having access to the expressions API makes writing list widgets incredibly more complicated, requiring to set up everything through UI definition files and never modifying the objects programmatically.

Case study: constructing and disposing

While most of the API in PyGObject is built through introspection, the base wrapper for the GObject class is still very much written in CPython. This requires, among other things, wiring the class’s virtual functions manually, and figuring out the interactions between Python, GObject, and the type system wrappers. This means Python types that inherit from GObject don’t have automatic access to the GObjectClass.constructed and GObjectClass.dispose virtual functions. Normally, this would not be an issue, but modern GObject-based libraries have started to depend on being able to control construction and destruction sequences.

For instance, it is necessary for any type that inherits from GtkWidget to ensure that all its child widgets are disposed manually. While that was possible through the “destroy” signal in GTK3, in GTK4 the signal was removed and everything should go through the GObjectClass.dispose virtual function. Since PyGObject does not allow overriding or implementing that virtual function, your Python class cannot inherit from GtkWidget, but must inherit from ancillary classes like GtkBox or AdwBin. That’s even more relevant for the disposal of resources created through the composite template API. Theoretically, using Gtk.Template would take care of this, but since we cannot plug the Python code into the underlying CPython, we’re stuck.

Case study: documentation, examples, and tutorials

While I have been trying to write Python examples for the GNOME developers documentation, I cannot write everything by myself. Plus, I can’t convince Google to only link to what I write. The result is that searching for “Python” and “GNOME” or “GTK” will inevitably lead people to the GTK3 tutorials and references.

The fragmentation of the documentation is also an issue. The PyGObject website is off to readthedocs.org, which is understandable for a Python projects; then we have the GTK3 tutorial, which hasn’t been updated in a while, and for which there are no plans to have a GTK 4 version.

Additionally, the Python API reference is currently stuck on GTK3, with Python references for GTK4 and friends off on the side.

It would be great to unify the reference sites, and possibly have them under the GNOME infrastructure; but at this point, I’d settle for having a single place to find everything, like GJS did.

What can you do?

Pick up PyGObject. Learn how it works, and how the GObject and CPython API interact.

Write Python overrides to ensure that API like GTK and GIO are nice to use with idiomatic Python.

Write tests for PyGObject.

Write documentation.

Port existing tutorials, examples, and demos to GTK4.

Help Christoph out when it comes to triaging issues, and reviewing merge requests.

Join GNOME Python on Matrix, or the #gnome-python channel on Libera.

Ask questions and provide answers on Discourse.

What happens if nobody does anything?

Right now, we’re in maintenance mode. Things work because of inertia, and because nobody is really pushing the bindings outside of their existing functionality.

Let’s be positive, for a change, and assume that people will show up. They did for Vala when I ranted about it five years ago after a particularly frustrating week dealing with constant build failures in GNOME, so maybe the magic will happen again.

If people do not show up, though, what will likely happen is that Python will just fall on the wayside inside GNOME. Python developers won’t use GTK to write GUI applications; existing Python applications targeting GNOME will either wither and die, or get ported to other languages. The Python bindings themselves may stop working with newer versions of Python, which will inevitably lead downstream distributors to jettison the bindings themselves.

We have been through this dance with the C# bindings and Mono, and the GNOME and Mono communities are all the poorer for it, so I’d like to avoid losing another community of talented developers. History does not need to repeat itself.

#72 Automated Testing

Update on what happened across the GNOME project in the week from November 25 to December 02.

Core Apps and Libraries

Mutter

A Wayland display server and X11 window manager and compositor library.

Georges Stavracas (feaneron) announces

The Mutter & Shell team published an article about the recent developments on automating the testing of the compositor of the GNOME desktop. Have a read!

GTK

Cross-platform widget toolkit for creating graphical user interfaces.

Jordan Petridis reports

After more than a year of work, the rusty GStreamer Paintable Sink for GTK 4 has received support for GL Textures, reducing dramatically the CPU consumed (from 400-500% to 10-15% for the a 4k stream scenarios we tested) and allowing for zero-copy rendering when used with hardware decoders.

Settings

Configure various aspects of your GNOME desktop.

Georges Stavracas (feaneron) announces

GNOME Settings continues to receive polish, reviews, and new designs from various contributors. We’re already stashing exciting improvements for the next release:

  • Continuous improvements to the Device Security panel have been added. These improvements range from better wording of the security features, new designs for the dialogs, and making the panel more actionable. Thanks to Kate Hsuan and Richard Hughes for diligently working on this.
  • The Accessibility panel was redesigned. This is the first panel implementing a more modern navigation model in Settings. More panels should be redesigned for this navigation pattern in the future. Thanks to Sadiq for this.
  • The Date & Time panel is now more mobile friendly, by using a two-column layout to the month selector. This change was also thanks to Sadiq.
  • The Network & Wi-Fi panels now use libnma’s own security widgets for managing connections. This is a massive cleanup in the codebase, and allows us to concentrate efforts on a single place. This change was made possible by Lubomir Rintel.
  • Various polishes and smaller improvements to lots of panels, like Users, Wacom, Region and Language, and others

Circle Apps and Libraries

Gaphor

A simple UML and SysML modeling tool.

Dan Yeaw says

Gaphor, the simple modeling tool, feature release version 2.13.0 is now out! It includes some great updates, including:

  • Auto-layout for diagrams
  • Relations to actors can connect below actor name
  • Export to EPS
  • Zoom with Ctrl+scroll wheel works again

It is also got some great UI improvements, with GTK4 and libadwaita now the default for Linux and Windows.

Third Party Projects

lwildberg announces

I am announcing the first release of the new app “Meeting Point”. It is a video conferencing client using BigBlueButton in the background. At the moment the features exposed in the still experimental UI are:

  • join meetings (also with passwords) hosted by senfcall.de, a free BigBlueButton provider
  • watch webcam video streams of participants
  • read the public group chat
  • see a list of all participants
  • listen to audio (can be turned off)
  • delete the group chat history, if you are moderator

The source code is here. To try Meeting Point out, open it with GNOME Builder and hit the run button. New features will come soon! I will be also happy to guide anyone who is interested through the internals :)

Khaleel Al-Adhami says

Introducing Converter, a libadwaita GTK4 app that lets you convert and manipulate images in a sleek interface. It is built on top of imagemagick as well as other python libraries.

Converter 1.1.0 was released with SVG support and resizing operations! It’s available on flathub: https://flathub.org/apps/details/io.gitlab.adhami3310.Converter

Girens for Plex

Girens is a Plex Gtk client for playing movies, TV shows and music from your Plex library.

tijder says

Girens version 2.0.1 just released. With this version the transcoding protocol is changed to DASH. Thanks to changing the transcode protocol some resume playback bugs are fixed. Also now if items are loaded from the server in the section view a loading icon is showed. The sidebar containing the titles of the section have now an icon next to it. Also the translations are updated. For more details about the changes see the changelog.

Blueprint

A markup language for app developers to create GTK user interfaces.

James Westman reports

blueprint-compiler v0.6.0 is out! This is mostly a bugfix release, but it also adds a typeof() operator for specifying GType properties like Gio.ListStore:item-type. It also includes some internal refactoring that paves the way for some great features in the future, including alternate output formats!

BlackFennec

A beautiful and easy to use application for viewing and editing semi-structured data like JSON.

Simon announces

BlackFennec v0.10 is out!

This version introduces actions! It is now possible to execute functions on elements . Try out one of the new actions such as to lower on strings and clear collection on lists.

We have also added the ability to undo and redo any changes you made to your data; a handy feature if you accidentally cleared a collection 😉

My favorite new addition however is copy and paste. You can export or import any element to and from JSON via your clip board!

That’s all for this week!

See you next week, and be sure to stop by #thisweek:gnome.org with updates on your own projects!

Niepce November 2022 updates

Here are the updates in Niepce development for November 2022.

I'll try to bring these updates on a more regular basis. In case you missed it, there was a May update.

No more cbindgen

For this one, from a user standpoint, there is nothing really visible beside regressions, but given there won't be a release for a while, it's a non issue. Things will be addressed as they are found.

I decided to bite the bullet and get rid of cbindgen. A few reasons:

  • The way I implemented it caused build time to be much longer unless I disabled the bindings generation. Turn out that cbindgen parses the whole code based a second time to automatically generated the bindings, and I did make this step opt-out instead of opt-in to make sure the code was in sync. There would probably have been a different way to go about it, but I didn't.
  • cbindgen is one way (calling Rust from C/C++) and this limited its purpose. I had at one point bidirectional bindings with bindgen but it was also fragile. Boath are great tools, but like any tool they have limitations. I would still use cbindgen to generate the C headers of an API I want to export, and bindgen to generate the Rust FFI (Foreign Function Interface) for Rust bindings to a C API.
  • It's still a C API with all the issues. Raw pointers and manual cleanup, this made things more fragile at the FFI level.

I transitionned to cxx. cxx is a fantastic tool for bi-directional safe bindings between C++ and Rust. It provide strongly typed bindings, support for Box<> and C++ smart pointers, and C++ class syntax.

It wasn't an easy transition though, and I had already tried in the past. But really it was worth it.

Using cxx

The gist of cxx is that you write a ffi section that will declare, using Rust syntax, an interface for your code at the edge of Rust and C++. And then cxx will generate the glue code to call a C++ method from Rust or have C++ code call Rust methods. As a bonus, it will allow direct use of strings, vector, Box, std::shared_ptr, std::unique_ptr, which mean that you can have reasonable expectations of resource management and standard types that cbindgen required you to deal with. This is truly a game changer in comparison. A lot of the convoluted FFI I had to write before is not handled by the tooling. And the code is more idiomatic.

use crate::mod1::Type1;

#[cxx::bridge(namespace = "ffi")]
mod ffi {
    extern "Rust" {
        type Type1;

        #[cxx_name = "Type1_create"]
        fn create() -> Box<Type1>;
        fn method1(&self, name: &str) -> i32
    }
}

In C++ to use this you'd write:

#include "the_bindings.hpp"

void f() {
    ::rust::Box<ffi::Type1> object = Type1_create();
    uint32_t value = object->method1("some_name");
}

Note that the_bindings.hpp here is a generated header, so is its counterpart the_bindings.cpp (you pick the name) that should be compiled with the rest of your code.

However, there are pitfalls. Niepce is split is three crates. This is a problem as cxx doesn't handle cross-crate bindings as easily as it could.

For example, imagine there is a type A in crate_one, that likely have a binding. crate_two has a type B where one of the method needs to return a type of crate_one::A. You can't easily. One of the trick is to declare A as a C++ type which requires implementing the trait cxx::ExternType. But if you don't control the crate, then it won't work as you can't implement a trait outside of either the crate that declares it or that declare the type you are implementing the trait for.

This led to the other problem: the code uses gtk-rs and the bindings needs to deal with gtk types. The solution I found is to declare extern C++ types that are name like the gtk type and in the Rust code to cast the pointer to match the underlying _sys types. For example a *mut ffi::GtkWidget as *mut gtk4_sys::GtkWidget. The compiler can't help validating the type, but at least it's localized in the bindings, and it's not worse than before.

Moving forward

And in parallel, as this was the goal, I ported large chunk of the UI code to Rust. Removing C++ code is so satisfying.

So moving forward, this lay the ground work to move more code to Rust as to not write new C++ code.

Lightroom Import

This was a big feature branch I finally merged almost a year later. With that previous change the merge was bit paintful. In various order, this provide a few new features that are not specific to the importer:

  • catalog update (when the database format changes)
  • albums, with a limited feature set
  • import of Lightroom™ 4 and 6 catalogs
  • provide some infrastructure to import others (I have some plans)

This is currently very experimental as there are possibly a lot of uncovered issues, but having it in the main branch is really the first step.

Album support

With Lightroom™ import come the need to support albums, ie collection of images. So now you can create and delete albums, and the Ligthroom™ importer will import them.

The main user interaction for album is drag and drop. Drag an image onto an album to add it. Drag and drop isn't implemented. Also renaming them isn't implemented either.

Preparing for drag and drop support

Before implementing drag and drop between list widgets (the grid view or thumbnail strip and the workspace), we'd better stop using the now deprecated widgets. This mean it is time to remove GtkIconView and GktTreeView as I would hate having to implement drag and drop twice.

Now, the grid view / thumbnail strip no longer uses GtkIconView. There are still a few visual adjustments to perform, notably how to get square cells, or rethink the layout. This also got rid of the last leftovers from cbindgen.

The workspace (the treeview that list the content of the catalog) is a bit more tricky as the API for tree view with GtkListView has a few limitations and complexities, but it's mostly done as to have the functionality. It's the last big patch in November.

On the left, before the change. On the right, after the change.

GtkTreeView

These expander triangles are what needs fixing and are caused by limitation in Gtk4: Gtk.TreeListRow:expandable is immutable, and the expander is always visible if the property is true.

Adwaita

The single most visible change is the addition to libadwaita support that by itself change the way the UI look by adding just one line of code.

Before:

Before Adwaita

After:

After Adwaita

It landed before the workspace change. The only other code change needed was the handling for the dark/light them preference.

Current short term plans

There is a long list of stuff I want to do for an initial release, which is a decade overdue ;-), and which is unlikely to happen soon.

Upcoming things are, beside finishing album support, porting the build system to meson (patch in progress), finish the file import (and port it to Rust), finish the library import with possible more formats, with actual performance improvements as import is still very slow. Notably the USB Mass Storage use libgphoto2 instead of copying directly, and library import hitting the sqlite database quite hard.

There is also the raw processing question. Currently I use libopenraw for thumbnailing, and GEGL built-in processing for rendering (it uses LibRaw). Nothing that I consider functional, just the plain minimum to view an image. I have a few ideas involving "importing" a lot of other third-party code (C/C++), but I want to experiment first with it, as there is a large network of dependencies.

Thank you for reading.

November 30, 2022

Making unphishable 2FA phishable

One of the huge benefits of WebAuthn is that it makes traditional phishing attacks impossible. An attacker sends you a link to a site that looks legitimate but isn't, and you type in your credentials. With SMS or TOTP-based 2FA, you type in your second factor as well, and the attacker now has both your credentials and a legitimate (if time-limited) second factor token to log in with. WebAuthn prevents this by verifying that the site it's sending the secret to is the one that issued it in the first place - visit an attacker-controlled site and said attacker may get your username and password, but they won't be able to obtain a valid WebAuthn response.

But what if there was a mechanism for an attacker to direct a user to a legitimate login page, resulting in a happy WebAuthn flow, and obtain valid credentials for that user anyway? This seems like the lead-in to someone saying "The Aristocrats", but unfortunately it's (a) real, (b) RFC-defined, and (c) implemented in a whole bunch of places that handle sensitive credentials. The villain of this piece is RFC 8628, and while it exists for good reasons it can be used in a whole bunch of ways that have unfortunate security consequences.

What is the RFC 8628-defined Device Authorization Grant, and why does it exist? Imagine a device that you don't want to type a password into - either it has no input devices at all (eg, some IoT thing) or it's awkward to type a complicated password (eg, a TV with an on-screen keyboard). You want that device to be able to access resources on behalf of a user, so you want to ensure that that user authenticates the device. RFC 8628 describes an approach where the device requests the credentials, and then presents a code to the user (either on screen or over Bluetooth or something), and starts polling an endpoint for a result. The user visits a URL and types in that code (or is given a URL that has the code pre-populated) and is then guided through a standard auth process. The key distinction is that if the user authenticates correctly, the issued credentials are passed back to the device rather than the user - on successful auth, the endpoint the device is polling will return an oauth token.

But what happens if it's not a device that requests the credentials, but an attacker? What if said attacker obfuscates the URL in some way and tricks a user into clicking it? The user will be presented with their legitimate ID provider login screen, and if they're using a WebAuthn token for second factor it'll work correctly (because it's genuinely talking to the real ID provider!). The user will then typically be prompted to approve the request, but in every example I've seen the language used here is very generic and doesn't describe what's going on or ask the user. AWS simply says "An application or device requested authorization using your AWS sign-in" and has a big "Allow" button, giving the user no indication at all that hitting "Allow" may give a third party their credentials.

This isn't novel! Christoph Tafani-Dereeper has an excellent writeup on this topic from last year, which builds on Nestori Syynimaa's earlier work. But whenever I've talked about this, people seem surprised at the consequences. WebAuthn is supposed to protect against phishing attacks, but this approach subverts that protection by presenting the user with a legitimate login page and then handing their credentials to someone else.

RFC 8628 actually recognises this vector and presents a set of mitigations. Unfortunately nobody actually seems to implement these, and most of the mitigations are based around the idea that this flow will only be used for physical devices. Sadly, AWS uses this for initial authentication for the aws-cli tool, so there's no device in that scenario. Another mitigation is that there's a relatively short window where the code is valid, and so sending a link via email is likely to result in it expiring before the user clicks it. An attacker could avoid this by directing the user to a domain under their control that triggers the flow and then redirects the user to the login page, ensuring that the code is only generated after the user has clicked the link.

Can this be avoided? The best way to do so is to ensure that you don't support this token issuance flow anywhere, or if you do then ensure that any tokens issued that way are extremely narrowly scoped. Unfortunately if you're an AWS user, that's probably not viable - this flow is required for the cli tool to perform SSO login, and users are going to end up with broadly scoped tokens as a result. The logs are also not terribly useful.

The infuriating thing is that this isn't necessary for CLI tooling. The reason this approach is taken is that you need a way to get the token to a local process even if the user is doing authentication in a browser. This can be avoided by having the process listen on localhost, and then have the login flow redirect to localhost (including the token) on successful completion. In this scenario the attacker can't get access to the token without having access to the user's machine, and if they have that they probably have access to the token anyway.

There's no real moral here other than "Security is hard". Sorry.

comment count unavailable comments

November 29, 2022

Going inside Cairo to add color management

Before going further you might want to read the previous blog post. Also, this:

I don't really have prior experience with color management, Cairo internals or the like. I did not even look at the existing patchsets for this. They are fairly old so they might have bitrotted and debugging that is not particularly fun. This is more of a "the fun is in the doing" kind of thing. What follows is just a description of things tried, I don't know if any of it would be feasible for real world use.

Basically this is an experiment. Sometimes experiments fail. That is totally fine.

Main goals

There are two things that I personally care about: creating fully color managed PDFs (in grayscale and CMYK) and making the image backend support images in colorspaces other than sRGB (or, more specifically, "uncalibrated RGB which most of the time is sRGB but sometimes isn't"). The first of these two is simpler as you don't need to actually do any graphics manipulations, just specify and serialize the color data out to the PDF file. Rendering it is the PDF viewer's job. So that's what we are going to focus on.

Color specification

Colors in Cairo are specified with the following struct:

struct _cairo_color {
    double red;
    double green;
    double blue;
    double alpha;

    unsigned short red_short;
    unsigned short green_short;
    unsigned short blue_short;
    unsigned short alpha_short;
};

As you can probably tell it is tied very tightly to the fact that internally everything works with (uncalibrated) RGB, the latter four elements are used for premultiplied alpha computations. It also has a depressingly amusing comment above it:

Fortunately this struct is never exposed to the end users. If it were it could never be changed without breaking backwards compatibility. Somehow we need to change this so that it supports other color models. This struct is used a lot throughout the code base so changing it has the potential to break many things. The minimally invasive change I could come up with was the following:

struct _comac_color {
    comac_colorspace_t colorspace;
    union {
        struct _comac_rgb_color rgb;
        struct _comac_gray_color gray;
        struct _comac_cmyk_color cmyk;
    } c;
};

The definition of rbg color is the original color struct. With this change every usage of this struct inside the code base becomes a compile error which is is exactly what we want. At every point the code is changed so that it first asserts that colorspace is RGB and then accesses the rgb part of the union. In theory this change should be a no-op and unless someone has done memcpy/memset magic, it is also a no-op in practice. After some debugging (surprisingly little, in fact) this change seemed to work just fine.

Color conversion

Ideally you'd want to use LittleCMS but making it a hard dependency seems a bit suspicious. There are also use cases where people would like to use other color management engines and even select between them at run time. So a callback functions it is:

typedef void (*comac_color_convert_cb) (comac_colorspace_t,
                                        const double *,
                                        comac_colorspace_t,
                                        double *,
                                        comac_rendering_intent_t,
                                        void *);

This only converts a single color element. A better version would probably need to take a count so it could do batch operations. I had to put this in the surface struct, since they are standalone objects that can be manipulated without a drawing context.

Generating a CMYK PDF stream is actually fairly simple for solid colors. There are only two color setting operators, one for stroking and one for nonstroking color (there may be others in e.g. gradient batch definitions, I did not do an exhaustive search). That code needs to be changed to convert the color to match the format of the output PDF and then serialize it out.

CMYK PDF output

With these changes creating simple CMYK PDF files becomes fairly straightforward. All you need to do as the end user is to specify color management details on surface creation:

comac_pdf_surface_create2 (
    "cmyktest.pdf",
    COMAC_COLORSPACE_CMYK,
    COMAC_RENDERING_INTENT_RELATIVE_COLORIMETRIC,
    comac_default_color_convert_func,
    NULL,
    595.28,
    841.89);

and then enjoy the CMYK goodness:

What next?

Probably handling images with ICC color profiles.

November 28, 2022

are ephemerons primitive?

Good evening :) A quick note, tonight: I've long thought that ephemerons are primitive and can't be implemented with mark functions and/or finalizers, but today I think I have a counterexample.

For context, one of the goals of the GC implementation I have been working on on is to replace Guile's current use of the Boehm-Demers-Weiser (BDW) conservative collector. Of course, changing a garbage collector for a production language runtime is risky, and for Guile one of the mitigation strategies for this work is that the new collector is behind an abstract API whose implementation can be chosen at compile-time, without requiring changes to user code. That way we can first switch to BDW-implementing-the-new-GC-API, then switch the implementation behind that API to something else.

Abstracting GC is a tricky problem to get right, and I thank the MMTk project for showing that this is possible -- you have user-facing APIs that need to be implemented by concrete collectors, but also extension points so that the user can provide some compile-time configuration too, for example to provide field-tracing visitors that take into account how a user wants to lay out objects.

Anyway. As we discussed last time, ephemerons are usually have explicit support from the GC, so we need an ephemeron abstraction as part of the abstract GC API. The question is, can BDW-GC provide an implementation of this API?

I think the answer is "yes, but it's very gnarly and will kill performance so bad that you won't want to do it."

the contenders

Consider that the primitives that you get with BDW-GC are custom mark functions, run on objects when they are found to be live by the mark workers; disappearing links, a kind of weak reference; and finalizers, which receive the object being finalized, can allocate, and indeed can resurrect the object.

BDW-GC's finalizers are a powerful primitive, but not one that is useful for implementing the "conjunction" aspect of ephemerons, as they cannot constrain the marker's idea of graph connectivity: a finalizer can only prolong the life of an object subgraph, not cut it short. So let's put finalizers aside.

Weak references have a tantalizingly close kind of conjunction property: if the weak reference itself is alive, and the referent is also otherwise reachable, then the weak reference can be dereferenced. However this primitive only involves the two objects E and K; there's no way to then condition traceability of a third object V to E and K.

We are left with mark functions. These are an extraordinarily powerful interface in BDW-GC, but somewhat expensive also: not inlined, and going against the grain of what BDW-GC is really about (heaps in which the majority of all references are conservative). But, OK. They way they work is, your program allocates a number of GC "kinds", and associates mark functions with those kinds. Then when you allocate objects, you use those kinds. BDW-GC will call your mark functions when tracing an object of those kinds.

Let's assume firstly that you have a kind for ephemerons; then when you go to mark an ephemeron E, you mark the value V only if the key K has been marked. Problem solved, right? Only halfway: you also have to handle the case in which E is marked first, then K. So you publish E to a global hash table, and... well. You would mark V when you mark a K for which there is a published E. But, for that you need a hook into marking V, and V can be any object...

So now we assume additionally that all objects are allocated with user-provided custom mark functions, and that all mark functions check if the marked object is in the published table of pending ephemerons, and if so marks values. This is essentially what a proper ephemeron implementation would do, though there are some optimizations one can do to avoid checking the table for each object before the mark stack runs empty for the first time. In this case, yes you can do it! Additionally if you register disappearing links for the K field in each E, you can know if an ephemeron E was marked dead in a previous collection. Add a pre-mark hook (something BDW-GC provides) to clear the pending ephemeron table, and you are in business.

yes, but no

So, it is possible to implement ephemerons with just custom mark functions. I wouldn't want to do it, though: missing the mostly-avoid-pending-ephemeron-check optimization would be devastating, and really what you want is support in the GC implementation. I think that for the BDW-GC implementation in whippet I'll just implement weak-key associations, in which the value is always marked strongly unless the key was dead on a previous collection, using disappearing links on the key field. That way a (possibly indirect) reference from a value V to a key K can indeed keep K alive, but oh well: it's a conservative approximation of what should happen, and not worse than what Guile has currently.

Good night and happy hacking!

November 26, 2022

Crowdfunding Elgato Key Lights

Since I created Boatswain, earlier this year, a lot has happened. Recently it was accepted as part of the GNOME Circle! As the app gets more popular, people are asking for more useful features that I cannot implement without having access to the actual hardware they depend on.

Specifically, I’ve received multiple requests to integrate Boatswain with Elgato Key Lights. This makes a lot of sense, and I’m happy to do so, but without hardware for testing the changes it’s not feasible. Of course, this is free software, someone with sufficient programming skills could contribute that feature; but as it turns out, the intersection of people using Linux, GNOME, Boatswain, Elgato Key Lights, and knows how to program in C + GObject is minuscule, so that’s unlikely to happen.

Thus I started a mini crowdfunding on my Ko-Fi page to raise funds for one Elgato Key Light, one Elgato Key Light Air, and a handful of work hours. This is the first time I explicitly account for development time in a Ko-Fi goal, so I’m not sure how well this is going to be received, nor how well this will go.

Achieving these goals and actually integrating Elgato Key Lights with Boatswain will be yet another step towards making Linux – and GNOME in particular – a viable platform for streaming.

Actually, I think we have come a long way already; after adding Wayland support for OBS Studio, improving screencasting on GNOME, allowing screencast sessions to be restored by applications, enabling Elgato Stream Deck devices, and of course reaping the benefits of these improvements myself over the past 3 years of streaming on my channel, I feel like we’re actually concretely approaching the point where streaming is simple and accessible for everybody on Linux with GNOME and Flatpak. If you appreciate these improvements, consider sponsoring a coffee!

Buy Me a Coffee at ko-fi.com

What Linux distributions to recommend to computer scientists

Here is my recommendation about what kind of Linux distribution to choose for computer scientists wanting to use Linux professionally or for learning purposes, on their desktops. (Not specifically for contributing to free software).

My suggestion is to go for:

  • one of the mainstream distributions;
  • and one that has great documentation to learn a lot about system administration.

To have great and enough documentation, a distribution like Fedora or a non-LTS Ubuntu version is not what I recommend. They are released every 6 months approximately, so their system administration guide is often outdated or lacks a lot of information. These versions of distributions are more appropriate for contributors (including testers, etc).

So here is my list, in alphabetical order:

I especially recommend an enterprise distribution simply because the documentation is more complete. An "LTS" version of a distribution also has the benefit to be more stable (less bugs).

Under the documentation angle, you may give a different choice than simply comparing the graphical differences (for the installer, the default desktop and how to install packages, mainly).

Footnote: for the story, I'm now using openSUSE and I'm quite happy with the SLE / SLED documentation. In the past I've also used CentOS for certain installations, and the RHEL docs were helpful. Although I've used Debian and Ubuntu during several years each, I haven't read much of their documentation, but I suppose it's good too.

Even though it's not possible to write comments on this blog, don't hesitate to drop me an email. I do read them, and like to have feedbacks and to exchange ideas.

gedit crowdfunding

A short message to announce that gedit is again accepting donations!

More details on Liberapay.

Many thanks for your support.

Even though it's not possible to write comments on this blog, don't hesitate to drop me an email. I do read them, and like to have feedbacks and to exchange ideas.

November 25, 2022

#71 Increased Circle

Update on what happened across the GNOME project in the week from November 18 to November 25.

Circle Apps and Libraries

Boatswain

A guiding hand when navigating through streams.

Sophie announces

This week, Boatswain joined GNOME Circle. Boatswain allows you to control Elgato Stream Deck devices. Congratulations!

Third Party Projects

martinszeltins announces

Introducting a new game written in C, GTK4, Blueprint and Libadwaita - Who Wants To Be a Millionaire. Test your knowledge to see if you can answer all questions and become a virtual millionaire! This is the very first release and I am open to contributions to make this game even better if anyone wants to contribute.

The game has landed on Flathub - https://flathub.org/apps/details/lv.martinsz.millionaire

Emmanuele Bassi says

If you like the Perl programming language, and you wish to use to write your GNOME applications, you can now do so by using these two modules:

They are currently pretty bare bones, but the plan is to improve them with additional, more idiomatic overrides for the C API, before being published on CPAN. Help is welcome!

Tagger

An easy-to-use music tag (metadata) editor.

Nick reports

Tagger V2022.11.2 is here! This is a small release that fixes some issues users were facing. Here’s the full changelog:

  • Tagger will now properly set an album art’s mime type to be properly displayed in some music players
  • Changed ‘Delete Tags’ shortcut to Shift+Delete to allow for Delete button to work in entry widgets
  • Added Croatian translation (Thanks @milotype!)

Money

A personal finance manager.

Nick reports

Money V2022.11.1 is here! This release features a brand new look for Money! This new redesign features a new way of organizing groups and transactions, allowing for a quick overview of all transactions and an easy way to filter your account view. The TransactionDialog was also redesigned to make creating and editing transactions easier. Besides the big redesign, we also added the ability to transfer money from one account to another with ease. Here’s the full changelog:

  • We completely redesigned the application to provide an easier and more efficient way to manage your accounts, groups, and transactions
  • Added the ‘Transfer Money’ action to allow for transferring money to another account file
  • Added support for filtering transactions by type, group, or date
  • You can now double-click a .nmoney file and it will open directly in Money
  • The CSV delimiter has been changed to a semicolon (;)
  • Fixed an issue where some monetary values were displayed incorrectly
  • Fixed an issue where repeated transactions would not assign themselves to a group

Loupe

A simple and modern image viewer.

Sophie reports

Loupe gained support for zooming and scrolling in images via many different input types, including touchpad and touchscreen gestures. Combined with some cleanups and added keyboard shortcuts, Loupe now provides the basic features of an image viewer.

Gradience

Change the look of Adwaita, with ease.

Daudix UFO reports

Gradience 0.3.2 is out! This version fixes some major issues and introduces some under-the-hood improvements, as well as some new features, some of them are:

  • Issues with the Firefox GNOME theme plugin under Flatpak are fixed
  • CSS now loads correctly after applying a preset
  • Fixed an issue with presets always being saved as User.json
  • Presets are now removed correctly
  • The internal structure was refactored
  • Various typos were fixed
  • The README was fully rewritten
  • All screenshots are now in high resolution
  • New and updated translations

That’s all for this week!

See you next week, and be sure to stop by #thisweek:gnome.org with updates on your own projects!

How to install Nix on Fedora Silverblue

There is a lot to like about Fedora Silverblue. Updates are atomic and if there is something wrong with the newest version, you can always roll back. You always move between immutable images of your operating system, but that also means that installing packages with dnf doesn't work anymore. For GUI applications, the answers to this are flatpak and its app store flathub. For everything else you can enter a mutable Fedora container with the help of toolbx. There, the dnf command is readily available and can be used as usual. This is convenient for development, but not necessarily outside it. Whenever you want to use an installed CLI tool, you now have to enter the toolbx beforehand. Also, there are a couple of system directories that are inaccessible from within the container.

Nix

Nix is a cross-platform package manager with the largest repository at the time of this writing. Like with Silverblue, upgrades in Nix are atomic and can be rolled back. The only problem is that Nix expects to be able to store its data at /nix, which cannot simply be created on Silverblue.

Mount /nix

What we can do however, is to have the nix store at a different directory and then mount this directory at /nix. First, we add a systemd service unit which ensures that the directory /nix is present. For that, the service has to temporarily disable the immutability of / with chattr. Run the following command to create or modify the service. Also, make sure to replace YOUR_USER with your actual username.

$ sudo systemctl edit --full --force ensure-nix-dir.service
[Unit]
Description=Ensure /nix is present
[Service]
Type=oneshot
ExecStartPre=chattr -i /
ExecStart=mkdir -p -m 0755 /nix
ExecStart=chown -R YOUR_USER
EcecStop=chattr +i /

Now we create a mount unit which mounts /nix from ~/.nix during start up. Again, replace YOUR_USER with your username.

$ sudo systemctl edit --full --force nix.mount
[Unit]
Description=Mount /nix from ~/.nix
After=local-fs.target var-home.mount ensure-nix-dir.service
Wants=ensure-nix-dir.service
[Mount]
Options=bind,nofail
What=/home/YOUR_USER/.nix
Where=/nix
[Install]
WantedBy=multi-user.target

In order to immediately activate the mount, execute the following:

$ sudo systemctl enable --now nix.mount

If you now check up on /nix, it should be owned by your user and have the following permission bits:

$ ls -ld /nix
drwxr-xr-x. 1 YOUR_USER root 16 22. Nov 18:08 /nix/

Install Nix

Next, we perform a single-user installation of Nix as described in the Nix manual.

$ sh <(curl -L https://nixos.org/nix/install) --no-daemon

In order to have the corresponding tools in your $PATH, either restart your shell or source nix.sh as shown below.

$ source $HOME/.nix-profile/etc/profile.d/nix.sh

Now you can install tools on your machine with nix-env. If that's all you want, then you can skip over to the last section.

Home Manager

Home Manager is a tool that allows declarative configuration of user specific packages and dotfiles. This not only helps you to keep track which changes you made to your user environment, but also greatly reduces the amount of work necessary to migrate to a new machine. Convinced? Then let's get started!

First, we add the home-manager channel to our nix channels.

$ nix-channel --add https://github.com/nix-community/home-manager/archive/master.tar.gz home-manager

Then we update our nix-channels.

$ nix-channel --update

Home Manager requires NIX_PATH to be set before we install it, so let's export it.

$ export NIX_PATH=$HOME/.nix-defexpr/channels:/nix/var/nix/profiles/per-user/root/channels${NIX_PATH:+:$NIX_PATH}

Now we can install home-manager with nix-shell.

$ nix-shell '<home-manager>' -A install

Log off and log in again. After opening your terminal, home-manager should be in your $PATH. You can edit its config file by executing:

$ home-manager edit

This should open a file with a couple of values already set. In the following snippet you see:

{ config, pkgs, ... }:

{
  # Leave `home.username`, `home.homeDirectory`, `home.stateVersion`
  # and `programs.home-manager.enable` as they are 
  
  # Set git config 
  programs.git = {
    enable = true;
    userName  = "Julian Hofer";
    userEmail = "julianhofer@gnome.org";
    extraConfig = {
      init = {
        defaultBranch = "main";
      };
      pull = {
        rebase = true;
      };
      rebase = {
        autostash = true;
      };
    };
  };
  
  # Ensure that the following packages are installed
  home.packages = with pkgs; [
    bat
    fd
    gh
    glab
    ripgrep
  ];  
}

Now we activate our config file by executing

$ home-manager switch

This also means that if you remove a setting or package in the config file, it will be removed on your system as well.

In order to update your environment run:

$ nix-channel --update
...
unpacking channels...
$ home-manager switch

Toolbx

Most of the things you can do with a toolbx you can also do with Nix, but there is a steep learning curve. At least at the beginning you will want to be able to access the config and packages managed by Home Manager inside your toolbx.

First enter your toolbx.

$ toolbox enter

Then create a symlink from ~/.nix to /nix.

$ sudo ln -s ~/.nix /nix

Leave toolbox and enter again. To see if everything is working as expected, you can try to view your git config with the cat-replacement bat:

$ bat ~/.config/git/config

The content of git config on a terminal as displayed by bat with syntax highlighting

You can find the discussion at this Mastodon post.

References

November 24, 2022

Concurrency, Parallelism, I/O Scheduling, Thread Pooling, and Work-Stealing

Around 15 years ago I worked on some interesting pieces of software which are unfortunately still not part of my daily toolbox in GNOME and GTK programming. At the time, the world was going through changes to how we would write thread pools, particularly with regards to wait-free programming and thread-pooling.

New trends like work-stealing were becoming a big thing, multiple-CPUs with multiple NUMA nodes were emerging on easy to acquire computers. We all were learning that CPU frequency was going to stall and that non-heterogeneous CPUs were going to be the “Next Big Thing”.

To handle those changes gracefully, we were told that we need to write software differently. Intel pushed that forward with Threading Building Blocks (TBB). Python had been doing things with Twisted which had an ergonomic API and of course “Stackless Python” and similar was a thing. Apple eventually came out with Grand Central Dispatch. Microsoft Research had the Concurrency and Coordination Runtime (CCR) which I think came out of robotics work.

Meanwhile, we had GThreadPool which honestly hasn’t changed that much since. Eventually the _async/_finish paring we’re familiar with today emerged followed by GTask to provide a more ergonomic API on top of it.

A bit before the GTask we all know today, I had written libgtask which was more of a Python Twisted-style API which provided “deferreds” and nice ways to combine them together. That didn’t come across into GLib unfortunately. To further the pain there, what we got in the form of GTask has some serious side-effects which make it unsuitable as a general construct in my humble opinion.

After realizing libgtask was eclipsed by GLib itself, I set off on another attempt in the form of libiris. That took a different approach that tried to merge the mechanics of CCR (message passing, message ports, coordination arbiters, actors), the API ergonomics of Python Twisted’s Deferred, and Apple’s NSOperation. It provided a wait-free work-stealing scheduler to boot. But it shared a major drawback of GLib’s GTask (beyond correctness bugs that plague it today). Primarily that thread pools can only process the work queue and therefore if you need to combine poll() or GSource that attach to a GMainContext you’re going to require code-flow to repeatedly bounce between threads.

This is because you can simplify a thread pool worker to while (has_work()) do_work ();. Any GSource or I/O either needs to bounce to the main thread where the applications GMainContext exists or to another I/O worker thread if doing synchronous I/O. On Linux, for a very long time, synchronous I/O was the “best” option if you wanted to actually use the page cache provided by the kernel, so that’s largely what GLib and GIO does.

The reason we couldn’t do something else is that to remove an item from the global work queue required acquiring a GMutex and blocking until an item is available. On Linux at least, we didn’t have APIs to be able to wait on a Futex while also poll()ing a set of file-descriptors. (As a note I should mention for posterity that FD_FUTEX was a thing for a short while, but never really usable).

In the coming years, we got a lot of new features in Linux that allowed improvements to the stack. We got signalfd to be able to poll() on incoming Unix signals. We got eventfd() which allowed a rather low-overhead way to notify coordinating code with a poll()able file-descriptor. Then EFD_SEMAPHORE was added so that we can implement sem_t behavior with a file-descriptor. It even supports O_NONBLOCK.

The EFD_SEMAPHORE case is so interesting to me because it is provides the ability to do something similar to what IRIX did 20+ years ago, which is a pollable semaphore! Look for usnewpollsema() if you’re interested.

There was even some support in GLib to support epoll(), but that seems to have stalled out. And honestly, making it use io_uring might be smarter option now.

After finishing the GTK 4 port of GNOME Builder I realized how much code I’ve been writing in the GAsyncReadyCallback style. I don’t particularly love it and I can’t muster the energy to write more code in that style. I feel like I’m writing top-half/bottom-half interrupt handlers yet I lack the precision to pin things to a thread as well as having to be very delicate with ownership to tightly control object finalization. That last part is so bad we basically don’t use GLib’s GTask in Builder in favor of IdeTask which is smart about when and where to release object references to guarantee finalization on a particular thread.

One thing that all these previous projects and many hundreds of thousands of async-oriented C code written has taught me is that all these components are interlinked. Trying to solve one of them without the others is restrictive.

That brings me to 2022, where I’m foolishly writing another C library that solves this for the ecosystem I care most about, GTK. It’s goal is to provide the pieces I need in both applications as well as toolkit authoring. For example, if I were writing another renderer for GTK, this time I’d probably built it on something like this. Given the requirements, that means that some restrictions exist.

  • I know that I need GMainContext to work on thread pool threads if I have any hope of intermixing workloads I care about on worker threads.
  • I 100% don’t care about solving distributed computing workloads or HTTP socket servers. I refuse to race for requests-per-second at the cost of API ergonomics or power usage.
  • I know that I want work stealing between worker threads and that it should be wait-free to avoid lock contention.
  • Worker threads should try to pin similar work to their own thread to avoid bouncing between NUMA nodes. This increases data cacheline hits as well as reduces chances of allocations and frees moving between threads (something you want minimized).
  • I know that if I have 32 thread pool threads and 4 jobs are added to the global queue, I don’t want 32 threads waking up from poll() to try to take that those work items.
  • The API needs to be simple, composable, and obvious. There is certainly a lot of inspiration to be taken from things like std::future, JavaScript’s Promise, Python’s work on deferred execution.
  • GObject has a lot of features and because of that it goes through great lengths to provide correctness. That comes at great costs for things you want to feel like a new “primative”, so avoiding it makes sense. We can still use GTypeInstance though, following in the footsteps of GStreamer and GTK’s Scene Kit.
  • Cancellation is critical and not only should it cause the work you created to cancel, that cancellation should propagate to the things your work depended on unless non-cancelled work also depends on it.

I’ve built most of the base of this already. The primary API you interact with is DexFuture and there are many types of futures. You have futures for IO (using io_uring). Futures for unix signals. A polled semaphore where dex_semaphore_wait() is a future. It can wrap _async/_finish pairs and provide the result as a Future. Thread pools are efficient in waiting on work (so staying dormant until necessary and minimal thread wake-ups) while also coordinating to complete work items.

There is still a lot of work to go, and so far I’ve only focused on the abstractions and Linux implementations. But I feel like there is promise (no pun intended) and I’m hoping to port swaths of code in Builder to this in the coming months. If you’d like to help, I’d be happy to have you, especially if you’d like to focus on alternate DexAioBackends, DexSemaphore using something other than eventfd() on BSD/Solaris/macOS/Windows, and additional future types. Additionally, working to GLib to support GMainContext directly using io_uring would be appreciated.

You can find the code here, but it will likely change in the near future.

November 22, 2022

Recovering a truncated Zoom meeting recording on Endless OS

One of my colleagues was recording a Zoom meeting. The session ended in such a way that the recording was left unfinished, named video2013876469.mp4.tmp. Trying to play it in Videos just gave the error “This file is invalid and cannot be played”. ffplay gave the more helpful error “moov atom not found”. It wasn’t too hard to recover the file, but it did involve a bit of podman knowledge, so I’m writing it up here for posterity.

A bit of DuckDuckGo-ing tells us that the moov atom is, roughly, the index of the video. Evidently Zoom writes it at the end of the file when the recording is stopped cleanly; so if that doesn’t happen, this metadata is missing and the file cannot be played. A little more DuckDuckGo-ing tells us that untrunc can be used to recover such a truncated recording, given an untruncated recording from the same source. Basically it uses the metadata from the complete recording to infer what the metadata should be for the truncated recording, and fill it in.

I built untrunc with podman as follows:

git clone https://github.com/ponchio/untrunc.git
cd untrunc
podman build -t untrunc .

I made a fresh directory in my home directory and placed the truncated video and a successful Zoom recording into it:

mkdir ~/tmp
# Make the directory world-writable (see later note)
chmod 777 ~/tmp
cp ~/Downloads/video2013876469.mp4.tmp ~/tmp
cp ~/Downloads/zoom_0.mp4 ~/tmp

Now I ran the untrunc container, mounting $HOME/tmp from my host system to /files in the container, and passing paths relative to that:

podman run --rm -it -v $HOME/tmp:files \
    localhost/untrunc:latest \
    /files/zoom_0.mp4 \
    /files/video2013876469.mp4.tmp

and off it went:

Reading: /files/zoom_0.mp4
Repair: /files/video2013876469.mp4.tmp
Mdat not found!

Trying a different approach to locate mdat start
Repair: /files/video2013876469.mp4.tmp
Backtracked enough!

Trying a different approach to locate mdat start
Repair: /files/video2013876469.mp4.tmp
Mdat not found!

[…]

Found 71359 packets.
Found 39648 chunks for mp4a
Found 31711 chunks for avc1
Saving to: /files/video2013876469.mp4_fixed.mp4

The fixed file is now at ~/tmp/video2013876469.mp4_fixed.mp4.

Tinkering with the directory permissions was needed because untrunc’s Dockerfile creates a user within the container and runs the entrypoint as that user, rather than running as root. I believe in the Docker world this is considered good practice because root in a Docker container is root on the host, too. However I’m using podman as an unprivileged user; so UID 0 in the container is my user on the host; and the untrunc user in the container ends up mapped to UID 166535 on the host, which doesn’t have write access to the directory I mounted into the container. Honestly I don’t know how you’re supposed to manage this, so I just made the directory world-writable and moved on with my life.

November 21, 2022

2022-11-21 Monday

  • Mail chew, catch up with Mike, planning call. Poked at some bugs and did a bit of hacking in passing. B&A out for a nice lunch in Cambridge with H,N,M. Caught up with them in the evening.

November 20, 2022

2022-11-20 Sunday

  • All Saints in the morning, played with Cedric, Robert spoke, back for lunch with Bruce & Anne over to stay, chatted with them in the afternoon. Watched an SAS video later.

November 19, 2022

Status update, 19/11/2022

Audio Developer Conference

I was at ADC 2022 last week – thanks to Codethink as always for covering the cost and allowing me 2 days time off to attend. It was my first time attending in person, and besides the amazing talks (which will appear online here around the end of this month), I had somehow never realized how many players in the music tech world are British. Perhaps because I always hang out in Manchester and further north while all the activity is happening in Cambridge and London.

Indeed the creator of the famous JUCE Framework is a Brit and was busy at the conference announcing a new(ish) language designed for DSP pipelines and plugins, cleverly named Cmajor.

ADC has no lightning talks but instead an “Open mic night”. I naively pictured a concert in a pub somewhere and signed up to play, but in practice it was closer to lightning talks and there were no instruments to play. For better or worse I had the session for Maharajah on a laptop and did an improvised 5 minute version with judicious (or not) or use of timestretching to speed through the slow parts. Even more fun than regular lightning talks.

I highly recommend the event! Despite the surfeit of Mac users!

Microblogging

Every article published during November 2022 must mention the Twitter meltdown and this is no exception! I’ve personally enjoyed using reading Twitter for years, always without Infinite Scroll and with “Latest tweets” instead of “Home” (i.e. content from people I follow, not entertaining clickbait for my monkey-brain). The sites only value was always its enourmous userbase and now there’s a real wave of migration to Mastodon, like, celebrities and all, I’m spending more time as @vladimirchicken@mastodon.art.

I made that account a while back, initially just for half-baked song ideas, and I will try to keep it more playful and positive than my old Twitter account. For work-related content I already have a very serious blog, after all. (That’s this blog!)

Nushell

At work I’m still spending every day deep in JSON and CSV data dumps and using Nushell to make sense of it all.

It’s really good for exploring data, and you can immediately make a guess at what this Nushell pipeline might do:

fetch http://date.jsontest.com/ | get date

I find the syntax a lot easier than jq, where even basic filters quickly start to look like K or Brainfuck.

I wrote my first Nushell scripts, which is is also very nice, shell functions (“custom commands”) can have type annotations for arguments, default values and each script can have a main function where you define the commandline arguments. 400% nicer than Bash scripts.

I don’t have time to get involved enough in the project right now to open useful bug reports, so I’m going to post a list of gripes here instead. Hopefully its useful for other folk evaluating Nushell so you can get an idea of what’s still missing as of version 0.70:

  • Writing a long line (one that wraps to the next line) makes cursor movement increasingly slow… at least on WSL
  • where can compare a column to a value, e.g. where date < 11-19-2020, but you can’t compare two columns, which is weird, SQL where can do that.
  • The CTRL+C behaviour sucks. Run 100 slow python scripts in a loop, and you have to ctrl+c out of every single one.

November 18, 2022

Boatswain 0.2.2 is out!

Last week, I’ve rolled out the 0.2.1 release of Boatswain, the Elgato Stream Deck application for Linux.

After the massive push to get the initial releases out, the development pace naturally slowed down, but I managed to pick up some new fancy features for this release. The most visible new feature is dragging button actions around:

It’s not super sophisticated, and you can’t create folders with it yet, but it’s a handy first step. Boatswain now shows hundreds of symbolic icons in the icon picker menu:

Little details here and there were polished. Light background colors now force symbolic icons to have a dark appearance. Icon sizing was improved. Thanks to Jonathan Kang, Boatswain now supports Elgato Stream Deck XL gen2 devices.

Overall, it was a good release. Smaller, focused releases make me happy. Hope they make you happy too 🙂

You can grab the latest release of Boatswain on Flathub!

Buy Me a Coffee at ko-fi.com

November 17, 2022

[GNOME] GNOME Files and custom file icons: setting a cute 2x2 image preview for photo albums

Update: credit to David in the comments of my previous post, there's a much fancier implementation of this here: https://github.com/flozz/cover-thumbnailer


Going further in my delight at belatedly discovering the "metadata::custom-icon" GVFS attribute used by Nautilus, I extended beyond just music album covers to write a script that did a fun 2x2 grid for photo album covers.

From this:

To this:


This script is uglier, so maybe write your own, but you can have a look-see to see what I did. I learned some new syntax for ImageMagick in the process (using ^ when scaling to treat the geometry as a minimum rather than a maximum, using "-gravity center" to crop from the center, etc.),
#!/bin/sh

# Run in a directory containing photo album folders.  Will generate 2x2 grid thumbnail images of photos within, and set them as the
# folder icon.  (Recurses down through all subfolders.)

# set GVFS attribute "metadata::custom-icon"
# e.g. gio set /path/to/album/dir "metadata::custom-icon" "file:///home/USER/files/music/ARTIST NAME/ALBUM NAME/cover.jpg"
# to reset it:
#      gio set /path/to/album/dir "metadata::custom-icon" -d

function crop {
    FILE="${1}";
    DIM="${2}";  # e.g. "200x200"

    # scale so that the min-dimension is 200 (we'll crop out whatever exceeds it in the max dimension)
    # so if we're given an 400x1000 photo here, we'll scale it to 200x500, thanks to ^
    convert "${FILE}" -scale "${DIM}+0+0^" "${FILE}.scaled";

    # crop a 200x200 square from the center, so if we have a 200x500, we'll lose the top and bottom 50
    convert "${FILE}.scaled" -gravity center -crop "${DIM}+0+0" "${FILE}.cropped"
    # now we have a nice 200x200 version, centered

    # emit the name of the newly cropped file
    echo "${FILE}.cropped";
}

# provide a '-f' option to regenerate existing photo album covers; by default, it skips dirs where one already exists
if [ "${1}" == -f ]; then
    FORCE=1
else
    FORCE=0
fi


find ./ -type d |
    while read DIR; do
        # skip . hidden directories
        if basename "${DIR}" | grep "^\."; then
            continue;
        fi

        echo "${DIR}...";
        
        # check if a generated photo album already exists and skip redoing it (unless -f set)
        if [ -f "${DIR}/.npac_cover.jpg" ] && [ "${FORCE}" -eq 0 ]; then
            echo "    Already generated, skipping (use -f to force regeneration)";
            continue;
        fi

        # create a temporary directory to do image cropping and stitching in
        TMPDIR="$(mktemp -d)";

        # find 4 images below dir for potential thumbnailing; use 'sort -R' to randomize the list
        find "${DIR}" -regextype egrep -iregex ".*\.(jpg|jpeg|png|webp)$" |
            sort -R |
            while read COVER_PART; do
                # because I sometimes have duplicate files for various reasons, and 
                # I don't want an image repeated, I work with them using their hash
                HASH="$(md5sum "${COVER_PART}" | cut -b 1-32)";

                if ! [ -f "${TMPDIR}/${HASH}" ]; then
                    echo "    ${COVER_PART}";
                    cp "${COVER_PART}" "${TMPDIR}/${HASH}.part"
                fi

                if [ "$(ls "${TMPDIR}" | wc -l)" -ge 4 ]; then
                    break;
                fi
            done;

        # lets create a 2x2 grid of 4 images, but also support fewer if necessary
        (
            cd "${TMPDIR}";
            NUM_PARTS="$(ls | wc -l)";
            P1="$(ls | head -n 1 | tail -n 1)";
            P2="$(ls | head -n 2 | tail -n 1)";
            P3="$(ls | head -n 3 | tail -n 1)";
            P4="$(ls | head -n 4 | tail -n 1)";

            if [ "${NUM_PARTS}" -ge 3 ]; then
                # stitch the first two together side-by-side for the top half of a 3 or 4 grid
                convert "$(crop "${P1}" 200x200)" "$(crop "${P2}" 200x200)" -background black +append "top.jpg"

                if [ "${NUM_PARTS}" -ge 4 ]; then
                    # stitch two together for the bottom, then stitch top and bottom together for the grid, yay
                    convert "$(crop "${P3}" 200x200)" "$(crop "${P4}" 200x200)" -background black +append "bottom.jpg"
                    convert "top.jpg" "bottom.jpg" -background black -append "grid.jpg"
                elif [ "${NUM_PARTS}" -eq 3 ]; then
                    # only have 3 images, so make the 3rd one take up the whole bottom
                    convert "top.jpg" "$(crop "${P3}" 400x200)" -background black -append "grid.jpg"
                fi
            elif [ "${NUM_PARTS}" -eq 2 ]; then
                # only have two, so stack them vertically
                convert "$(crop "${P1}" 400x200)" "$(crop "${P2}" 400x200)" -background black -append "grid.jpg"
            elif [ "${NUM_PARTS}" -eq 1 ]; then
                # only have one, so it'll be the whole grid!
                cp "$(crop "${P1}" 400x400)" grid.jpg
            else
                echo "WARNING: nautilus_photo_album_cover.sh: no images found for dir '${DIR}' in tmpdir '${TMPDIR}'.";
            fi
        )

        # if we successfully generated a grid of images, copy it to the photo album dir as a 
        # hidden thumbnail image (.npac_cover.jpg), set "metadata::custom-icon" in GVFS, and then remove the temp dir
        if [ -f "${TMPDIR}/grid.jpg" ]; then
            cp "${TMPDIR}/grid.jpg" "${DIR}/.npac_cover.jpg"
            gio set "${DIR}" metadata::custom-icon "file://$(realpath "${DIR}/.npac_cover.jpg")";
            rm -rf "${TMPDIR}"
        fi
    done
                

To do:

  • Add a cute folder emblem to the bottom right of the 2x2 icons, and maybe put a border around it, to make it more obviously still a folder!
  • It fails on some folders for some reason, look into it :D
  • Correct orientation issues

[GNOME] GNOME Files and custom file icons: setting album art on directories!

Update: credit to David in the comments, there's a much fancier implementation of this here: https://github.com/flozz/cover-thumbnailer


While playing with GNOME 43 on my recent upgrade to Fedora 37, I saw that nautilus aka GNOME Files lets me set arbitrary images as a custom icon for files and folders, replacing the default icon/thumbnail.  (The support for "metadata::custom-icon" has apparently been around for a while already.)

I quickly hopped into my music folder and wrote a small script to replace all of the (legally purchased) folders of albums of music's icons with images of their album art if an image was already present.

 My Camera Obscura band's folder of albums went from this:

to this

Cool!

 


The quick code:
#!/bin/sh

find ./ -type d |
    while read DIR; do
        if basename "${DIR}" | grep "^\."; then
            # skip . directories
            continue;
        fi

        echo "${DIR}...";
        
        touch /tmp/.naa.coversearch

        # ranked cover options by preference, as some dirs have multiple; 
        # break after we find highest one
        echo "front.(jpg|png)
cover.(jpg|png)
[Ff]older.(jpg|png)
box.jpg
album\-[0-9a-f]+\-[0-9a-f]+\.jpeg" |
            while read COVER_PATTERN; do
                # echo "  ${COVER_PATTERN}...";

                find "${DIR}" -regextype egrep -regex ".*/${COVER_PATTERN}$" |
                    head -n 1 | # grab first and go            
                    while read COVER_RELPATH; do
                        rm /tmp/.naa.coversearch # we found a cover, end search

                        echo "    found cover ${COVER_RELPATH}!";
                        
                        COVER_ABSPATH="$(realpath "${COVER_RELPATH}")";
                        gio set "${DIR}" metadata::custom-icon "file://${COVER_ABSPATH}"
                    done

                # if we're not still searching, skip other patterns and go to next dir
                test -f /tmp/.naa.coversearch || break 
            done
    done

For the curious, I purchased a lot of legally-licensed, DRM-free music through eMusic back in the day, before streaming took over everything. I still purchase CDs as merch at shows sometimes, and then extract those to OGG Vorbis (sorry, not FLAC). Sometimes, I still open up Rhythmbox and play actual files, ooo.

To do:

  • corner folder icons: Generate album folder thumbnails that have a little folder icon in the bottom corner still, so I can tell that it's still a folder
  • artist custom icons as potential grids: My folder structure is artist/album/track.ogg. So right now, the artist folder also acquires the first album cover it finds. If an artist has multiple album covers available, I want to do a small grid of up to 2x2.
  • photo albums: do something similar for photo albums folders.

November 16, 2022

Get ready for Google Summer of Code 2023 with GNOME!

Google has recently announced the 2023 edition of Google Summer of Code.

The 2022 changes in the program’s format have been successful and are continuing for 2023, with just a small adjustment around eligibility (described in the link above).

GNOME is certainly going to apply to be a mentoring organization, and we hope to once again be part of the program.

If you are a new contributor interested in a summer sponsorship to work in the GNOME project, this is a great time to start preparing!

Visit our Newcomers tutorial to learn how to make your first contribution, join our communication channels to discuss ideas with the community, and stay tuned to our gsoc.gnome.org website!

If you have any doubts/questions, feel free to open a topic in our Discourse tag.

November 10, 2022

Running Shell in Builder

Builder has been absolutely wonderful for a designer to dive in and fix up graphics assets for Application. It allows to easily build and test run patches before submitting a merge/pull request on apps hosted on gitlab or github. Ideally you’d press the run button and voilá.

What has been far from wonderful — doing even one line fixes for the GNOME Shell was very hard to test for anyone not building shell daily. getting the environment ready every release has been a chore. From virtual machines, jhbuild, toolbox, jhbuild in VMs to jhbuild in toolbox there was a dozen of way to fail building the latest shell.

Toolbox/Podman targets in Builder

Builder 43, impressively marathoned into perfection, features (among many other things) the ability to target Toolbox/podman containers as development environments. So I was really happy to see Bylan McCall have a nested session going on the run button.

Now I couldn’t get the full detailed receipe out of him, but as they say it’s easier to get the internet prove you wrong than to give you advice, here’s my terrible way (with some help from the most authoritative capacities that probably don’t want anything to do with this pkexec hack):

1) Create a toolbx container and install all gnome-shell dependencies.

toolbox create shell
toolbox enter shell
sudo dnf install gnome-shell mutter ninja-build meson
sudo dnf builddep gnome-shell mutter

While at the beginning of a cycle packages cover most of the dependencies, at the later stages you’ll have to build more things (Usually gsettings-desktop-schemas, libgweather, gjs, gcr and mutter). We’re in a container so I just oldschool brute install to --prefix=/usr. To build mutter for example I’d

mkdir ~/Projects
cd ~/Projects
git clone https://gitlab.gnome.org/GNOME/mutter.git
cd mutter
meson --prefix=/usr build '-Dman=false'
ninja -C build
sudo ninja -C build install

So assuming we have successfuly built shell in the shell toolbox container, we’ll move over to Builder.

2) In Builder, open the project we cloned to ~/Projects/gnome-shell. Create a new configuration (Alt+,) by duplicating the Default. Change the runtime to shell (the toolbx containers are below the usual flatpak runtimes). Set the Installation Prefix to /usr. Don’t forget to make it active (Make Active).

3) Still in configuration, in the Commands section, create a new command (call it Run Nested Shell):

dbus-run-session gnome-shell --wayland --nested

At the bottom of the dialog add a Variable:

MUTTER_DEBUG_DUMMY_MODE_SPECS=1920x1080

This will make the shell run at least in HD, rather than the super tiny default size.

4) In the Application section, change the Run Command from Automatically Discover to our Run Nested Shell. Now we only need to do the last nasty bit to allow for the install to /usr to succeed.

5) Open a new runtime terminal (Ctrl+Alt+T) and replace pkexec with a just-do-it script. It’s a container, it’s fiiiiine, chill.

cd /usr/bin
mv pkexec pkexec.orig

Now with your preferred CLI editor create the following shell script in place of pkexec:

#!/bin/bash
su -c "$*"

Set it executable with chmod +x pkexec.

Build Configurations

Now pressing the run button should build your shell, install it and run the nested session. Hopefully stylesheet patches will be less painful now, maybe?

Please do not hesistate to toot at me what the proper way is rather than installing to /usr

Wrapping things up | GSoC22@Pitivi

Overview

GSoC 2022 is nearing its end and now it’s time to wrap things up. My task was to Improve the Timeline component of Pitivi and this is my status report.

Work Report

Fix a dot appearing when clicking on the timeline - !436

Pitivi uses a custom widget for showing the selected area while dragging & selecting. This Marquee rectangle gets drawn when the user performs a drag action. We only moved this rectangle to a new position when the user starts dragging. So when there occurs a mouse click, the Marquee box get’s drawn at the previous drag_end position which appears as a dot.

It was fixed by adding the code to move the marquee to the callback of the event when a user first clicks. Now when clicking, the box gets drawn under the mouse.

More precise audio waveforms - !444

Pitivi now shows 5x more detailed waveforms for audio! image These are some snaps of before and after (top & bottom respectively).

Bring back the auto aligner ⭐ - !445

If I had to pick a favorite, this is definitely it. Developing AutoAligner was just pure bliss. To be honest, this was the scariest task on my to-do list because I thought it would be a very math-intensive & complex feature to implement.

Pitivi had this cool feature some time before but it got lost when the method of waveform generation and its caching changed.

image

This feature allows users to align audio clips recorded from different sources. Here is the basic idea behind how it currently works:

Consider the two waves given below.

  • To calculate how well these waves match, we add the products of values from each wave at a given time. This final sum can be used to measure the cross-correlation between these two waves. image
  • Then the second wave is shifted with respect to the first one so that the pair of values from each wave at a given time is different than before and the calculation is repeated. image As we can see the correlation value is highest when the peaks of waves are most similar.

These videos helped a lot in understanding the basics:

This is the basic idea behind the alignment. The actual calculations are handled by SciPy which makes the alignment process very fast ⚡.

Allow separate selection of ungrouped clips - !448

We use GES.Layer.get_clips_in_interval to get the clips in a layer within an interval. The x coordinates of drag-selection are converted to time units(start & end) and they are passed to the above-mentioned function to get the clips that are to be selected.

The problem with this is: although Audio & Video clips reside in the same layer, they exist in different halves of the layer. Video clips are drawn in the upper half and Audio clips are in the lower half of the layer. So even when the user meant to just select the video clip by dragging over it, the function also returns audio clips, if any, overlapping with the same interval.

This was fixed by calculating the position of the selection-rectangle edges relative to the layer and deciding which half does the selection area starts/ends. Then we filter the clips returned from the get_clips_in_interval and return only the to-be-selected clips.

Migrate to GStreamer’s new mono-repo - !451

As of 28 September 2021, Gstreamer modules have been merged into a single git repository to simplify development workflows and CI. We are working on porting to monorepo and updating the GNOME SDK used in flatpak.

Others

Update colors when the theme is changed - !427

Fix ticks in ruler disappearing at some intervals - !437

Fix ctrl+click clears the whole selection - !440

Fix waveforms not scaling when changing clip speed - !443

Modify how transitions are represented - !450

List of all merge requests

Rest of the issues

My GSoC proposal had some more issues mentioned mostly relating to dragging events. These were unable to complete as Pitivi is being ported to Gtk4.0 parallelly. The common source of the issues was that the drag events won’t get registered if the mouse pointer leaves the source element. In Gtk4 these events get delivered until the mouse is released.

So I started trying to help Aryan with porting stuff and currently, I am working on finding a replacement for gtksink which works in gtk4.

Finalizing

Debugging issues in such a large codebase (relative to what I have worked on till now) was very hard, but the joy and excitement you get when you find the source and fix those bugs makes them totally worth it. After completing the WIP issues, I would like to continue contributing and help in finishing the gtk4 port.

Finally my mentors - Alexandru Băluț & Fabián Orccón, I cannot thank them enough for their support and guidance. They made a safe and friendly environment for me to ask for help without any resistance. When I get stuck on something they helped a lot to guide me to the solution. Once again, Thank you so much!

That’s it! I’m concluding here with a smile on my face :)

See you soon 👋

November 07, 2022

Crosswords 0.3.6: “The Man from G.N.O.M.E”

Note: a half-written version of this blog post was accidentally published earlier, and was briefly picked up by planet.gnome.org. The [Publish] and [Preview] buttons are unfortunately right next to each other 🙁

Release

It’s time for another GNOME Crosswords release! This one is particularly exciting to me as it brings a lot of new content. The major highlights include new puzzles, a new distribution channel, and the start of adaptive behavior with an eye to working on mobile. It’s available for download in flathub.

New Puzzles

First — and most important — is the addition of new puzzles. We added three more puzzle sets from different sources. I wrote to a number of crossword creators in the local FOSS community and asked if I could include their puzzles in this release. Thanks to Bob, Carl, and dew.mountain for kindly agreeing to add their puzzles to the distribution.

Next, Rosanna and I sat down and together wrote some quick mini-puzzles. These were pretty  fun to create, so hopefully we can build a good collection over time. Let us know if you’re interested in helping us.

Finally, I wrote to Māyā of Auckland to ask if they minded if I included their puzzles. Māyā writes cryptic crosswords that are of a very high quality — and that are a lot of fun to solve. They very kindly agreed to let me distribute them, so I’ve included ~90 cryptics.

This adds up to over 100 new puzzles available in this version.

Adding these puzzles took a surprising amount of time. They ended up highlighting a half-dozen or so bugs in the code base, and lead to substantial rewriting of a number of sections. In detail:

  • One of Bob’s puzzles came in puz format and used circled letters, so I added circle support to the puz importer. This is used fairly commonly. Now that we support it, I’m noticing circles a lot more in the wild.
  • Some of Māyā’s puzzles were made available in JPZ format. As a result, I finally wrote a convertor for this format. I also refactored the convertor to make it possible to add other formats in the future.
    NOTE: I did not have a lot of jpz puzzles to test while writing this. If you have some, please try this feature out and let me know if it doesn’t load correctly.
  • Māyā’s puzzles also thoroughly broke the enumeration code. The old regex-based parser wasn’t good enough to handle them, so I wrote a state-machine-based parser to handle this properly. There were really cute enumerations in those puzzles like:

    After first first exchange, 10 relates to blood vessels (7-)
    Solo star of this? Yes and no (3,3,4,1.1.1.1.1)that couldn’t be parsed with the old approach.
  • These puzzles also forced me to get much more strict about the text handling. The ipuz spec allows a subset of html in certain tags, and you certainly run into puzzles with this in the wild. Complicating matters, you also come across random ampersands (&) that weren’t escaped, as well as other random entities.I wrote a simple html_to_markup() function that parses html (as best as I can) and converts it to valid GMarkup tags to pass to labels. This is mostly a matter of handling entities and escaping unknown tags, but there were definitely some subtleties involved. If someone has a better version of this functionality, let me know. On the other hand, maybe this implementation is helpful to someone else out there.

Thanks again to these puzzle authors!

Adaptive layout

The next big feature we added is making Crosswords work well at different screen sizes. I had a couple requests to make this work on mobile devices and Sam did a great design, so we tried to make this plausible. Unfortunately, there’s not a lot of formal guidance as to what’s needed here, other than to support small screens. I’d love if the GNOME Mobile folks would put together a checklist of things to support, as well as instructions as how best to test / simulate that environment without having dedicated hardware.

Nevertheless, we now support smaller screens! Here’s a video of it in action.

To make this work proved to be pretty tricky, as that packing is impossible to do with a standard GtkBox layout. The challenge is that that scrolled window holding the grid needs to have vexpand=TRUE set, but also have a maximum size so that the clue at the bottom doesn’t extend beyond the grid. I finally gave up and wrote a custom widget to do this sizing:

Note: I got a lot closer to a working solution with constraints, and was really impressed with how well they worked. Emmanuele has done a fantastic job with them. There’s a bit of a learning curve to get started, but they’re definitely worth a second look if you haven’t tried using them before. They’re more powerful than regular box-based solutions, and ended up being easier to maintain and experiment with.

For the sizing behavior to feel right, I also wrote code to restore the window size on restart. Fortunately, there’s a great example of how to do this in the tutorial. Unfortunately, the example doesn’t work correctly. After figuring out why, I wrote an MR to fix the documentation. Unfortunately I don’t know vala well enough to update that example, maybe someone who does can help me update it?

Availability

One more fantastic contribution arrived  towards the end of the release cycle: Davide and Michel packaged Crosswords for Fedora. I had never actually tried to install the game locally, but meson magically worked and they got it built. Now, running ‘sudo dnf install crosswords‘ works with f36/f37. As it’s Fedora, I may have to spend some time next cycle to straighten out all the licenses across the code base and puzzles, but that’s time worth spending.

One spooky Halloween-themed bug came with this work: word-list-tests test failure on s390x. That’s a category of bug I never expected to have to deal with ever again in my life!

In general, I’m really excited about the future of flatpak and what it means to the larger Linux ecosystem. I will continue to focus my efforts on that. But having it on Fedora is nice validation, and hopefully will lead to more people solving crosswords. I’d love it if other distros follow suit and ship this as well.

Odds and Ends

Other fixes this cycle include:

  • Massive cleanup to event code (Federico). This was huge, and moves us to a simpler event system. Really appreciated!
  • Also, Federico let me call him a couple Saturdays to get general design advice
  • New setting to avoid changing crossword direction when arrowing through the grid (Jonathan and Rosanna)
  • Mime types defined for ipuz, jpz, and puz files (Davide)
  • Cleanup of the color and contrast of shapes in cells (Jonathan)
  • New Italian translation (Davide)
  • MacOS fixes and testing (Vinson)
  • Loads of bug fixes (all)

Until next time!

November 05, 2022

Second request for Linux backlight testing for changes planned for 6.2

As mentioned in my previous blog post, I have written a new patch series for 6.2 to try to avoid having multiple entries in /sys/class/backlight for a single panel again.

This new series might cause regressions on a different set of even older laptop models then the one affected by the 6.1 backlight work. So I'm again looking for people willing to run a few quick tests.

To see if your laptop is possibly affected by the 6.2 change please run:

  • ls /sys/class/backlight


if the output of this command contains both:

  1. A GPU native backlight device, with intel/nv/nvidia/amd/radeon/psb/oaktrail in the name; and

  2. A vendor backlight device, with either a model-series: eeepc, ideapad, thinkpad, etc; or a vendor-name: acer, asus, dell, toshiba, sony, etc. in the name


then 6.2 will cause a behavior change on your device, it will hide the vendor backlight device in preference of the native backlight device.

If your laptop shows only a native or only a vendor backlight device (possibly in combination with another type of backlight device such as acpi_video#), then your laptop will not be affected by the planned changes for 6.2.

Note that I expect only very old models to be affected, e.g. the Sony Vaio PCG-FRV3
from 2003 is known to be affected.

If your laptop has both native + vendor backlight devices, then please do 2 things:


  1. Please run the following commands:

    1. ls /sys/class/backlight > ls-backlight.txt

    2. sudo dmesg > dmesg.txt

    3. sudo dmidecode > dmidecode.txt

    4. sudo acpidump -o acpidump.txt


  2. Please test if the native backlight interface works, the example below assumes the native backlight is called "intel_backlight":

    1. cd /sys/class/backlight/intel_backlight

    2. cat max_brightness

    3. <the "cat max_brightness" will show the maximum brightness value supported>

    4. echo $max_brightness_value > brightness

    5. echo $half-of-max_brightness_value > brightness


Where if for example cat max_brightness returns 255 then $max_brightness_value
is 255 and $half-of-max_brightness_value is 127. And then check if the brightness of the backlight actually changes when you do this ?


After generating the 4 .txt files and running the native backlight tests, please send me an email about this at hdegoede@redhat.com with the results of the native backlight tests (panel brightness changes when echo-ing values to brightness or not?) and with the 4 generated .txt files attached.

If the native backlight interface works then things should keep working fine with 6.2 and typically you will get more fine-grained brightness control as an added bonus. Please send me an email with the test results even if the native backlight interface works.

Results of requested backlight testing for 6.1

I have received quite a few test reports in response to my previous blog post. Many thanks to everyone who has run the tests and send me their results!

These tests show that as a result of the current 6.1 changes quite a few laptop models will end up with an empty "/sys/class/backlight", breaking users ability to control their laptop panel's brightness.

I have submitted a patch-set for 6.1 upstream to fix this.

More detailed summary/analysis of the received test reports:

  • Known Windows laptop models affected by this:

    • Acer Aspire 1640

    • HP Compaq nc6120

    • IBM ThinkPad X40

    • System76 Starling Star1


  • Known MacBook models affected by this:

    • Apple MacBook 2.1

    • Apple MacBook 4.1

    • Apple MacBook Pro 7.1


  • 30 unaffected models

  • The Dell Inspiron N4010 has acpi_video support and acpi_osi_is_win8() returns false, so acpi_video_get_backlight_type() returns acpi_video, but acpi_video fails to register a backlight device due to a _BCM eval error. This model already has a DMI quirk for this, so it is unaffected.

  • The following laptop models use vendor backlight control method, while also having a native backlight entry under /sys/class/backlight:

    • Asus EeePC 901 (native backlight confirmed to work better then vendor)

    • Dell Latitude D610 (native backlight confirmed to also work)

    • Sony Vaio PCG-FRV3 (native backlight control does not work, BIOS from 2003!)



The fixes for 6.1 restore the behavior where userspace can see multiple entries under "/sys/class/backlight" for a single panel and the kernel leaves figuring out which one actually works up to userspace. This is undesirable and having more then 1 backlight device for a single panel also blocks the new backlight userspace API work which I have planned.

This first round of testing has shown that native works well even on systems so old that they don't have acpi_video backlight control support.

So I have prepared a patch series to try again with 6.2 by making native be preferred over vendor, which should avoid the problems seen with the 6.1 changes before the fixes.

November 04, 2022

Stop Using QtWebKit

Today, WebKit in Linux operating systems is much more secure than it used to be. The problems that I previously discussed in this old, formerly-popular blog post are nowadays a thing of the past. Most major Linux operating systems now update WebKitGTK and WPE WebKit on a regular basis to ensure known vulnerabilities are fixed. (Not all Linux operating systems include WPE WebKit. It’s basically WebKitGTK without the dependency on GTK, and is the best choice if you want to use WebKit on embedded devices.) All major operating systems have removed older, insecure versions of WebKitGTK (“WebKit 1”) that were previously a major security problem for Linux users. And today WebKitGTK and WPE WebKit both provide a webkit_web_context_set_sandbox_enabled() API which, if enabled, employs Linux namespaces to prevent a compromised web content process from accessing your personal data, similar to Flatpak’s sandbox. (If you are a developer and your application does not already enable the sandbox, you should fix that!)

Unfortunately, QtWebKit has not benefited from these improvements. QtWebKit was removed from the upstream WebKit codebase back in 2013. Its current status in Fedora is, unfortunately, representative of other major Linux operating systems. Fedora currently contains two versions of QtWebKit:

  • The qtwebkit package contains upstream QtWebKit 2.3.4 from 2014. I believe this is used by Qt 4 applications. For avoidance of doubt, you should not use applications that depend on a web engine that has not been updated in eight years.
  • The newer qt5-qtwebkit contains Konstantin Tokarev’s fork of QtWebKit, which is de facto the new upstream and without a doubt the best version of QtWebKit available currently. Although it has received occasional updates, most recently 5.212.0-alpha4 from March 2020, it’s still based on WebKitGTK 2.12 from 2016, and the release notes bluntly state that it’s not very safe to use. Looking at WebKitGTK security advisories beginning with WSA-2016-0006, I manually counted 507 CVEs that have been fixed in WebKitGTK 2.14.0 or newer.

These CVEs are mostly (but not exclusively) remote code execution vulnerabilities. Many of those CVEs no doubt correspond to bugs that were introduced more recently than 2.12, but the exact number is not important: what’s important is that it’s a lot, far too many for backporting security fixes to be practical. Since qt5-qtwebkit is two years newer than qtwebkit, the qtwebkit package is no doubt in even worse shape. And because QtWebKit does not have any web process sandbox, any remote code execution is game over: an attacker that exploits QtWebKit gains full access to your user account on your computer, and can steal or destroy all your files, read all your passwords out of your password manager, and do anything else that your user account can do with your computer. In contrast, with WebKitGTK or WPE WebKit’s web process sandbox enabled, attackers only get access to content that’s mounted within the sandbox, which is a much more limited environment without access to your home directory or session bus.

In short, it’s long past time for Linux operating systems to remove QtWebKit and everything that depends on it. Do not feed untrusted data into QtWebKit. Don’t give it any HTML that you didn’t write yourself, and certainly don’t give it anything that contains injected data. Uninstall it and whatever applications depend on it.

Update: I forgot to mention what to do if you are a developer and your application still uses QtWebKit. You should ensure it uses the most recent release of QtWebEngine for Qt 6. Do not use old versions of Qt 6, and do not use QtWebEngine for Qt 5.

How to install Fedora on an HP X2 Chromebook

We have been working lately with Enric Balletbo and Dorinda Bassey to improve the support for the HP X2 Chromebook in Fedora. This post explains how to install Fedora on that Chromebook.

The article ended being up being longer than I thought, so for the impatient this is the summary:

  • Switch the Chromebook to Developer Mode.

  • Boot from the internal disk.

  • Go to a virtual terminal with Ctrl+Alt+F2 and login as root.

  • Enable developer mode boot from external disk (USB/microSD):

      $ crossystem dev_boot_usb=1
    
  • Install packages needed by the chromebook-setup.sh script:

      $ sudo dnf install bc curl util-linux gdisk lz4 e2fsprogs \
        uboot-tools udisks2 vboot-utils guestfs-tools
    
  • Clone Enric’s chromebooks scripts repo:

      $ git clone https://github.com/eballetbo/chromebooks.git
      $ pushd chromebooks
    
  • Flash a Fedora image to a storage media (replace /dev/sda with your block device):

      $ sudo ./chromebook-setup.sh deploy_fedora \
        --architecture=arm64 --storage=/dev/sda \
        --kparams="clk_ignore_unused deferred_probe_timeout=30"
    
  • Plug the USB/microSD device into the Chromebook and choose to boot from an external device.

  • After the Fedora initial setup, install the following packages:

      $ sudo dnf install uboot-tools vboot-utils lz4 -y
    
  • Remove packages that expects the grub2 bootloader to be used:

      $ sudo dnf remove grubby kexec-tools -y
    
  • Enjoy Fedora on the Chromebook 🙂

Now the longer version…

Challenges supporting the Chromebooks in Fedora

Supporting the Chromebooks is not trivial, because these laptops use a different firmware (Coreboot) and boot stack (Depthcharge) than what is used by all the other aarch64 machines supported by Fedora (UEFI and GRUB). There are good reasons why that is the case, but it poses some challenges and complicates making these laptops to work in Fedora out-of-the-box.

For this reason, a standard Fedora ISO image can’t just be booted on a Chromebook to start an installation process.

Current approach used to install Fedora on the Chromebooks

To overcome this, Enric wrote a set of chromebook scripts that can be used to setup a block device (i.e: USB drive or microSD card) and write a Fedora image that can be booted directly. The script also adds to the system a kernel-install plugin, written by Dorinda, that takes the Linux kernel image being installed and package it in the format (FIT) expected by the Chromebook bootloader.

That way, the Fedora installation would look and behave exactly the same than in any other system.

In the future support for Chromebooks might be added to the Anaconda OS installer used by Fedora, but in the meantime using this script allows us to do experimentation and make further customization that wouldn’t be suitable for a general system installer.

Following are the instructions to install Fedora on a HP X2 Chromebook using the chromebook-setup.sh script.

Switch the Chromebook to Developer Mode

In the default mode the Chromebook can only boot binaries that are trusted by its firmware. This means that nothing besides ChromeOS can be installed. To boot a different OS, the mode should be change to Developer. In this mode, any binary that is signed with the Google’s developer key can be booted. That key is available for anyone so is what is used to sign the Linux images when generating the FIT images during the Fedora kernel packages installations.

The ChromiumOS project has excellent articles explaining the Developer Mode and how to enable it on Chromebooks without a physical keyboard, such as the HP X2.

After enabling developer mode, boot from the internal disk and switch to a virtual terminal with Ctrl+Alt+F2. Then login as root and execute the following to enable booting from an external disk (USB/microSD):

$ crossystem dev_boot_usb=1

Flashing a Fedora image

The chromebook-setup.sh script can be used to flash fedora images to a block device and do all the needed setup to make it bootable.

It supports many different options but it also has reasonable defaults. The list of options can be listed with ./chromebook-setup --help.

Following are the steps to flash a Fedora image.

Install packages needed by the chromebook-setup.sh script:

$ sudo dnf install bc curl util-linux gdisk lz4 e2fsprogs \
  uboot-tools udisks2 vboot-utils guestfs-tools

Clone Enric’s chromebooks scripts repository:

$ git clone https://github.com/eballetbo/chromebooks.git
$ pushd chromebooks

Execute the script, for example:

$ sudo ./chromebook-setup.sh deploy_fedora \
  --architecture=arm64 --storage=/dev/sda \
  --kparams="clk_ignore_unused deferred_probe_timeout=30"

The deploy_fedora option will install a Fedora image in the specified storage media. By default the latest Fedora Workstation Rawhide image will downloaded and used, but a different image can be chosen using the --image=$image option.

The --architecture and --storage options specify the architecture and block device used respectively.

Finally, the --kparams option allows to set additional kernel command line parameters.

The clk_ignore_unused parameter is currently needed because there is a bug when the MSM/snapdragon DRM driver is built as a module. Some needed clocks are gated before the driver probe function is executed, causing it to fail.

And the deferred_probe_timeout=30 is needed because there are a lot of drivers probe deferrals and the default 10 seconds expires causing drivers to fail to probe due timeouts.

Hopefully these two issues would be fixed soon and the parameters won’t be needed anymore.

One the script finishes flashing the image, plug the USB drive or insert the microSD in the Chromebook and choose "Boot from external media".

The system should boot and start the Fedora initial setup program to configure the system and create a user. Once that is done, start a terminal and install the following packages needed by the kernel-install Chromebook plugin:

$ sudo dnf install uboot-tools vboot-utils lz4 -y

Remove packages that expects the grub2 bootloader to be used:

$ sudo dnf remove grubby kexec-tools -y

And that’s it. Now the Fedora should behave like in any other system. If there are any bugs, please file issues in the chromebooks scripts repository.

Happy hacking!

November 02, 2022

Linux Boot Partitions

💽 Linux Boot Partitions and How to Set Them Up 🚀

Let’s have a look how traditional Linux distributions set up /boot/ and the ESP, and how this could be improved.

How Linux distributions traditionally have been setting up their “boot” file systems has been varying to some degree, but the most common choice has been to have a separate partition mounted to /boot/. Usually the partition is formatted as a Linux file system such as ext2/ext3/ext4. The partition contains the kernel images, the initrd and various boot loader resources. Some distributions, like Debian and Ubuntu, also store ancillary files associated with the kernel here, such as kconfig or System.map. Such a traditional boot partition is only defined within the context of the distribution, and typically not immediately recognizable as such when looking just at the partition table (i.e. it uses the generic Linux partition type UUID).

With the arrival of UEFI a new partition relevant for boot appeared, the EFI System Partition (ESP). This partition is defined by the firmware environment, but typically accessed by Linux to install or update boot loaders. The choice of file system is not up to Linux, but effectively mandated by the UEFI specifications: vFAT. In theory it could be formatted as other file systems too. However, this would require the firmware to support file systems other than vFAT. This is rare and firmware specific though, as vFAT is the only file system mandated by the UEFI specification. In other words, vFAT is the only file system which is guaranteed to be universally supported.

There’s a major overlap of the type of the data typically stored in the ESP and in the traditional boot partition mentioned earlier: a variety of boot loader resources as well as kernels/initrds.

Unlike the traditional boot partition, the ESP is easily recognizable in the partition table via its GPT partition type UUID. The ESP is also a shared resource: all OSes installed on the same disk will share it and put their boot resources into them (as opposed to the traditional boot partition, of which there is one per installed Linux OS, and only that one will put resources there).

To summarize, the most common setup on typical Linux distributions is something like this:

Type Linux Mount Point File System Choice
Linux “Boot” Partition /boot/ Any Linux File System, typically ext2/ext3/ext4
ESP /boot/efi/ vFAT

As mentioned, not all distributions or local installations agree on this. For example, it’s probably worth mentioning that some distributions decided to put kernels onto the root file system of the OS itself. For this setup to work the boot loader itself [sic!] must implement a non-trivial part of the storage stack. This may have to include RAID, storage drivers, networked storage, volume management, disk encryption, and Linux file systems. Leaving aside the conceptual argument that complex storage stacks don’t belong in boot loaders there are very practical problems with this approach. Reimplementing the Linux storage stack in all its combinations is a massive amount of work. It took decades to implement what we have on Linux now, and it will take a similar amount of work to catch up in the boot loader’s reimplementation. Moreover, there’s a political complication: some Linux file system communities made clear they have no interest in supporting a second file system implementation that is not maintained as part of the Linux kernel.

What’s interesting is that the /boot/efi/ mount point is nested below the /boot/ mount point. This effectively means that to access the ESP the Boot partition must exist and be mounted first. A system with just an ESP and without a Boot partition hence doesn’t fit well into the current model. The Boot partition will also have to carry an empty “efi” directory that can be used as the inner mount point, and serves no other purpose.

Given that the traditional boot partition and the ESP may carry similar data (i.e. boot loader resources, kernels, initrds) one may wonder why they are separate concepts. Historically, this was the easiest way to make the pre-UEFI way how Linux systems were booted compatible with UEFI: conceptually, the ESP can be seen as just a minor addition to the status quo ante that way. Today, primarily two reasons remained:

  • Some distributions see a benefit in support for complex Linux file system concepts such as hardlinks, symlinks, SELinux labels/extended attributes and so on when storing boot loader resources. – I personally believe that making use of features in the boot file systems that the firmware environment cannot really make sense of is very clearly not advisable. The UEFI file system APIs know no symlinks, and what is SELinux to UEFI anyway? Moreover, putting more than the absolute minimum of simple data files into such file systems immediately raises questions about how to authenticate them comprehensively (including all fancy metadata) cryptographically on use (see below).

  • On real-life systems that ship with non-Linux OSes the ESP often comes pre-installed with a size too small to carry multiple Linux kernels and initrds. As growing the size of an existing ESP is problematic (for example, because there’s no space available immediately after the ESP, or because some low-quality firmware reacts badly to the ESP changing size) placing the kernel in a separate, secondary partition (i.e. the boot partition) circumvents these space issues.

File System Choices

We already mentioned that the ESP effectively has to be vFAT, as that is what UEFI (more or less) guarantees. The file system choice for the boot partition is not quite as restricted, but using arbitrary Linux file systems is not really an option either. The file system must be accessible by both the boot loader and the Linux OS. Hence only file systems that are available in both can be used. Note that such secondary implementations of Linux file systems in the boot environment – limited as they may be – are not typically welcomed or supported by the maintainers of the canonical file system implementation in the upstream Linux kernel. Modern file systems are notoriously complicated and delicate and simply don’t belong in boot loaders.

In a trusted boot world, the two file systems for the ESP and the /boot/ partition should be considered untrusted: any code or essential data read from them must be authenticated cryptographically before use. And even more, the file system structures themselves are also untrusted. The file system driver reading them must be careful not to be exploitable by a rogue file system image. Effectively this means a simple file system (for which a driver can be more easily validated and reviewed) is generally a better choice than a complex file system (Linux file system communities made it pretty clear that robustness against rogue file system images is outside of their scope and not what is being tested for.).

Some approaches tried to address the fact that boot partitions are untrusted territory by encrypting them via a mechanism compatible to LUKS, and adding decryption capabilities to the boot loader so it can access it. This misses the point though, as encryption does not imply authentication, and only authentication is typically desired. The boot loader and kernel code are typically Open Source anyway, and hence there’s little value in attempting to keep secret what is already public knowledge. Moreover, encryption implies the existence of an encryption key. Physically typing in the decryption key on a keyboard might still be acceptable on desktop systems with a single human user in front, but outside of that scenario unlock via TPM, PKCS#11 or network services are typically required. And even on the desktop FIDO2 unlocking is probably the future. Implementing all the technologies these unlocking mechanisms require in the boot loader is not realistic, unless the boot loader shall become a full OS on its own as it would require subsystems for FIDO2, PKCS#11, USB, Bluetooth network, smart card access, and so on.

File System Access Patterns

Note that traditionally both mentioned partitions were read-only during most parts of the boot. Only later, once the OS is up, write access was required to implement OS or boot loader updates. In today’s world things have become a bit more complicated. A modern OS might want to require some limited write access already in the boot loader, to implement boot counting/boot assessment/automatic fallback (e.g., if the same kernel fails to boot 3 times, automatically revert to older kernel), or to maintain an early storage-based random seed. This means that even though the file system is mostly read-only, we need limited write access after all.

vFAT cannot compete with modern Linux file systems such as btrfs when it comes to data safety guarantees. It’s not a journaled file system, does not use CoW or any form of checksumming. This means when used for the system boot process we need to be particularly careful when accessing it, and in particular when making changes to it (i.e., trying to keep changes local to single sectors). It is essential to use write patterns that minimize the chance of file system corruption. Checking the file system (“fsck”) before modification (and probably also reading) is important, as is ensuring the file system is put into a “clean” state as quickly as possible after each modification.

Code quality of the firmware in typical systems is known to not always be great. When relying on the file system driver included in the firmware it’s hence a good idea to limit use to operations that have a better chance to be correctly implemented. For example, when writing from the UEFI environment it might be wise to avoid any operation that requires allocation algorithms, but instead focus on access patterns that only override already written data, and do not require allocation of new space for the data.

Besides write access from the boot loader code (as described above) these file systems will require write access from the OS, to facilitate boot loader and kernel/initrd updates. These types of accesses are generally not fully random accesses (i.e., never partial file updates) but usually mean adding new files as whole, and removing old files as a whole. Existing files are typically not modified once created, though they might be replaced wholly by newer versions.

Boot Loader Updates

Note that the update cycle frequencies for boot loaders and for kernels/initrds are probably similar these days. While kernels are still vastly more complex than boot loaders, security issues are regularly found in both. In particular, as boot loaders (through “shim” and similar components) carry certificate/keyring and denylist information, which typically require frequent updates. Update cycles hence have to be expected regularly.

Boot Partition Discovery

The traditional boot partition was not recognizable by looking just at the partition table. On MBR systems it was directly referenced from the boot sector of the disk, and on EFI systems from information stored in the ESP. This is less than ideal since by losing this entrypoint information the system becomes unbootable. It’s typically a better, more robust idea to make boot partitions recognizable as such in the partition table directly. This is done for the ESP via the GPT partition type UUID. For traditional boot partitions this was not done though.

Current Situation Summary

Let’s try to summarize the above:

  • Currently, typical deployments use two distinct boot partitions, often using two distinct file system implementations

  • Firmware effectively dictates existence of the ESP, and the use of vFAT

  • In userspace view: the ESP mount is nested below the general Boot partition mount

  • Resources stored in both partitions are primarily kernel/initrd, and boot loader resources

  • The mandatory use of vFAT brings certain data safety challenges, as does quality of firmware file system driver code

  • During boot limited write access is needed, during OS runtime more comprehensive write access is needed (though still not fully random).

  • Less restricted but still limited write patterns from OS environment (only full file additions/updates/removals, during OS/boot loader updates)

  • Boot loaders should not implement complex storage stacks.

  • ESP can be auto-discovered from the partition table, traditional boot partition cannot.

  • ESP and the traditional boot partition are not protected cryptographically neither in structure nor contents. It is expected that loaded files are individually authenticated after being read.

  • The ESP is a shared resource — the traditional boot partition a resource specific to each installed Linux OS on the same disk.

How to Do it Better

Now that we have discussed many of the issues with the status quo ante, let’s see how we can do things better:

  • Two partitions for essentially the same data is a bad idea. Given they carry data very similar or identical in nature, the common case should be to have only one (but see below).

  • Two file system implementations are worse than one. Given that vFAT is more or less mandated by UEFI and the only format universally understood by all players, and thus has to be used anyway, it might as well be the only file system that is used.

  • Data safety is unnecessarily bad so far: both ESP and boot partition are continuously mounted from the OS, even though access is pretty restricted: outside of update cycles access is typically not required.

  • All partitions should be auto-discoverable/self-descriptive

  • The two partitions should not be exposed as nested mounts to userspace

To be more specific, here’s how I think a better way to set this all up would look like:

  • Whenever possible, only have one boot partition, not two. On EFI systems, make it the ESP. On non-EFI systems use an XBOOTLDR partition instead (see below). Only have both in the case where a Linux OS is installed on a system that already contains an OS with an ESP that is too small to carry sufficient kernels/initrds. When a system contains a XBOOTLDR partition put kernels/initrd on that, otherwise the ESP.

  • Instead of the vaguely defined, traditional Linux “boot” partition use the XBOOTLDR partition type as defined by the Discoverable Partitions Specification. This ensures the partition is discoverable, and can be automatically mounted by things like systemd-gpt-auto-generator. Use XBOOTLDR only if you have to, i.e., when dealing with systems that lack UEFI (and where the ESP hence has no value) or to address the mentioned size issues with the ESP. Note that unlike the traditional boot partition the XBOOTLDR partition is a shared resource, i.e., shared between multiple parallel Linux OS installations on the same disk. Because of this it is typically wise to place a per-OS directory at the top of the XBOOTLDR file system to avoid conflicts.

  • Use vFAT for both partitions, it’s the only thing universally understood among relevant firmwares and Linux. It’s simple enough to be useful for untrusted storage. Or to say this differently: writing a file system driver that is not easily vulnerable to rogue disk images is much easier for vFAT than for let’s say btrfs. – But the choice of vFAT implies some care needs to be taken to address the data safety issues it brings, see below.

  • Mount the two partitions via the “automount” logic. For example, via systemd’s automount units, with a very short idle time-out (one second or so). This improves data safety immensely, as the file systems will remain mounted (and thus possibly in a “dirty” state) only for very short periods of time, when they are actually accessed – and all that while the fact that they are not mounted continuously is mostly not noticeable for applications as the file system paths remain continuously around. Given that the backing file system (vFAT) has poor data safety properties, it is essential to shorten the access for unclean file system state as much as possible. In fact, this is what the aforementioned systemd-gpt-auto-generator logic actually does by default.

  • Whenever mounting one of the two partitions, do a file system check (fsck; in fact this is also what systemd-gpt-auto-generatordoes by default, hooked into the automount logic, to run on first access). This ensures that even if the file system is in an unclean state it is restored to be clean when needed, i.e., on first access.

  • Do not mount the two partitions nested, i.e., no more /boot/efi/. First of all, as mentioned above, it should be possible (and is desirable) to only have one of the two. Hence it is simply a bad idea to require the other as well, just to be able to mount it. More importantly though, by nesting them, automounting is complicated, as it is necessary to trigger the first automount to establish the second automount, which defeats the point of automounting them in the first place. Use the two distinct mount points /efi/ (for the ESP) and /boot/ (for XBOOTLDR) instead. You might have guessed, but that too is what systemd-gpt-auto-generator does by default.

  • When making additions or updates to ESP/XBOOTLDR from the OS make sure to create a file and write it in full, then syncfs() the whole file system, then rename to give it its final name, and syncfs() again. Similar when removing files.

  • When writing from the boot loader environment/UEFI to ESP/XBOOTLDR, do not append to files or create new files. Instead overwrite already allocated file contents (for example to maintain a random seed file) or rename already allocated files to include information in the file name (and ideally do not increase the file name in length; for example to maintain boot counters).

  • Consider adopting UKIs, which minimize the number of files that need to be updated on the ESP/XBOOTLDR during OS/kernel updates (ideally down to 1)

  • Consider adopting systemd-boot, which minimizes the number of files that need to be updated on boot loader updates (ideally down to 1)

  • Consider removing any mention of ESP/XBOOTLDR from /etc/fstab, and just let systemd-gpt-auto-generator do its thing.

  • Stop implementing file systems, complex storage, disk encryption, … in your boot loader.

Implementing things like that you gain:

  • Simplicity: only one file system implementation, typically only one partition and mount point

  • Robust auto-discovery of all partitions, no need to even configure /etc/fstab

  • Data safety guarantees as good as possible, given the circumstances

To summarize this in a table:

Type Linux Mount Point File System Choice Automount
ESP /efi/ vFAT yes
XBOOTLDR /boot/ vFAT yes

A note regarding modern boot loaders that implement the Boot Loader Specification: both partitions are explicitly listed in the specification as sources for both Type #1 and Type #2 boot menu entries. Hence, if you use such a modern boot loader (e.g. systemd-boot) these two partitions are the preferred location for boot loader resources, kernels and initrds anyway.

Addendum: You got RAID?

You might wonder, what about RAID setups and the ESP? This comes up regularly in discussions: how to set up the ESP so that (software) RAID1 (mirroring) can be done on the ESP. Long story short: I’d strongly advise against using RAID on the ESP. Firmware typically doesn’t have native RAID support, and given that firmware and boot loader can write to the file systems involved, any attempt to use software RAID on them will mean that a boot cycle might corrupt the RAID sync, and immediately requires a re-synchronization after boot. If RAID1 backing for the ESP is really necessary, the only way to implement that safely would be to implement this as a driver for UEFI – but that creates certain bootstrapping issues (i.e., where to place the driver if not the ESP, a file system the driver is supposed to be used for), and also reimplements a considerable component of the OS storage stack in firmware mode, which seems problematic.

So what to do instead? My recommendation would be to solve this via userspace tooling. If redundant disk support shall be implemented for the ESP, then create separate ESPs on all disks, and synchronize them on the file system level instead of the block level. Or in other words, the tools that install/update/manage kernels or boot loaders should be taught to maintain multiple ESPs instead of one. Copy the kernels/boot loader files to all of them, and remove them from all of them. Under the assumption that the goal of RAID is a more reliable system this should be the best way to achieve that, as it doesn’t pretend the firmware could do things it actually cannot do. Moreover it minimizes the complexity of the boot loader, shifting the syncing logic to userspace, where it’s typically easier to get right.

Addendum: Networked Boot

The discussion above focuses on booting up from a local disk. When thinking about networked boot I think two scenarios are particularly relevant:

  1. PXE-style network booting. I think in this mode of operation focus should be on directly booting a single UKI image instead of a boot loader. This sidesteps the whole issue of maintaining any boot partition at all, and simplifies the boot process greatly. In scenarios where this is not sufficient, and an interactive boot menu or other boot loader features are desired, it might be a good idea to take inspiration from the UKI concept, and build a single boot loader EFI binary (such as systemd-boot), and include the UKIs for the boot menu items and other resources inside it via PE sections. Or in other words, build a single boot loader binary that is “supercharged” and contains all auxiliary resources in its own PE sections. (Note: this does not exist, it’s an idea I intend to explore with systemd-boot). Benefit: a single file has to be downloaded via PXE/TFTP, not more. Disadvantage: unused resources are downloaded unnecessarily. Either way: in this context there is no local storage, and the ESP/XBOOTLDR discussion above is without relevance.

  2. Initrd-style network booting. In this scenario the boot loader and kernel/initrd (better: UKI) are available on a local disk. The initrd then configures the network and transitions to a network share or file system on a network block device for the root file system. In this case the discussion above applies, and in fact the ESP or XBOOTLDR partition would be the only partition available locally on disk.

And this is all I have for today.

Many thanks & good luck to Neil McGovern

As President of the GNOME Foundation, I wanted to post a quick note to pass on the thanks from the Board, the Foundation staff team and membership to our outgoing Executive Director, Neil McGovern. I had the pleasure of passing on GNOME’s thanks in person at the Casa Bariachi this summer at GUADEC in Guadelajara, at the most exellent mariachi celebration of GNOME’s 25th Anniversary. 🤠 Kindly they stopped the music and handed me the microphone for the whole place, although I think many of the other guests celebrating their own birthdays were less excited about Neil’s tenure as Executive Director and the Free and Open Source desktop in general. 🤣

Neil’s 6-month handover period came to an end last month and he handed over the reins to myself and Thibault Martin on the Executive Committee, and Director of Operations Rosanna Yuen has stepped up to act as Chief of Staff and interface between the Board and the staff team for the time being. Our recruitment is ongoing for a new Executive Director although the search is a little behind schedule (mostly down to me!), and we’re hugely grateful to a few volunteers who have joined our search committee to help us source, screen and interview applicants.

I have really enjoyed working closely with Neil in my time on the GNOME board, and we are hugely grateful for his contributions and achievements over the past 5 years which I posted about earlier in the year. Neil is this month starting a new role as the Executive Director of Ruby Central. Our very best wishes from the GNOME community and good luck with your new role. See you soon!

(also posted to Discourse if you wish to add any thanks or comments of your own)

November 01, 2022

Balance of Power: A rematch served cold

There's an old video game the memory of which recently escalated itself to my attention: Chris Crawford's Balance of Power, a geopolitics simulator first released for the Macintosh in 1985. According to Wikipedia it sold about a quarter million units, which was a lot at the time, and I must've been somewhere in the impressionable age range of 10 to 12 years old when my father bought Incredible Technologies' port for the Commodore Amiga.

Go ahead and boop the nook

Its Workbench icon featured a mushroom cloud with a hand over it in the universal that's-not-what-I-ordered gesture (though possibly gently petting it — or simply shielding the observer's eyes?), but the game itself had a minimalist, academic look, and beyond a simple dissolve effect on the title screen it featured no explosions or indeed animations of any kind. This was unusual on the Amiga, a platform known for its ability to make the rubble bounce in Technicolor.

Crawford's maze

A strange game

Although I had an idea of what was going on in the world — I'd observed the cultural touchstones at friends' birthday parties, caught glimpses of something less watchable but much more memorable, and seen the inside of a bomb shelter — I didn't have the toolbox for nuclear brinkmanship, let alone internalizing what I now recognize to be a beautiful 87-page manual with a hardcover sleeve and an extensive bibliography. In the end, for all the sneaking into my dad's office to play this in the dead of night, I couldn't really figure it out.

So. Since Halloween seems like a good occasion to indulge in a little psychological horror (no other reason), I decided to do a rematch of sorts — this time with the help of fs-uae.

Crawford sez: RTFM

Crawford released an updated game in 1989 (simply called the 1990 edition), with the Amiga port credited to Elaine Ditton. It introduced the multipolar mode, which had been left out of the 1985 release:

Unfortunately, this multipolar view of the world is just too cerebral for most game players. It has something to do with our expectations of games; a mature, otherwise sophisticated adult will sit down with this game and ask, "How do I nuke the Commies?" Games, like stories, must have conflict, but people have been so inundated with the brutal, violent conflict standard in computer games that they are unable to grasp the subtle, indirect conflict arising from a multipolar world. This was a very painful discovery, and it forced a shift in the game from a multipolar view toward a more bipolar view. Minor countries had been able to pursue their own foreign policies; now they are passive pawns. Neutralist policies were entirely feasible; now minor countries choose up sides along left-wing/right-wing lines. The result is less realistic but more suited to the needs of the game-playing audience. Perhaps someday a more sophisticated game will be possible.

BOP manual, p. 74

Ok, so it's 2022 now, and we're all sophisticated down here. We want the multipolar. Furthermore, although the game didn't anticipate certain pivotal events of 1991, we'll play as the USA just to be on the safe side.

Here in the future, we also come armed with source code courtesy of the man himself: In this case, it's written in Pascal with a tiny morsel of assembler, of which the latter mostly exists to calculate square roots. BOP doesn't really have any spoilers apart from the obvious one, and a peek here and there should be interesting.

For the RAND analyst in you

Eh, close enough

The centerpiece of the game is, logically enough, the world map. It reflects some of the assumptions and technical limitations of the time: Many countries were removed because they'd make too small click targets or were thought to be strategically redundant. For instance, my own little corner of the world has been unceremoniously rolled into Sweden, which is fine (I, for one, welcome the Kalmar Union redux).

Next up, "Finlandization" is represented in the game, but Finland is not. This seems unfair to the Finnish. Of course, there's the unfairness of the entire, uh — (gesturing vaguely at the map) — to consider. It's very much not a game about curing the world's ills. But we'll live. Probably.

So neutral

Each country has a large number of variables attached. Some are directly manipulable, contingent on the limitations of physics and diplomacy. For instance, you could send military aid to the Swedish government but not the rebels, since there aren't any in Sweden to speak of. However, you could still intervene on behalf of these "rebels" and bootstrap an insurgency on some flimsy pretext. There are also subtle ways to change the existing government's mind and eventually sign a treaty that would let you station troops with them.

The important variables, such as government stability, security, affiliation and power, are only indirectly manipulable. Some of them are hidden, and most of them are obfuscated by natural language. There's no way you can break out your calculator to predict exactly what'll happen on the next turn, no big "Democracy" button to slam for a +2 bonus to semiconductors or whatever. You're forced to read reports and think in terms of… well, not exactly the real world, but at least a facet of it.

The game plays out over eight turns representing the passage of calendar years. On each turn, countries make their policy moves and optionally dispute each other's moves, and then the simulation ticks. The last part is where the magic happens. As an example, here's how country i's desire to Finlandize to superpower j is calculated:

y:=MilPowr[i]-InsgPowr[i];
FOR j:=1 TO 2 DO
  BEGIN
    x:=InsgIMax(j,i);
    ProjPowr[j]:=(IntvConv(x)*ord4(MilPowr[j])) div MilMen[j];
    x:=Treaty[3-j,i];
    x:=(Should(x)*ord4(MilPowr[3-j])) div 128;
    SelfPowr[j]:=y+(x*ord4(Integrty[3-j])) div 128;
    IF SelfPowr[j]<1 THEN SelfPowr[j]:=1;
    temp:=((ord4(Adventur[j]-DipAff^^[j,i])*ProjPowr[j]*(Pressure[j,i]+4))
           div SelfPowr[j]);
    IF temp<0 THEN temp:=0; 
    IF temp>2048 THEN temp:=2048;
    FinlProb[j,i]:=temp div 8;
  END;

This is an interesting function with a couple of variables in play:

  • The government's military strength relative to insurgents' (MilPowr[i]-InsgPowr[i]).
  • The threatening superpower's military spending (MilPowr[j]/MilMen[j]) and ability to intervene in the area. InsgIMax(j,i) considers troops stationed in countries geographically adjacent to i.
  • The supporting superpower's military strength, treaty level and history of honoring its commitments (Integrty[3-j]).
  • The level of any diplomatic pressure campaign — harsh words, basically — aimed at the country (Pressure[j,i]). There's a small constant added to it, so the multiplier can never be zero; posting lots of soldiers next door can be quite menacing even if it's done quietly.
  • The adventurism factor (Adventur[j]), proportional to the demonstrated combativeness (Pugnacty[j]) of the threatening superpower relative to that of the other's plus the overall political tension in the world (Nastiness). This goes up if the situation seems dangerous and uncertain.
  • The diplomatic relationship between the threatening superpower and country i. This is a signed integer with 0 being neutral. Effectively, being on good terms neutralizes the threat posed by soldiers, but not that of general craziness.

Another delightful wrinkle is what happens if the superpowers put boots on the ground on opposing sides of a conflict, meaning the US and USSR are in a shooting war:

IF ((IntvGovt^^[1,i]>0) AND (IntvRebl^^[2,i]>0)) OR
   ((IntvGovt^^[2,i]>0) AND (IntvRebl^^[1,i]>0)) THEN
  BEGIN	{USA fights with USSR}
    DipAff^^[1,2]:=-127; 
    DipAff^^[2,1]:=-127;
    Nastiness:=127;
    Pugnacty[1]:=127; 
    Pugnacty[2]:=127;
  END;

BOP treats this as an extremely serious situation and cranks up the Nastiness and Pugnacty factors, causing worldwide ripple effects (conflicts may erupt or worsen, weak governments may collapse, etc).

There's a lot more to it, but you get the idea: It's an interconnected world with gradual change, tipping points and cascading effects. I think few games pull this off — or dare attempt it — because it can leave a very narrow road between the extremes of boring (too stable) and incomprehensible (too chaotic). Throw in expectations of realism, and it's a tall order.

BOP manages fine, in part due to carefully chosen initial conditions and constraints on what it deems "minor" (non-superpower) countries. Here's Canada:

InitCountry(14,'Canada',2040,228,441,-10,-125,10000,0,36,20,9,56,255,77,99,12);
  InitCont(14,1);
  MinorSph^^[14,17]:=TRUE;
  MinorSph^^[14,18]:=TRUE;
  MinorSph^^[14,20]:=TRUE;
FiniCntry(14);

Defined as being contiguous with the USA (country #1), it adds Britain, France and West Germany to its sphere of influence, so when the AI hammers out its foreign policy, it will only consider these four counterparts. Britain for its part has 15, and France is up there with 12, but Sweden has only the one on its eastern border.

Frank exchange of opinions

FUNCTION Crisis;
{Returns a value of TRUE if the missiles fly}

Of course, countries don't get to undermine and invade each other willy-nilly. There are checks on this activity, such as the local neighborhood superpower getting on the phone and politely telling them to knock it off. When this happens to a minor country, it will pull back and just quietly stew for a bit, souring diplomatic relations, but with superpowers on both ends, things get exciting: It turns into a game of chicken. The superpowers take turns escalating, starting with a zero-stakes back-channel discussion, through diplomatic and military crises, until either one of them backs down or a nuclear war starts. The prestige penalty for backing down increases as the crisis escalates.

This can be frustrating, since it's easy to misjudge the computer's commitment (though it drops hints in its diplomatic missives, e.g. when it "categorically refuses" you may be about to have a bad day). The 1990 edition added advisors, in theory making it easier to get a read on the situation.

Mr. B has an idea

You can usually sort of interpolate their various opinions, but it's harder when they disagree wildly on an issue — like the above, where the Soviets just dispatched 20,000 soldiers to topple the government of Sudan. It sounds like a mean thing to do, and — more importantly — it would strengthen Soviet influence in the region and weaken ours. I don't think they're serious about this, so let's put our chips on advisor number four, the one with the mustache. I like his attitude.

No u

Mr. Gorbachev turns a deaf ear, so we go public with a diplomatic challenge. Still no. The world is watching now, and since there's an uncomfortable amount of prestige at risk, we press the issue and threaten a military response.

NUH UH

They escalate to DEFCON 4, we go to DEFCON 3, and they respond by escalating again. That's DEFCON 2, with bombers in the air and humanity collectively holding its breath. We're left with two choices: either let rip (and end the game) or back down and suffer crushing humiliation. This is bad. The prestige hit will affect our diplomatic relations, hamstringing us and effectively cutting our lead in half. Good thing we can absorb the loss without falling hopelessly behind. Otherwise, well…

Anyway. How could advisor #4 lead us astray? The answer lies in this excerpt from the GImpt() function:

x:=DipAff^^[i,Obj] div 4;
y:=(Should(Treaty[i,Obj]) div 4)+1;
z:=(ord4(DontMess[Obj])*1280) div SumDMess;
t:=Adventur[i] div 2;

CASE Bias OF
  0: BEGIN END;
  1: BEGIN x:=x*MySqrt(Abs(x)); y:=MySqrt(y); END;
  2: BEGIN y:=y*MySqrt(y); z:=MySqrt(z); END;
  3: BEGIN z:=z*MySqrt(z); t:=MySqrt(t); END;
  4: BEGIN t:=t*MySqrt(t);
       IF x>0 THEN x:=MySqrt(x) ELSE x:=-MySqrt(Abs(x));
     END;
END;

The enum values 1-4 correspond to the different advisors. Looking at the fourth one, our guy is giving more consideration to Adventur[i] — the adventurism factor again, which frankly must've been pretty high at this point in the game — and less to our relations with Sudan. Our belligerence emboldened him, and we got some bad advice in return.

Again there's a lot more going on under the hood, and a lesson for budding diplomats:

This system produces some behaviors that may surprise you. For example, suppose that you as an American player gave economic aid to Nigeria. The Soviets take objection to this and start a crisis. You escalate, they escalate, and a nuclear war starts. The question on your lips is, why would those idiots annihilate the world over economic aid to Nigeria? The answer is, because you were willing to annihilate the world over Nigeria. Remember, it takes two to make a crisis. The computer figured that this just wasn't an important issue for you, and that, while trivial, it was still a more important issue for itself. It therefore stuck to its guns. […]

This raises a very important point about geopolitics. You could protest loudly, "But I do care about Nigeria! The computer can't assume what I'm really thinking!" You are absolutely right. Real-world diplomats don't know what's really going on in the minds of their interlocutors. They know perfectly well that today's words are only an expression of today's exigencies. The only thing they can rely on are the substantial events of the past. If you have built up a record of close relations with Nigeria, your behavior in a crisis will have to be taken seriously. If your record is of weak relations, then your behavior will not be taken seriously. The computer treats it that way. If you want to convince people that you're serious, you've got to lay the groundwork.

BOP manual, p. 77-78

Enough of that. Are ya winning, son?

Yeah, about that rematch. This isn't tic-tac-toe, so you can play it and win. Sort of.

Everyone gets a trophy

The main difficulty lies in us and humanity getting to this screen — although "kept the peace" tends to belie some dismal stuff unleashed along the way — after which we can relax and judge the result for ourselves.

As you can see, mistakes were made in 1995. Lest you judge me, the (now former) government of Mexico forced my hand! What followed made a lot of people unhappy, and then the French admin collapsed amidst the general chaos.

I did keep the green line up, though, so we'll call it the good ending. It took a few tries.

Unforeseen consequences

resource 'STR ' (705, purgeable) {
	"2* response* reply* answer* reaction*"
};

resource 'STR ' (706, purgeable) {
	"2* is coming* is being sent* is on its way* will arrive*"
};

resource 'STR ' (707, purgeable) {
	"2* via the North Pole.* over your northern border.* by ICBM.* by m"
	"issile courier.*"
};

It's not an easy game. You could know the rules like the back of your hand, and it still wouldn't eliminate the risks.

The crisis dialogue has one final surprise in store for us. As you click through the DEFCONs, there's a random factor that becomes increasingly relevant. It represents the likelihood of an oopsie:

x:=0;
CASE CrisisLevel OF
  2: x:=16;
  3: x:=8;
  4: x:=2;
END;
IF DipAff^^[1,2]>0 THEN x:=0 ELSE x:=(x*(-DipAff^^[1,2])) div 64;
y:=Random div 128;
IF x>(Abs(y)) THEN BEGIN CrisisLevel:=1; ANWFlag:=TRUE; END;

Watch out for that ANWFlag! The risk is at its highest when going to DEFCON 2 with superpower relations bottomed out (as they'd be in a shooting war): (16 * 127 / 64) / 128 = 0.248, or roughly a 25% chance of unintended fireworks.

In most cases, the probability will be much lower. If your relations are merely kind of bad at -32, and you go to DEFCON 4, the risk is 1/128, or 0.8%. In the real world that's not so low, but for the purposes of BOP it's a low probability/high impact risk you'll be taking again and again.

Good thing, then, that we only have to make it through eight years. The risks add up over time, and even if you play quite carefully, you will come to fear a certain arresting memo:

Thanks, I'm good

The "END" button drops you back to the desktop. In order to succeed at this game you need patience, a cool head and plenty of luck.

October 31, 2022

ephemerons and finalizers

Good day, hackfolk. Today we continue the series on garbage collection with some notes on ephemerons and finalizers.

conjunctions and disjunctions

First described in a 1997 paper by Barry Hayes, which attributes the invention to George Bosworth, ephemerons are a kind of weak key-value association.

Thinking about the problem abstractly, consider that the garbage collector's job is to keep live objects and recycle memory for dead objects, making that memory available for future allocations. Formally speaking, we can say:

  • An object is live if it is in the root set

  • An object is live it is referenced by any live object.

This circular definition uses the word any, indicating a disjunction: a single incoming reference from a live object is sufficient to mark a referent object as live.

Ephemerons augment this definition with a conjunction:

  • An object V is live if, for an ephemeron E containing an association betweeen objects K and V, both E and K are live.

This is a more annoying property for a garbage collector to track. If you happen to mark K as live and then you mark E as live, then you can just continue to trace V. But if you see E first and then you mark K, you don't really have a direct edge to V. (Indeed this is one of the main purposes for ephemerons: associating data with an object, here K, without actually modifying that object.)

During a trace of the object graph, you can know if an object is definitely alive by checking if it was visited already, but if it wasn't visited yet that doesn't mean it's not live: we might just have not gotten to it yet. Therefore one common implementation strategy is to wait until tracing the object graph is done before tracing ephemerons. But then we have another annoying problem, which is that tracing ephemerons can result in finding more live ephemerons, requiring another tracing cycle, and so on. Mozilla's Steve Fink wrote a nice article on this issue earlier this year, with some mitigations.

finalizers aren't quite ephemerons

All that is by way of introduction. If you just have an object graph with strong references and ephemerons, our definitions are clear and consistent. However, if we add some more features, we muddy the waters.

Consider finalizers. The basic idea is that you can attach one or a number of finalizers to an object, and that when the object becomes unreachable (not live), the system will invoke a function. One way to imagine this is a global association from finalizable object O to finalizer F.

As it is, this definition is underspecified in a few ways. One, what happens if F references O? It could be a GC-managed closure, after all. Would that prevent O from being collected?

Ephemerons solve this problem, in a way; we could trace the table of finalizers like a table of ephemerons. In that way F would only be traced if O is live already, so that by itself it wouldn't keep O alive. But then if O becomes dead, you'd want to invoke F, so you'd need it to be live, so reachability of finalizers is not quite the same as ephemeron-reachability: indeed logically all F values in the finalizer table are live, because they all will be invoked at some point.

In the end, if F references O, then F actually keeps O alive. Whether this prevents O from being finalized depends on our definition for finalizability. We could say that an object is finalizable if it is found to be unreachable after a full trace, and the finalizers F are in the root set. Or we could say that an object is finalizable if it is unreachable after a partial trace, in which finalizers are not themselves in the initial root set, and instead we trace them after determining the finalizable set.

Having finalizers in the initial root set is unfortunate: there's no quick check you can make when adding a finalizer to signal this problem to the user, and it's very hard to convey to a user exactly how it is that an object is referenced. You'd have to add lots of gnarly documentation on top of the already unavoidable gnarliness that you already had to write. But, perhaps it is a local maximum.

Incidentally, you might think that you can get around these issues by saying "don't reference objects from their finalizers", and that's true in a way. However it's not uncommon for finalizers to receive the object being finalized as an argument; after all, it's that object which probably encapsulates the information necessary for its finalization. Of course this can lead to the finalizer prolonging the longevity of an object, perhaps by storing it to a shared data structure. This is a risk for correct program construction (the finalized object might reference live-but-already-finalized objects), but not really a burden for the garbage collector, except in that it's a serialization point in the collection algorithm: you trace, you compute the finalizable set, then you have to trace the finalizables again.

ephemerons vs finalizers

The gnarliness continues! Imagine that O is associated with a finalizer F, and also, via ephemeron E, some auxiliary data V. Imagine that at the end of the trace, O is unreachable and so will be dead. Imagine that F receives O as an argument, and that F looks up the association for O in E. Is the association to V still there?

Guile's documentation on guardians, a finalization-like facility, specifies that weak associations (i.e. ephemerons) remain in place when an object becomes collectable, though I think in practice this has been broken since Guile switched to the BDW-GC collector some 20 years ago or so and I would like to fix it.

One nice solution falls out if you prohibit resuscitation by not including finalizer closures in the root set and not passing the finalizable object to the finalizer function. In that way you will never be able to look up E×OV, because you don't have O. This is the path that JavaScript has taken, for example, with WeakMap and FinalizationRegistry.

However if you allow for resuscitation, for example by passing finalizable objects as an argument to finalizers, I am not sure that there is an optimal answer. Recall that with resuscitation, the trace proceeds in three phases: first trace the graph, then compute and enqueue the finalizables, then trace the finalizables. When do you perform the conjunction for the ephemeron trace? You could do so after the initial trace, which might augment the live set, protecting some objects from finalization, but possibly missing ephemeron associations added in the later trace of finalizable objects. Or you could trace ephemerons at the very end, preserving all associations for finalizable objects (and their referents), which would allow more objects to be finalized at the same time.

Probably if you trace ephemerons early you will also want to trace them later, as you would do so because you think ephemeron associations are important, as you want them to prevent objects from being finalized, and it would be weird if they were not present for finalizable objects. This adds more serialization to the trace algorithm, though:

  1. (Add finalizers to the root set?)

  2. Trace from the roots

  3. Trace ephemerons?

  4. Compute finalizables

  5. Trace finalizables (and finalizer closures if not done in 1)

  6. Trace ephemerons again?

These last few paragraphs are the reason for today's post. It's not clear to me that there is an optimal way to compose ephemerons and finalizers in the presence of resuscitation. If you add finalizers to the root set, you might prevent objects from being collected. If you defer them until later, you lose the optimization that you can skip steps 5 and 6 if there are no finalizables. If you trace (not-yet-visited) ephemerons twice, that's overhead; if you trace them only once, the user could get what they perceive as premature finalization of otherwise reachable objects.

In Guile I think I am going to try to add finalizers to the root set, pass the finalizable to the finalizer as an argument, and trace ephemerons twice if there are finalizable objects. I think this wil minimize incoming bug reports. I am bummed though that I can't eliminate them by construction.

Until next time, happy hacking!

October 30, 2022

Pixel Inktober

Just like last year, October was filled with quick pixel dailies. I decided to only post on mastodon, but due to the twitter exodus couldn’t quite post the 30kB images for the two remaining days. Good old blog post it is!

x 1. Gargoyle 2. Scurry 3. Bat 4. Scallop 5. Flame 6. Bouquet 7. Trip 8. Match 9. Nest 10. Crabby 11. Eagle 12. Forget 13. Kind 14. Empty 15. Armadillo 16. Fowl 17. Salty 18. Scrape 19. Ponytail 20. Bluff 21. Bad Dog 22. Heist 23. Booger 24. Fairy 25. Tempting 26. Ego 27. Snack 28. Camping 29. Uh-oh 30. Gear 31. Farm

Previously, Previously, Previously.

On deprecations

If you are paying attention to GTK’s git repository, you may have noticed a change in the last weeks.

We have a directory gtk/deprecations, which is destined to contain source files that implement deprecated APIs and will be dropped in the next major release. For the 4.0 release, we emptied it out, and it has been empty ever since. But recently, it started to accumulate files again.

This is a good opportunity to remind folks how we are using deprecations in GTK. But first, lets take a look at the details.

The details, part 1: cell renderers

In GTK 4, we introduced a new family of list and grid widgets that are based around list models: GtkListView, GtkColumnView, GtkGridView. There is also a new combo box implementation using list models, called GtkDropDown. Taken together, these are meant to provide replacements for everything you can do with cell renderers in GTK 3.

The ultimate goal was to remove cell renderers, since they are a whole separate rendering and layout system that tends to interfere with GTK’s CSS and layout machinery, and makes everything more complicated.

But we did not quite get to the finish line for 4.0, mainly because we still had significant uses of treeviews in GTK itself. First and foremost, the file chooser.  Since the filechooser is getting ported to use a GtkColumnView in 4.10, now is the right time to deprecate the cell renderer machinery and all the widgets that use them.

This is a significant amount of code, more than 75.000 lines.

The details, part 2: dialogs

In GTK 4, we dropped gtk_main() and gtk_dialog_run(), since recursive mainloops are best avoided. Again, we did not get to the finish line and could not remove GtkDialog itself, since it is used as the base class for all our complex dialogs.

GTK 4.10 introduces replacement APIs for our ‘Chooser’ dialogs. The new APIs follow the gio async pattern. Here is an example:

GtkFileDialog * gtk_file_dialog_new (void);

void            gtk_file_dialog_open (GtkFileDialog *self,
                                      GtkWindow *parent,
                                      GFile *current_file,
                                      GCancellable *cancellable,
                                      GAsyncReadyCallback callback,
                                      gpointer user_data);

GFile *        gtk_file_dialog_open_finish (GtkFileDialog *self,
                                            GAsyncResult *result,
                                            GError **error);

This may look a bit unwieldy in C, but it translates very nicely to languages that have a concept of promises and exceptions:

try {
  const file = await dialog.open(parent, ...);
  
  ...
} catch (e) {
  ...
};

To learn more about the new APIs, you can look at their online docs: GtkColorDialog, GtkFontDialog, GtkFileDialog, GtkAlertDialog.

With these replacements in place, we could deprecate the Chooser interfaces, their widget implementations, and their base class GtkDialog.

No need to panic

Deprecations in GTK are an early outlook at changes that will appear in the next major release that is breaking API compatibility.  But the eventual GTK 5 release is still far away. We have not even made a plan for it yet.

There is absolutely no need to rush towards ‘deprecation cleanup’. You only need to remove all uses of deprecations when you want to port to GTK 5 – which does not exist yet.

There are still things you can do, though. We are introducing deprecations in 4.10 as a way to give our users time to adapt, and to provide feedback on our ideas. If you want to do so, you can file an issue in gitlab, start a discussion in discourse, or find us on matrix.

In the meantime…

Deprecation warnings can be annoying, but thankfully there are easy ways to turn them off. For the occasional call to a deprecated function, it is best to just wrap it in G_GNUC_BEGIN/END_IGNORE_DEPRECATIONS:

G_GNUC_BEGIN_IGNORE_DEPRECATIONS
gtk_dialog_add_button (dialog, "Apply", GTK_RESPONSE_APPLY);
G_GNUC_END_IGNORE_DEPRECATIONS

If you are sure that you never ever want to see any deprecation warnings, you can also just pass -Wno-deprecated-declarations to gcc.

October 23, 2022

Brave New Trusted Boot World

🔐 Brave New Trusted Boot World 🚀

This document looks at the boot process of general purpose Linux distributions. It covers the status quo and how we envision Linux boot to work in the future with a focus on robustness and simplicity.

This document will assume that the reader has comprehensive familiarity with TPM 2.0 security chips and their capabilities (e.g., PCRs, measurements, SRK), boot loaders, the shim binary, Linux, initrds, UEFI Firmware, PE binaries, and SecureBoot.

Problem Description

Status quo ante of the boot logic on typical Linux distributions:

  • Most popular Linux distributions generate initrds locally, and they are unsigned, thus not protected through SecureBoot (since that would require local SecureBoot key enrollment, which is generally not done), nor TPM PCRs.

  • Boot chain is typically Firmware → shimgrub → Linux kernel → initrd (dracut or similar) → root file system

  • Firmware’s UEFI SecureBoot protects shim, shim’s key management protects grub and kernel. No code signing protects initrd. initrd acquires the key for encrypted root fs from the user (or TPM/FIDO2/PKCS11).

  • shim/grub/kernel is measured into TPM PCR 4, among other stuff

  • EFI TPM event log reports measured data into TPM PCRs, and can be used to reconstruct and validate state of TPM PCRs from the used resources.

  • No userspace components are typically measured, except for what IMA measures

  • New kernels require locally generating new boot loader scripts and generating a new initrd each time. OS updates thus mean fragile generation of multiple resources and copying multiple files into the boot partition.

Problems with the status quo ante:

  • initrd typically unlocks root file system encryption, but is not protected whatsoever, and trivial to attack and modify offline

  • OS updates are brittle: PCR values of grub are very hard to pre-calculate, as grub measures chosen control flow path, not just code images. PCR values vary wildly, and OS provided resources are not measured into separate PCRs. Grub’s PCR measurements might be useful up to a point to reason about the boot after the fact, for the most basic remote attestation purposes, but useless for calculating them ahead of time during the OS build process (which would be desirable to be able to bind secrets to future expected PCR state, for example to bind secrets to an OS in a way that it remain accessible even after that OS is updated).

  • Updates of a boot loader are not robust, require multi-file updates of ESP and boot partition, and regeneration of boot scripts

  • No rollback protection (no way to cryptographically invalidate access to TPM-bound secrets on OS updates)

  • Remote attestation of running software is needlessly complex since initrds are generated locally and thus basically are guaranteed to vary on each system.

  • Locking resources maintained by arbitrary user apps to TPM state (PCRs) is not realistic for general purpose systems, since PCRs will change on every OS update, and there’s no mechanism to re-enroll each such resource before every OS update, and remove the old enrollment after the update.

  • There is no concept to cryptographically invalidate/revoke secrets for an older OS version once updated to a new OS version. An attacker thus can always access the secrets generated on old OSes if they manage to exploit an old version of the OS — even if a newer version already has been deployed.

Goals of the new design:

  • Provide a fully signed execution path from firmware to userspace, no exceptions

  • Provide a fully measured execution path from firmware to userspace, no exceptions

  • Separate out TPM PCRs assignments, by “owner” of measured resources, so that resources can be bound to them in a fine-grained fashion.

  • Allow easy pre-calculation of expected PCR values based on booted kernel/initrd, configuration, local identity of the system

  • Rollback protection

  • Simple & robust updates: one updated file per concept

  • Updates without requiring re-enrollment/local preparation of the TPM-protected resources (no more “brittle” PCR hashes that must be propagated into every TPM-protected resource on each OS update)

  • System ready for easy remote attestation, to prove validity of booted OS, configuration and local identity

  • Ability to bind secrets to specific phases of the boot, e.g. the root fs encryption key should be retrievable from the TPM only in the initrd, but not after the host transitioned into the root fs.

  • Reasonably secure, automatic, unattended unlocking of disk encryption secrets should be possible.

  • “Democratize” use of PCR policies by defining PCR register meanings, and making binding to them robust against updates, so that external projects can safely and securely bind their own data to them (or use them for remote attestation) without risking breakage whenever the OS is updated.

  • Build around TPM 2.0 (with graceful fallback for TPM-less systems if desired, but TPM 1.2 support is out of scope)

Considered attack scenarios and considerations:

  • Evil Maid: neither online nor offline (i.e. “at rest”), physical access to a storage device should enable an attacker to read the user’s plaintext data on disk (confidentiality); neither online nor offline, physical access to a storage device should allow undetected modification/backdooring of user data or OS (integrity), or exfiltration of secrets.

  • TPMs are assumed to be reasonably “secure”, i.e. can securely store/encrypt secrets. Communication to TPM is not “secure” though and must be protected on the wire.

  • Similar, the CPU is assumed to be reasonably “secure”

  • SecureBoot is assumed to be reasonably “secure” to permit validated boot up to and including shim+boot loader+kernel (but see discussion below)

  • All user data must be encrypted and authenticated. All vendor and administrator data must be authenticated.

  • It is assumed all software involved regularly contains vulnerabilities and requires frequent updates to address them, plus regular revocation of old versions.

  • It is further assumed that key material used for signing code by the OS vendor can reasonably be kept secure (via use of HSM, and similar, where secret key information never leaves the signing hardware) and does not require frequent roll-over.

Proposed Construction

Central to the proposed design is the concept of a Unified Kernel Image (UKI). These UKIs are the combination of a Linux kernel image, and initrd, a UEFI boot stub program (and further resources, see below) into one single UEFI PE file that can either be directly invoked by the UEFI firmware (which is useful in particular in some cloud/Confidential Computing environments) or through a boot loader (which is generally useful to implement support for multiple kernel versions, with interactive or automatic selection of image to boot into, potentially with automatic fallback management to increase robustness).

UKI Components

Specifically, UKIs typically consist of the following resources:

  1. An UEFI boot stub that is a small piece of code still running in UEFI mode and that transitions into the Linux kernel included in the UKI (e.g., as implemented in sd-stub, see below)

  2. The Linux kernel to boot in the .linux PE section

  3. The initrd that the kernel shall unpack and invoke in the .initrd PE section

  4. A kernel command line string, in the .cmdline PE section

  5. Optionally, information describing the OS this kernel is intended for, in the .osrel PE section (derived from /etc/os-release of the booted OS). This is useful for presentation of the UKI in the boot loader menu, and ordering it against other entries, using the included version information.

  6. Optionally, information describing kernel release information (i.e. uname -r output) in the .uname PE section. This is also useful for presentation of the UKI in the boot loader menu, and ordering it against other entries.

  7. Optionally, a boot splash to bring to screen before transitioning into the Linux kernel in the .splash PE section

  8. Optionally, a compiled Devicetree database file, for systems which need it, in the .dtb PE section

  9. Optionally, the public key in PEM format that matches the signatures of the .pcrsig PE section (see below), in a .pcrpkey PE section.

  10. Optionally, a JSON file encoding expected PCR 11 hash values seen from userspace once the UKI has booted up, along with signatures of these expected PCR 11 hash values, matching a specific public key in the .pcrsig PE section. (Note: we use plural for “values” and “signatures” here, as this JSON file will typically carry a separate value and signature for each PCR bank for PCR 11, i.e. one pair of value and signature for the SHA1 bank, and another pair for the SHA256 bank, and so on. This ensures when enrolling or unlocking a TPM-bound secret we’ll always have a signature around matching the banks available locally (after all, which banks the local hardware supports is up to the hardware). For the sake of simplifying this already overly complex topic, we’ll pretend in the rest of the text there was only one PCR signature per UKI we have to care about, even if this is not actually the case.)

Given UKIs are regular UEFI PE files, they can thus be signed as one for SecureBoot, protecting all of the individual resources listed above at once, and their combination. Standard Linux tools such as sbsigntool and pesign can be used to sign UKI files.

UKIs wrap all of the above data in a single file, hence all of the above components can be updated in one go through single file atomic updates, which is useful given that the primary expected storage place for these UKIs is the UEFI System Partition (ESP), which is a vFAT file system, with its limited data safety guarantees.

UKIs can be generated via a single, relatively simple objcopy invocation, that glues the listed components together, generating one PE binary that then can be signed for SecureBoot. (For details on building these, see below.)

Note that the primary location to place UKIs in is the EFI System Partition (or an otherwise firmware accessible file system). This typically means a VFAT file system of some form. Hence an effective UKI size limit of 4GiB is in place, as that’s the largest file size a FAT32 file system supports.

Basic UEFI Stub Execution Flow

The mentioned UEFI stub program will execute the following operations in UEFI mode before transitioning into the Linux kernel that is included in its .linux PE section:

  1. The PE sections listed are searched for in the invoked UKI the stub is part of, and superficially validated (i.e. general file format is in order).

  2. All PE sections listed above of the invoked UKI are measured into TPM PCR 11. This TPM PCR is expected to be all zeroes before the UKI initializes. Pre-calculation is thus very straight-forward if the resources included in the PE image are known. (Note: as a single exception the .pcrsig PE section is excluded from this measurement, as it is supposed to carry the expected result of the measurement, and thus cannot also be input to it, see below for further details about this section.)

  3. If the .splash PE section is included in the UKI it is brought onto the screen

  4. If the .dtb PE section is included in the UKI it is activated using the Devicetree UEFI “fix-up” protocol

  5. If a command line was passed from the boot loader to the UKI executable it is discarded if SecureBoot is enabled and the command line from the .cmdline used. If SecureBoot is disabled and a command line was passed it is used in place of the one from .cmdline. Either way the used command line is measured into TPM PCR 12. (This of course removes any flexibility of control of the kernel command line of the local user. In many scenarios this is probably considered beneficial, but in others it is not, and some flexibility might be desired. Thus, this concept probably needs to be extended sooner or later, to allow more flexible kernel command line policies to be enforced via definitions embedded into the UKI. For example: allowing definition of multiple kernel command lines the user/boot menu can select one from; allowing additional allowlisted parameters to be specified; or even optionally allowing any verification of the kernel command line to be turned off even in SecureBoot mode. It would then be up to the builder of the UKI to decide on the policy of the kernel command line.)

  6. It will set a couple of volatile EFI variables to inform userspace about executed TPM PCR measurements (and which PCR registers were used), and other execution properties. (For example: the EFI variable StubPcrKernelImage in the 4a67b082-0a4c-41cf-b6c7-440b29bb8c4f vendor namespace indicates the PCR register used for the UKI measurement, i.e. the value “11”).

  7. An initrd cpio archive is dynamically synthesized from the .pcrsig and .pcrpkey PE section data (this is later passed to the invoked Linux kernel as additional initrd, to be overlaid with the main initrd from the .initrd section). These files are later available in the /.extra/ directory in the initrd context.

  8. The Linux kernel from the .linux PE section is invoked with with a combined initrd that is composed from the blob from the .initrd PE section, the dynamically generated initrd containing the .pcrsig and .pcrpkey PE sections, and possibly some additional components like sysexts or syscfgs.

TPM PCR Assignments

In the construction above we take possession of two PCR registers previously unused on generic Linux distributions:

  • TPM PCR 11 shall contain measurements of all components of the UKI (with exception of the .pcrsig PE section, see above). This PCR will also contain measurements of the boot phase once userspace takes over (see below).

  • TPM PCR 12 shall contain measurements of the used kernel command line. (Plus potentially other forms of parameterization/configuration passed into the UKI, not discussed in this document)

On top of that we intend to define two more PCR registers like this:

  • TPM PCR 15 shall contain measurements of the volume encryption key of the root file system of the OS.

  • [TPM PCR 13 shall contain measurements of additional extension images for the initrd, to enable a modularized initrd – not covered by this document]

(See the Linux TPM PCR Registry for an overview how these four PCRs fit into the list of Linux PCR assignments.)

For all four PCRs the assumption is that they are zero before the UKI initializes, and only the data that the UKI and the OS measure into them is included. This makes pre-calculating them straightforward: given a specific set of UKI components, it is immediately clear what PCR values can be expected in PCR 11 once the UKI booted up. Given a kernel command line (and other parameterization/configuration) it is clear what PCR values are expected in PCR 12.

Note that these four PCRs are defined by the conceptual “owner” of the resources measured into them. PCR 11 only contains resources the OS vendor controls. Thus it is straight-forward for the OS vendor to pre-calculate and then cryptographically sign the expected values for PCR 11. The PCR 11 values will be identical on all systems that run the same version of the UKI. PCR 12 only contains resources the administrator controls, thus the administrator can pre-calculate PCR values, and they will be correct on all instances of the OS that use the same parameters/configuration. PCR 15 only contains resources inherently local to the local system, i.e. the cryptographic key material that encrypts the root file system of the OS.

Separating out these three roles does not imply these actually need to be separate when used. However the assumption is that in many popular environments these three roles should be separate.

By separating out these PCRs by the owner’s role, it becomes straightforward to remotely attest, individually, on the software that runs on a node (PCR 11), the configuration it uses (PCR 12) or the identity of the system (PCR 15). Moreover, it becomes straightforward to robustly and securely encrypt data so that it can only be unlocked on a specific set of systems that share the same OS, or the same configuration, or have a specific identity – or a combination thereof.

Note that the mentioned PCRs are so far not typically used on generic Linux-based operating systems, to our knowledge. Windows uses them, but given that Windows and Linux should typically not be included in the same boot process this should be unproblematic, as Windows’ use of these PCRs should thus not conflict with ours.

To summarize:

PCR Purpose Owner Expected Value before UKI boot Pre-Calculable
11 Measurement of UKI components and boot phases OS Vendor Zero Yes
(at UKI build time)
12 Measurement of kernel command line, additional kernel runtime configuration such as systemd credentials, systemd syscfg images Administrator Zero Yes
(when system configuration is assembled)
13 System Extension Images of initrd
(and possibly more)
(Administrator) Zero Yes
(when set of extensions is assembled)
15 Measurement of root file system volume key
(Possibly later more: measurement of root file system UUIDs and labels and of the machine ID /etc/machine-id)
Local System Zero Yes
(after first boot once ll such IDs are determined)

Signature Keys

In the model above in particular two sets of private/public key pairs are relevant:

  • The SecureBoot key to sign the UKI PE executable with. This controls permissible choices of OS/kernel

  • The key to sign the expected PCR 11 values with. Signatures made with this key will end up in the .pcrsig PE section. The public key part will end up in the .pcrpkey PE section.

Typically the key pair for the PCR 11 signatures should be chosen with a narrow focus, reused for exactly one specific OS (e.g. “Fedora Desktop Edition”) and the series of UKIs that belong to it (all the way through all the versions of the OS). The SecureBoot signature key can be used with a broader focus, if desired. By keeping the PCR 11 signature key narrow in focus one can ensure that secrets bound to the signature key can only be unlocked on the narrow set of UKIs desired.

TPM Policy Use

Depending on the intended access policy to a resource protected by the TPM, one or more of the PCRs described above should be selected to bind TPM policy to.

For example, the root file system encryption key should likely be bound to TPM PCR 11, so that it can only be unlocked if a specific set of UKIs is booted (it should then, once acquired, be measured into PCR 15, as discussed above, so that later TPM objects can be bound to it, further down the chain). With the model described above this is reasonably straight-forward to do:

  • When userspace wants to bind disk encryption to a specific series of UKIs (“enrollment”), it looks for the public key passed to the initrd in the /.extra/ directory (which as discussed above originates in the .pcrpkey PE section of the UKI). The relevant userspace component (e.g. systemd) is then responsible for generating a random key to be used as symmetric encryption key for the storage volume (let’s call it disk encryption key _here, DEK_). The TPM is then used to encrypt (“seal”) the DEK with its internal Storage Root Key (TPM SRK). A TPM2 policy is bound to the encrypted DEK. The policy enforces that the DEK may only be decrypted if a valid signature is provided that matches the state of PCR 11 and the public key provided in the /.extra/ directory of the initrd. The plaintext DEK key is passed to the kernel to implement disk encryption (e.g. LUKS/dm-crypt). (Alternatively, hardware disk encryption can be used too, i.e. Intel MKTME, AMD SME or even OPAL, all of which are outside of the scope of this document.) The TPM-encrypted version of the DEK which the TPM returned is written to the encrypted volume’s superblock.

  • When userspace wants to unlock disk encryption on a specific UKI, it looks for the signature data passed to the initrd in the /.extra/ directory (which as discussed above originates in the .pcrsig PE section of the UKI). It then reads the encrypted version of the DEK from the superblock of the encrypted volume. The signature and the encrypted DEK are then passed to the TPM. The TPM then checks if the current PCR 11 state matches the supplied signature from the .pcrsig section and the public key used during enrollment. If all checks out it decrypts (“unseals”) the DEK and passes it back to the OS, where it is then passed to the kernel which implements the symmetric part of disk encryption.

Note that in this scheme the encrypted volume’s DEK is not bound to specific literal PCR hash values, but to a public key which is expected to sign PCR hash values.

Also note that the state of PCR 11 only matters during unlocking. It is not used or checked when enrolling.

In this scenario:

  • Input to the TPM part of the enrollment process are the TPM’s internal SRK, the plaintext DEK provided by the OS, and the public key later used for signing expected PCR values, also provided by the OS. – Output is the encrypted (“sealed”) DEK.

  • Input to the TPM part of the unlocking process are the TPM’s internal SRK, the current TPM PCR 11 values, the public key used during enrollment, a signature that matches both these PCR values and the public key, and the encrypted DEK. – Output is the plaintext (“unsealed”) DEK.

Note that sealing/unsealing is done entirely on the TPM chip, the host OS just provides the inputs (well, only the inputs that the TPM chip doesn’t know already on its own), and receives the outputs. With the exception of the plaintext DEK, none of the inputs/outputs are sensitive, and can safely be stored in the open. On the wire the plaintext DEK is protected via TPM parameter encryption (not discussed in detail here because though important not in scope for this document).

TPM PCR 11 is the most important of the mentioned PCRs, and its use is thus explained in detail here. The other mentioned PCRs can be used in similar ways, but signatures/public keys must be provided via other means.

This scheme builds on the functionality Linux’ LUKS2 functionality provides, i.e. key management supporting multiple slots, and the ability to embed arbitrary metadata in the encrypted volume’s superblock. Note that this means the TPM2-based logic explained here doesn’t have to be the only way to unlock an encrypted volume. For example, in many setups it is wise to enroll both this TPM-based mechanism and an additional “recovery key” (i.e. a high-entropy computer generated passphrase the user can provide manually in case they lose access to the TPM and need to access their data), of which either can be used to unlock the volume.

Boot Phases

Secrets needed during boot-up (such as the root file system encryption key) should typically not be accessible anymore afterwards, to protect them from access if a system is attacked during runtime. To implement this the scheme above is extended in one way: at certain milestones of the boot process additional fixed “words” should be measured into PCR 11. These milestones are placed at conceptual security boundaries, i.e. whenever code transitions from a higher privileged context to a less privileged context.

Specifically:

  • When the initrd initializes (“initrd-enter”)

  • When the initrd transitions into the root file system (“initrd-leave”)

  • When the early boot phase of the OS on the root file system has completed, i.e. all storage and file systems have been set up and mounted, immediately before regular services are started (“sysinit”)

  • When the OS on the root file system completed the boot process far enough to allow unprivileged users to log in (“complete”)

  • When the OS begins shut down (“shutdown”)

  • When the service manager is mostly finished with shutting down and is about to pass control to the final phase of the shutdown logic (“final”)

By measuring these additional words into PCR 11 the distinct phases of the boot process can be distinguished in a relatively straight-forward fashion and the expected PCR values in each phase can be determined.

The phases are measured into PCR 11 (as opposed to some other PCR) mostly because available PCRs are scarce, and the boot phases defined are typically specific to a chosen OS, and hence fit well with the other data measured into PCR 11: the UKI which is also specific to the OS. The OS vendor generates both the UKI and defines the boot phases, and thus can safely and reliably pre-calculate/sign the expected PCR values for each phase of the boot.

Revocation/Rollback Protection

In order to secure secrets stored at rest, in particular in environments where unattended decryption shall be possible, it is essential that an attacker cannot use old, known-buggy – but properly signed – versions of software to access them.

Specifically, if disk encryption is bound to an OS vendor (via UKIs that include expected PCR values, signed by the vendor’s public key) there must be a mechanism to lock out old versions of the OS or UKI from accessing TPM based secrets once it is determined that the old version is vulnerable.

To implement this we propose making use of one of the “counters” TPM 2.0 devices provide: integer registers that are persistent in the TPM and can only be increased on request of the OS, but never be decreased. When sealing resources to the TPM, a policy may be declared to the TPM that restricts how the resources can later be unlocked: here we use one that requires that along with the expected PCR values (as discussed above) a counter integer range is provided to the TPM chip, along with a suitable signature covering both, matching the public key provided during sealing. The sealing/unsealing mechanism described above is thus extended: the signature passed to the TPM during unsealing now covers both the expected PCR values and the expected counter range. To be able to use a signature associated with an UKI provided by the vendor to unseal a resource, the counter thus must be at least increased to the lower end of the range the signature is for. By doing so the ability is lost to unseal the resource for signatures associated with older versions of the UKI, because their upper end of the range disables access once the counter has been increased far enough. By carefully choosing the upper and lower end of the counter range whenever the PCR values for an UKI shall be signed it is thus possible to ensure that updates can invalidate prior versions’ access to resources. By placing some space between the upper and lower end of the range it is possible to allow a controlled level of fallback UKI support, with clearly defined milestones where fallback to older versions of an UKI is not permitted anymore.

Example: a hypothetical distribution FooOS releases a regular stream of UKI kernels 5.1, 5.2, 5.3, … It signs the expected PCR values for these kernels with a key pair it maintains in a HSM. When signing UKI 5.1 it includes information directed at the TPM in the signed data declaring that the TPM counter must be above 100, and below 120, in order for the signature to be used. Thus, when the UKI is booted up and used for unlocking an encrypted volume the unlocking code must first increase the counter to 100 if needed, as the TPM will otherwise refuse unlocking the volume. The next release of the UKI, i.e. UKI 5.2 is a feature release, i.e. reverting back to the old kernel locally is acceptable. It thus does not increase the lower bound, but it increases the upper bound for the counter in the signature payload, thus encoding a valid range 100…121 in the signed payload. Now a major security vulnerability is discovered in UKI 5.1. A new UKI 5.3 is prepared that fixes this issue. It is now essential that UKI 5.1 can no longer be used to unlock the TPM secrets. Thus UKI 5.3 will bump the lower bound to 121, and increase the upper bound by one, thus allowing a range 121…122. Or in other words: for each new UKI release the signed data shall include a counter range declaration where the upper bound is increased by one. The lower range is left as-is between releases, except when an old version shall be cut off, in which case it is bumped to one above the upper bound used in that release.

UKI Generation

As mentioned earlier, UKIs are the combination of various resources into one PE file. For most of these individual components there are pre-existing tools to generate the components. For example the included kernel image can be generated with the usual Linux kernel build system. The initrd included in the UKI can be generated with existing tools such as dracut and similar. Once the basic components (.linux, .initrd, .cmdline, .splash, .dtb, .osrel, .uname) have been acquired the combination process works roughly like this:

  1. The expected PCR 11 hashes (and signatures for them) for the UKI are calculated. The tool for that takes all basic UKI components and a signing key as input, and generates a JSON object as output that includes both the literal expected PCR hash values and a signature for them. (For all selected TPM2 banks)

  2. The EFI stub binary is now combined with the basic components, the generated JSON PCR signature object from the first step (in the .pcrsig section) and the public key for it (in the .pcrpkey section). This is done via a simple “objcopy” invocation resulting in a single UKI PE binary.

  3. The resulting EFI PE binary is then signed for SecureBoot (via a tool such as sbsign or similar).

Note that the UKI model implies pre-built initrds. How to generate these (and securely extend and parameterize them) is outside of the scope of this document, but a related document will be provided highlighting these concepts.

Protection Coverage of SecureBoot Signing and PCRs

The scheme discussed here touches both SecureBoot code signing and TPM PCR measurements. These two distinct mechanisms cover separate parts of the boot process.

Specifically:

  • Firmware/Shim SecureBoot signing covers bootloader and UKI

  • TPM PCR 11 covers the UKI components and boot phase

  • TPM PCR 12 covers admin configuration

  • TPM PCR 15 covers the local identity of the host

Note that this means SecureBoot coverage ends once the system transitions from the initrd into the root file system. It is assumed that trust and integrity have been established before this transition by some means, for example LUKS/dm-crypt/dm-integrity, ideally bound to PCR 11 (i.e. UKI and boot phase).

A robust and secure update scheme for PCR 11 (i.e. UKI) has been described above, which allows binding TPM-locked resources to a UKI. For PCR 12 no such scheme is currently designed, but might be added later (use case: permit access to certain secrets only if the system runs with configuration signed by a specific set of keys). Given that resources measured into PCR 15 typically aren’t updated (or if they are updated loss of access to other resources linked to them is desired) no update scheme should be necessary for it.

This document focuses on the three PCRs discussed above. Disk encryption and other userspace may choose to also bind to other PCRs. However, doing so means the PCR brittleness issue returns that this design is supposed to remove. PCRs defined by the various firmware UEFI/TPM specifications generally do not know any concept for signatures of expected PCR values.

It is known that the industry-adopted SecureBoot signing keys are too broad to act as more than a denylist for known bad code. It is thus probably a good idea to enroll vendor SecureBoot keys wherever possible (e.g. in environments where the hardware is very well known, and VM environments), to raise the bar on preparing rogue UKI-like PE binaries that will result in PCR values that match expectations but actually contain bad code. Discussion about that is however outside of the scope of this document.

Whole OS embedded in the UKI

The above is written under the assumption that the UKI embeds an initrd whose job it is to set up the root file system: find it, validate it, cryptographically unlock it and similar. Once the root file system is found, the system transitions into it.

While this is the traditional design and likely what most systems will use, it is also possible to embed a regular root file system into the UKI and avoid any transition to an on-disk root file system. In this mode the whole OS would be encapsulated in the UKI, and signed/measured as one. In such a scenario the whole of the OS must be loaded into RAM and remain there, which typically restricts the general usability of such an approach. However, for specific purposes this might be the design of choice, for example to implement self-sufficient recovery or provisioning systems.

Proposed Implementations & Current Status

The toolset for most of the above is already implemented in systemd and related projects in one way or another. Specifically:

  1. The systemd-stub (or short: sd-stub) component implements the discussed UEFI stub program

  2. The systemd-measure tool can be used to pre-calculate expected PCR 11 values given the UKI components and can sign the result, as discussed in the UKI Image Generation section above.

  3. The systemd-cryptenroll and systemd-cryptsetup tools can be used to bind a LUKS2 encrypted file system volume to a TPM and PCR 11 public key/signatures, according to the scheme described above. (The two components also implement a “recovery key” concept, as discussed above)

  4. The systemd-pcrphase component measures specific words into PCR 11 at the discussed phases of the boot process.

  5. The systemd-creds tool may be used to encrypt/decrypt data objects called “credentials” that can be passed into services and booted systems, and are automatically decrypted (if needed) immediately before service invocation. Encryption is typically bound to the local TPM, to ensure the data cannot be recovered elsewhere.

Note that systemd-stub (i.e. the UEFI code glued into the UKI) is distinct from systemd-boot (i.e. the UEFI boot loader than can manage multiple UKIs and other boot menu items and implements automatic fallback, an interactive menu and a programmatic interface for the OS among other things). One can be used without the other – both sd-stub without sd-boot and vice versa – though they integrate nicely if used in combination.

Note that the mechanisms described are relatively generic, and can be implemented and be consumed in other software too, systemd should be considered a reference implementation, though one that found comprehensive adoption across Linux distributions.

Some concepts discussed above are currently not implemented. Specifically:

  1. The rollback protection logic is currently not implemented.

  2. The mentioned measurement of the root file system volume key to PCR 15 is implemented, but not merged into the systemd main branch yet.

The UAPI Group

We recently started a new group for discussing concepts and specifications of basic OS components, including UKIs as described above. It's called the UAPI Group. Please have a look at the various documents and specifications already available there, and expect more to come. Contributions welcome!

Glossary

TPM

Trusted Platform Module; a security chip found in many modern systems, both physical systems and increasingly also in virtualized environments. Traditionally a discrete chip on the mainboard but today often implemented in firmware, and lately directly in the CPU SoC.

PCR

Platform Configuration Register; a set of registers on a TPM that are initialized to zero at boot. The firmware and OS can “extend” these registers with hashes of data used during the boot process and afterwards. “Extension” means the supplied data is first cryptographically hashed. The resulting hash value is then combined with the previous value of the PCR and the combination hashed again. The result will become the new value of the PCR. By doing this iteratively for all parts of the boot process (always with the data that will be used next during the boot process) a concept of “Measured Boot” can be implemented: as long as every element in the boot chain measures (i.e. extends into the PCR) the next part of the boot like this, the resulting PCR values will prove cryptographically that only a certain set of boot components can have been used to boot up. A standards compliant TPM usually has 24 PCRs, but more than half of those are already assigned specific meanings by the firmware. Some of the others may be used by the OS, of which we use four in the concepts discussed in this document.

Measurement

The act of “extending” a PCR with some data object.

SRK

Storage Root Key; a special cryptographic key generated by a TPM that never leaves the TPM, and can be used to encrypt/decrypt data passed to the TPM.

UKI

Unified Kernel Image; the concept this document is about. A combination of kernel, initrd and other resources. See above.

SecureBoot

A mechanism where every software component involved in the boot process is cryptographically signed and checked against a set of public keys stored in the mainboard hardware, implemented in firmware, before it is used.

Measured Boot

A boot process where each component measures (i.e., hashes and extends into a TPM PCR, see above) the next component it will pass control to before doing so. This serves two purposes: it can be used to bind security policy for encrypted secrets to the resulting PCR values (or signatures thereof, see above), and it can be used to reason about used software after the fact, for example for the purpose of remote attestation.

initrd

Short for “initial RAM disk”, which – strictly speaking – is a misnomer today, because no RAM disk is anymore involved, but a tmpfs file system instance. Also known as “initramfs”, which is also misleading, given the file system is not ramfs anymore, but tmpfs (both of which are in-memory file systems on Linux, with different semantics). The initrd is passed to the Linux kernel and is basically a file system tree in cpio archive. The kernel unpacks the image into a tmpfs (i.e., into an in-memory file system), and then executes a binary from it. It thus contains the binaries for the first userspace code the kernel invokes. Typically, the initrd’s job is to find the actual root file system, unlock it (if encrypted), and transition into it.

UEFI

Short for “Unified Extensible Firmware Interface”, it is a widely adopted standard for PC firmware, with native support for SecureBoot and Measured Boot.

EFI

More or less synonymous to UEFI, IRL.

Shim

A boot component originating in the Linux world, which in a way extends the public key database SecureBoot maintains (which is under control from Microsoft) with a second layer (which is under control of the Linux distributions and of the owner of the physical device).

PE

Portable Executable; a file format for executable binaries, originally from the Windows world, but also used by UEFI firmware. PE files may contain code and data, categorized in labeled “sections”

ESP

EFI System Partition; a special partition on a storage medium that the firmware is able to look for UEFI PE binaries in to execute at boot.

HSM

Hardware Security Module; a piece of hardware that can generate and store secret cryptographic keys, and execute operations with them, without the keys leaving the hardware (though this is configurable). TPMs can act as HSMs.

DEK

Disk Encryption Key; an asymmetric cryptographic key used for unlocking disk encryption, i.e. passed to LUKS/dm-crypt for activating an encrypted storage volume.

LUKS2

Linux Unified Key Setup Version 2; a specification for a superblock for encrypted volumes widely used on Linux. LUKS2 is the default on-disk format for the cryptsetup suite of tools. It provides flexible key management with multiple independent key slots and allows embedding arbitrary metadata in a JSON format in the superblock.

Thanks

I’d like to thank Alain Gefflaut, Anna Trikalinou, Christian Brauner, Daan de Meyer, Luca Boccassi, Zbigniew Jędrzejewski-Szmek for reviewing this text.

Making Rust attractive for writing GTK applications

Rust, the programming language, has been gaining traction across many software disciplines - support for it has landed in the upstream Linux kernel, developers have been using it for games, websites, low-level OS components, and desktop applications.

The gtk-rs team has been doing an impressive amount of work during the last few years to make the experience of using GObject-based libraries in Rust enjoyable by providing high-quality, memory-safe bindings around those libraries, generated with gir from the introspection data.

Approximately two years ago and a few months before the release of GTK 4, I decided to take over the maintenance of gtk4-rs and push forward the initial work made by Xiang Fan during a Google Summer of Code internship. Nowadays, these are the most used GTK 4 bindings out there with probably more than 100 applications written in it, ranging from simple applications like Contrast to complex ones like Telegrand or Spot.

In this post, I will talk about the current status and what we have achieved since the first release of gtk4-rs.

Bindings

As mentioned above, a good portion of the bindings is generated automatically using gir. But sometimes, a manual implementation is needed, like in the following cases:

  • Make the code more Rust idiomatic
  • Handling cases that are too specific to be supported by gir. e.g, a x_get_type function being exposed only in a specific version.
  • Writing the necessary infrastructure code to allow developers to create custom GObjects, e.g. custom widgets or GStreamer plugins

Currently, gtk4-rs is composed of ~170 000 lines of code automatically generated and ~26 000 manually written, which is approximately 13% of manual code.

Subclassing

In the early days of gtk3-rs, the infrastructure for writing custom subclassses wasn't fully there yet, especially the amount of manual code that had to be written for supporting all the virtual functions (that can be overridden by a sub-implementation) of gtk::Widget was a lot. That caused people to avoid writing custom GTK widgets and do plenty of hacks like having one single ObjectWrapper GObject that would serialize a Rust struct into a JSON to store it as a string property in order to store these Rust types in a gio::ListModel and use it with gtk::ListBox::bind_model/gtk::FlowBox::bind_model.

Thankfully that is no longer the case for gtk4-rs as a huge amount of work went into manually implementing the necessary traits to support almost all of the types that can be subclassed with the exception of gtk::TreeModel (which would be deprecated starting from GTK 4.10), see https://github.com/gtk-rs/gtk4-rs/pull/169 for details why that didn't happen yet.

As more people started writing custom GTK widgets/GStreamer plugins, more people looking into simplifying the whole experience grew.

Creating a very simple and useless custom GTK widget looked like this the days of gtk-rs-core 0.10:

mod imp {
    pub struct SimpleWidget;

    impl ObjectSubclass for SimpleWidget {
        const NAME: &'static str = "SimpleWidget";
        type ParentType = gtk::Widget;
        type Instance = subclass::simple::InstanceStruct<Self>;
        type Class = subclass::simple::ClassStruct<Self>;

        glib_object_subclass!();

        fn new() -> Self {
            Self
        }
    }

    impl ObjectImpl for SimpleWidget {
        glib_object_impl!();
    }
    impl WidgetImpl for SimpleWidget {}
}
glib::wrapper! {
    pub struct SimpleWidget(ObjectSubclass<imp::SimpleWidget>)
        @extends gtk::Widget;
}

Nowadays it looks like this:

mod imp {
    #[derive(Default)]
    pub struct SimpleWidget;

    #[glib::object_subclass]
    impl ObjectSubclass for SimpleWidget {
        const NAME: &'static str = "SimpleWidget";
        type ParentType = gtk::Widget;
    }

    impl ObjectImpl for SimpleWidget {}
    impl WidgetImpl for SimpleWidget {}
}
glib::wrapper! {
    pub struct SimpleWidget(ObjectSubclass<imp::SimpleWidget>)
        @extends gtk::Widget;
}

Of course, the overall code is still a bit too verbose, particularly when you have to define GObject properties. However, people have been experimenting with writing a derive macro to simplify the properties declaration part, as well as a gobject::class! macro for generating most of the remaining boilerplate code. Note, those macros are still experiments and would need more time to mature before eventually getting merged upstream.

Composite Templates

In short, composite templates allow you to make your custom GtkWidget subclass use GTK XML UI definitions for providing the widget structure, splitting the UI code from it logic. The UI part can be either inlined in the code, or written in a separate file, with optional runtime validation using the xml_validation feature (unless you are using GResources).

mod imp {
    #[derive(Default, gtk::CompositeTemplate)]
    #[template(string = r#"
    <interface>
      <template class="SimpleWidget" parent="GtkWidget">
        <child>
          <object class="GtkLabel" id="label">
            <property name="label">foobar</property>
          </object>
        </child>
      </template>
    </interface>
    "#)]
    pub struct SimpleWidget {
        #[template_child]
        pub label: TemplateChild<gtk::Label>,
    }

    #[glib::object_subclass]
    impl ObjectSubclass for SimpleWidget {
        const NAME: &'static str = "SimpleWidget";
        type ParentType = gtk::Widget;

        fn class_init(klass: &mut Self::Class) {
            klass.bind_template();
        }

        fn instance_init(obj: &gtk::glib::subclass::InitializingObject<Self>) {
            obj.init_template();
        }
    }

    impl ObjectImpl for SimpleWidget {
        fn dispose(&self) {
            // since gtk 4.8 and only if you are using composite templates
            self.obj().dispose_template(Self::Type);
            // before gtk 4.8, for each direct child widget
            // while let Some(child) = self.obj().first_child() {
            //     child.unparent();
            // }
        }
    }
    impl WidgetImpl for SimpleWidget {}
}
glib::wrapper! {
    pub struct SimpleWidget(ObjectSubclass<imp::SimpleWidget>)
        @extends gtk::Widget;
}

Composite templates also allow you to also set a function to be called when a specific signal is emitted:

mod imp {
    #[derive(Default, gtk::CompositeTemplate)]
    #[template(string = r#"
    <interface>
      <template class="SimpleWidget" parent="GtkWidget">
        <child>
          <object class="GtkButton" id="button">
            <property name="label">Click me!</property>
            <signal name="clicked" handler="on_clicked" swapped="true" />
          </object>
        </child>
      </template>
    </interface>
    "#)]
    pub struct SimpleWidget {
        #[template_child]
        pub label: TemplateChild<gtk::Label>,
    }

    #[glib::object_subclass]
    impl ObjectSubclass for SimpleWidget {
        const NAME: &'static str = "SimpleWidget";
        type ParentType = gtk::Widget;

        fn class_init(klass: &mut Self::Class) {
            klass.bind_template();
            klass.bind_template_instance_callbacks();
        }

        fn instance_init(obj: &gtk::glib::subclass::InitializingObject<Self>) {
            obj.init_template();
        }
    }

    impl ObjectImpl for SimpleWidget {
        fn dispose(&self) {
            // since gtk 4.8 and only if you are using composite templates
            self.obj().dispose_template(Self::Type);
            // before gtk 4.8, for each direct child widget
            // while let Some(child) = self.obj().first_child() {
            //     child.unparent();
            // }
        }
    }
    impl WidgetImpl for SimpleWidget {}
}
glib::wrapper! {
    pub struct SimpleWidget(ObjectSubclass<imp::SimpleWidget>)
        @extends gtk::Widget;
}

#[gtk::template_callbacks]
impl SimpleWidget {
    #[template_callback]
    fn on_clicked(&self, _button: &gtk::Button) {
        println!("Clicked!");
    }
}

More details can be found in https://gtk-rs.org/gtk4-rs/stable/latest/docs/gtk4_macros/derive.CompositeTemplate.html and https://gtk-rs.org/gtk4-rs/stable/latest/docs/gtk4_macros/attr.template_callbacks.html.

Book, Documentations, Examples

Great bindings by themselves sadly won't help newcomers trying to learn GTK for the first time, despite the increasing number of apps written using gtk4-rs.

For that reason, Julian Hofer has been writing a "GUI development with Rust and GTK 4" book that you can find at https://gtk-rs.org/gtk4-rs/stable/latest/book/.

On top of that, we spend a good amount of time ensuring the documentation we provide, which is based on the C library documentation, has valid intra-links, uses the images provided by GTK C documentation to represent the various widgets, and that most of the types and functions are properly documented.

The gtk4-rs repository also includes ~35 examples that you can find in https://github.com/gtk-rs/gtk4-rs/tree/master/examples. Finally there's a GTK & Rust application repository template that you can use to get started with your next project: https://gitlab.gnome.org/World/Rust/gtk-rust-template.

Flatpak & Rust SDKs

Beyond the building blocks provided by the bindings, we have also worked on providing stable and nightly Rust SDKs that can be used to either distribute your application as a Flatpak or as a development environment. The stable SDK comes with the Mold linker pre-installed as well, which is recommended for improving build times.

Flatpak can be used as a development environment with either GNOME Builder, VSCode with the flatpak-vscode extension, or fenv, if you prefer the CLI for building/running your application.

Portals

Modern Linux applications should make use of the portals to request runtime permissions to access resources, such as capturing the camera feed, starting a screencast session or picking a file.

ASHPD is currently the way to go for using portals from Rust as it provides a convenient and idomatic API on top of the DBus one.

Code example from ashpd-0.4.0-alpha.1 for using the color picker in a desktop environment agnostic way:

use ashpd::desktop::screenshot::ColorResponse;

async fn run() -> ashpd::Result<()> {
    let color = ColorResponse::builder().build().await?;
    println!("({}, {}, {})", color.red(), color.green(), color.blue());
    Ok(())
}

Keyring & oo7

Applications that need to store sensitive information, such as passwords, usually use the Secret service, a protocol supported by both gnome-keyring and kwallet nowadays. There were multiple attempts to provide a Rust wrapper for the DBus API but some common pitfalls were that they lacked async support, provided no integration with the secret portal, or they had no way to allow applications to migrate their secrets from the host keyring to the application sandboxed keyring.

Those were the primary reasons we started working on oo7. It is still in alpha stage, but should cover most of the use cases already.

GStreamer

Part of the experiments I did when porting Authenticator to GTK 4 was figuring out how I can replicate the internal GStreamer sink included in GTK to convert a video frame into a gdk::MemoryTexture in order to render it in some widget for QR code scanning purposes.

Jordan Petridis took over my WIP work and turned it into a proper GStreamer plugin written in Rust, see https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/tree/main/video/gtk4 for an example on how to use it in your application.

Missing Integrations

Better Gettext Support

Currently, gettext doesn't support Rust officially, and most importantly, it doesn't like the ! character, that is used by declarative macros in Rust, such as println! and format!, so we can't use string formatting for translatable texts.

Kévin Commaille has submitted a patch for upstream gettext but sadly it hasn't been reviewed yet :( For now people are working around this by manually replacing variable names with https://doc.rust-lang.org/std/primitive.str.html#method.replace, which is not ideal, but it is what we have for now.

Reduced Boilerplate in Subclassing Code

As mentioned above, subclassing code is still too verbose in some cases. Ideally, we would simplify most of it, since it is probably one of the most confusing things you have to deal with as a beginner to the gtk-rs ecosystem.

Even Better Documentation

I personally think our documentation has gotten a lot better in the last couple of releases but there are always things to improve. Here is my wishlist of things that I hope to find the time to work on for the next release:

Feel free to pick any of the above issues if you would like to help.

Automatically-Generated Subclassing Traits

Presently, we have to manually write all the necessary traits needed for making it possible to subclass a type or implement an interface.

Sadly, we can't do much about it today as we would need new gobject-introspection annotations, see e.g. https://gitlab.gnome.org/GNOME/gobject-introspection/-/issues/411.

Fix GNOME CI Template to Avoid Duplicated Builds

Most of the GNOME applications hosted on https://gitlab.gnome.org are using https://gitlab.gnome.org/GNOME/citemplates/ as a CI template to provide a Flatpak job. The template is inefficient when building a Rust application as it removes the Flatpak repository between a regular build and a test build, which means rebuilding all the crates a second time, see https://gitlab.gnome.org/GNOME/citemplates/-/issues/5. flatpak-builder itself already provides an easy way to run the tests of a specific module, which is what flatpak-github-actions uses, but I am yet to find the time and apply the same thing to GNOME's CI template.

Special Thanks

  • Julian Hofer for the gtk4-rs book
  • Christopher Davis, Jason Francis, Paolo Borelli for their work on composite templates macros
  • Sophie Herold for her awesome work implementing the portal backend of oo7
  • Zeeshan for zbus/zvariant which unlocked plenty of use cases
  • wayland-rs developers for making it possible to integrate Wayland clients with ASHPD
  • Every other contributor to the gtk-rs ecosystem
  • Ivan Molodetskikh, Sebastian Dröge and Christopher Davis for reviewing this post