September 24, 2023

2023-09-24 Sunday

  • Somehow ended up hacking over midnight; got AVX2 profiled - 2x faster than optimized C++ - for processing 8 pixels at once, a somewhat smaller win than expected.

Dia with GTK 3

If you are running Dia from the nigthly GNOME flatpak, the last update brought a big change: it is now a GTK 3 app. The UI shouldn't be much different, just some cosmetic adjustments due to theming and other like widget spacing. Also it now work with Wayland and HiDPI.

It is something I started over two years ago, and that for some reason I left on the back burner. So I did a final push and many thanks to Zander for reviewing and merging it. Now we shall hunt for regression. I already have a fix for the touchpad scrolling.

The version is numbered starting 0.98. So if it is Dia 0.98 or higher, then it's GTK 3.

There is still some more stuff to do. Fix a regression in the navigator widget, build without deprecation post GTK 3.8, etc.

September 23, 2023

2023-09-23 Saturday

  • Up; snatched breakfast at speed; got to the venue just in time to give a reasonably stock talk to students on why contributing to LibreOffice is awesome for student's skills as developers.
  • Call from H. trying to find a soldering iron to mend a failed Philips LCD screen PSU board; on to talk a little about a few economic themes.
    Some reflections on the economics of FLOSS (Hybrid PDF)
  • Caught up with Florian at some length; its been too long; bid 'bye to many friends, thanked Gabriel and other 1&1'ers - and headed for dinner with Caolan, Cor & Uwe.
  • Got to the Airport early; RyanAir flight delayed by around an hour shock - on-time RyanAir flights appear to be like the yetti for me.
  • Got blog updated, and slides up-loaded on the plane.

Outreachy Recap 2023

Making GNOME Platform Demos


The End

Hello! I didn't blog nearly as often as I should have, but I'm here now to give a comprehensive report of the work I managed to get done this summer! This has been such a rewarding experience, and I do not intend for this to be the end of my GNOME Contributions. I hope you all see more of me in the FOSS world.


Mentors

Special thanks to Sonny Piers and Andy Holmes for everything. I wasn't the perfect intern, but I couldn't have asked for better mentors. I'm so thankful to know such cool people


My teammates

Also another special thanks to my fellow interns that I got to work with this summer: Sriyansh Shivam and Akshay Warrier

Had a ton of fun and got a lot done with these awesome individuals.


Workbench

If you don't know what it is, please download it already. It is such a useful tool to have at your disposal.


My Contributions

Below are bulleted lists of Workbench Library entries and other contributions done in part by me.

User Interface Demos

  • Dialogs

  • ViewSwitcher

    alt

  • Preferences Window

    alt

  • Carousel

    alt

  • Stack

    alt

  • CenterBox

  • Breakpoints

  • Overlaysplit View

    alt

  • Drag-and-Drop

    alt

Media Demos

  • Video

    alt

  • Audio

  • Camera

    alt

  • Picture

  • Screencast

Text and Label Demos

  • Editable Label

  • Label

  • Pango

    alt

  • Spell Checker

Buttons and Menu Demos

  • Scale Button

  • Menu Button

    alt

  • DropDown

    alt

Additional Contributions

  • GTK CSS Example

  • Preview Screenshots now copy to clipboard \o/

  • Added new Libadwaita 1.4 widgets to older demos

  • HTTP Request Demo

  • Split File Dialog Entry into 3 Separate Demos


What I've Learned

  • I love working with GTK4 and Libadwaita. It allows us to create such beautiful and forward-thinking applications. The GNOME HIG is top-tier.

  • I need to ask or help when stuck.

  • How to write usfing Markdown, using that right now ;)

  • How awesome open-source development is.

  • I've learned what I want to do for the rest of my life. I want to contribute, even if I'm not the best at it or if life gets in the way. I might as well use what little skill I have to give back. The only appropriate attitude to wealth is to share it, and my coffers are filled with GNOME Documentation now.


What I Need to Do

  • Fix the Drop Zone entry

  • Contribute more

  • Get more involved in the community as a whole


Thanks and see you soon!

Edit: libmanette

I also completed a libmanette entry. libmanette is a simple GObject game controller library. The entry is functional, but libmanette is currently in need of a maintainer.

September 22, 2023

NewsFlash 3.0

The next version of NewsFlash is ready. And it comes packed with so much new features and speed improvements + a new look, that the jump to version 3 is more than justified.

Visual comparison to version 2.3

The most obvious difference is the use of the libadwaita 1.4 split views and toolbar views. The result are 3 columns slightly different in color. Each with boarder-less headerbars.

An awesome perk of using these widgets is nicer sizing behavior of the sidebar and the article list. With NewsFlash 2.3 these two columns stayed the exact same size at all times. All extra space was given to the article. Now additional space is distributed evenly

old fullscreen layout
new fullscreen layout

The article list features the new listview sections added in Gtk 4.12. Articles are grouped by day and the corresponding date is displayed as the section header.

Drag & Drop is back

And better than ever before.

The feature didn’t survive the transition from Gtk3 to Gtk4. Back in the NewsFlash 1.x.x days drag & drop was only usable to move a feed from one category to another. The position of elements in the sidebar would seemingly change at random after the drop happened.

A lot of work went into improving the experience of drag & drop. NewsFlash doesn’t try to keep the positions of items in sync with the backed service. But instead focuses on keeping things consistent locally. Items now move to the exact spot they were dragged to.

Subscribing to feeds works better on mobile

Clicking the plus icon used to spawn a popover that guided you through the process of adding either feeds, categories or tags.

Unfortunately this didn’t really work on mobile. Interacting with the on-screen-keyboard would close the popover making it impossible to complete the process.

So instead the popover now lets you spawn one of 3 dialogs.

Not only does this change make it possible to add feeds on mobile & other touch devices. But I would argue it is an improvement for desktop as well. No longer can the popover accidentally be closed by switching to another window.

The wizard guiding you through the process also got a facelift.

Speed

NewsFlash 3.0 should be a lot faster.

I went all-in on the tokio runtime and its ecosystem. No more spawning threads manually for a single operation. No more blocking locks. Everything is async and runs on a single tokio runtime which is way smarter about task managing than I ever could be.

At the same time the reqwest client is now reused as much as possible and only rebuild once relevant settings change.

Together these changes mean operations like syncing, marking articles as read and scraping full article content should feel more responsive.

The favicon cache is now based on moka which eliminates the case of starting multiple downloads of the same icon at the same time. Large favicons get scaled down to a reasonable size to improve loading times.

Remember the window state

One of the more often requested features: remember the window state when NewsFlash quits and restore everything on the next launch.

This can mean different things to different people: Some want the app to remember the window size, others if it displays all or only unread articles.

Here is a list of the things that are saved and restored:

  • window size (and maximized)
  • sidebar selection
  • article list mode (all, unread, starred)
  • search term
  • selected article
  • if original or scraped content is shown
  • article view zoom

Other things like exact scroll positions of lists are not saved.

View large images

Large images in the article can be clicked and open the image up in a new window. No fancy gestures or anything. But still a welcome feature.

The same dialog is used to display image attachments of articles.

More Thumbnails

Articles get more visual appealing with NewsFlash 3.0!

Thumbnails were supported before. But only if the article had an image attachment. And then only if the back-end supported attachments via their API (e.g. miniflux did not).

Going forward there is a heuristic to gather a “relevant” image from the article HTML and used that as a thumbnail.

Edit dialogs

The simple “rename” dialogs of old evolved into “edit” dialogs. The biggest change is the dialog for feeds. The feed can still be renamed, the feed URL is at least visible now (changing is not possible just yet) and it can be moved to another category from here as well (alternative to drag & drop).

Some math focused blogs use “$” as start and end marker for formulas. Always rendering text between “$” with mathjax lead to malformed articles that contained multiple dollar signs in their regular text. This can now be controlled on a feed-by-feed basis.

An idea for the future is to allow a feed-by-feed setting if an attempt to download the full article content right after a sync should be made.

Commafeed

Commafeed  is the newest addition to the list of supported services. Its an open source web based feed reader.

The implementation is not as battle tested as the others. So if you hit a roadblock with it please let me know.

More

Above only the more significant features and changes were listed. There are a lot of smaller features and bug fixes. So if you previously had trouble with anything it may be worth to try NewsFlash again.

Features that did not make it: Video player

You can not view video attachments or youtube videos in a new embedded player. Video streaming is hard. GStreamer does a lot of the work for you. But there are still a lot of gaps that an application has to fill to make a nice streaming player.

I was encountering image freezes, stuck videos and outright gstreamer crashes from time to time. Not to mention missing features like switching video streams on the fly, which I left out since it added even more problems to the table.

So its back to the drawing board. But here is how it could have landed in NewsFlash 3.0:

#114 Forty-five!

Update on what happened across the GNOME project in the week from September 15 to September 22.

This week we released GNOME 45!

This new major release of GNOME is full of exciting changes, such as a new activity indicator, a brand new image viewer and camera app, faster search, new styled sidebars, additional privacy indicators, and so much more! See the GNOME 45 release notes for more information.

Readers who have been following this site for a few weeks will already know some of the new features. If you want to follow the development of GNOME 46 (Spring 2024), keep an eye on this page - we’ll be posting exciting news every week!

GNOME Core Apps and Libraries

Calendar

A simple calendar application.

Dallas Strouse (she/her) 🧑‍🏫 says

Calendar has completely rethought its scrolling behavior; in version 44 and prior, the scrolling lacked overall smoothness and was too sensitive to scrolling, and was especially annoying to use on touchpads. Calendar 45 now features smooth scrolling that scrolls between weeks, rather than whole months, which also helped to improve performance! You can view the details here, courtesy of Georges.

For the past three months, Hari Rana (TheEvilSkeleton) has been working on more Calendar bugfixes to improve accessibility and otherwise improve the user experience. Calendar has also become more keyboard navigable, with some assistance from Lukáš Tyrychtr to ensure usability, and many more fixes are planned for the future. If you’d like to see the work that Hari has worked on for Calendar so far, you can do so here.

Calendar has also been updated to use the dazzling new adaptive widgets, thanks to Christopher Davis (BrainBlasted).

If you’d like to help contribute to Calendar and make the experience better for yourself and others, see if you can help nab any Accessibility issues, and ask about how you can help in the #gnome-calendar:gnome.org channel. Jeff Fortin has also created a tracking issue which you can read here.

In other news, Calendar’s number of open issues has been slimmed all the way down to 283, from 570 back in February!

Oh, hey, Georges, you still haven’t given Skel that candy you offered :P

GNOME Circle Apps and Libraries

Rafael Mardojai CM announces

Forge Sparks 0.2.0 is now available on Flathub.

This release includes several user interface improvements thanks to the GNOME Circle review.

The initial view has been improved for a better experience, and account management has been moved to a separate window for easier interaction. It has also been updated to match the style of GNOME 45.

A few bugs have also been fixed, making it a bit more robust.

Workbench

A sandbox to learn and prototype with GNOME technologies.

Sonny announces

Workbench 45 is out and available on Flathub 🎉

This is a massive release with many new features such as Rust support, 50 new Library demos (Map, Spelling, Camera, …), multi-windows, offline documentation viewer etc. See the release announcement https://github.com/sonnyp/Workbench/releases/tag/v45.0

NewsFlash feed reader

Follow your favorite blogs & news sites.

Jan Lukas announces

NewFlash 3.0 was just released. There is so much new stuff in this release that I wrote a blog post about it: https://blogs.gnome.org/jangernert/2023/09/22/newsflash-3-0/

Identity

Compare images and videos.

Ivan Molodetskikh says

A new version of Identity is out, ready to go with visual tweaks to match the GNOME 45 design. Major improvements include pixelated, rather than blurry, zoomed-in media display; memory leak fixes, and improved format support, like AV1, WebP, semitransparent, and 16-bit images and videos.

Cartridges

Launch all your games

kramo announces

Cartridges 2.4 is out, updated with a sidebar to filter sources and better adaptivity thanks to the newest Libadwaita!

Check it out on Flathub!

Third Party Projects

baarkerlounger reports

This week I released Jogger - a fitness tracker for Gnome Mobile. With it you can:

  • Track workouts such as running, cycling, rowing and more using Geoclue based location
  • Get time or distance based voice announcements during your workouts (local or higher quality network based)
  • Manually enter past workouts
  • View workout details such as route, pace, calories burned etc
  • Import workouts from Fitotrack (Android) for users switching to mobile Linux.

Thanks to kramo for the icon design and to everyone who has translated Jogger so far. If you would like to contribute with design suggestions, code, translations, bug reports or ideas you can do so on Codeberg

You can get Jogger on Flathub.

Note. that if you’re installing on a Librem 5 you’ll need to update to a newer version of xdg_desktop_portal than is available in Pure OS Byzantium to get an accurate location fix. See this issue for more details.

dabrain34 announces

GstPipelineStudio version 0.3.3 is finally out on flathub, MacOS and Windows.

You can have a quick overview of the project on its website: https://dabrain34.pages.freedesktop.org/GstPipelineStudio/

The main features are:

  • Windows and MacOS installer
  • Open a pipeline with its command line description
  • Multiple graphview tabs allowing to play with multiple pipeline at once.
  • Capsfilter support on the link
  • gstreamer-1.0 wrap support
  • Bugs fixing

schmiddi says

Railway (previously DieBahn) version 2.0.0 was released! Railway allows you to plan your travel by public transport like train or bus, and furthermore allows you to keep up-to-date on the latest information during your journey. Railway supports numerous providers, for example from Germany, Austria, Luxembourg but also some providers from America. Version 2.0.0 is a massive overhaul of the UI, for which I have to thank Tobias Bernard for the mockups and feedback and also camelCaseNick for helping with the implementation. Railway is available on Flathub.

Turtle

Manage git repositories in Nautilus.

Philipp announces

Turtle is finally available on flathub!

I also released Turtle 0.5 recently, which adds more settings (filter remotes in log, commit default behaviour) and plugins for Thunar, Nemo and Caja.

Turtle 0.5.1 adds some minor flatpak tweaks.

If you want to use the Nautilus plugin combined with flatpak, you have to install it manually. Please follow the instructions in the readme.

Letterpress

Create beautiful ASCII art

gregorni reports

This week, I’ve released version 2.0 of Letterpress! We’ve simplified the versioning scheme (as you might have already noticed) and changed the app ID to be more consistent with the app name. The app icon has been revised (props to kramo), and the interface itself is now even uses a matching accent color! We’ve also updated to the latest libadwaita widgets, which allow for a nice and clean looking flat header bar!

A Tips dialog has been added that explains what kind of pictures are suitable for ASCII art.

The conversion to ASCII art has been improved, meaning the output will no longer have inverted colors in light mode! Pictures taken on a phone in portrait mode will also no longer appear rotated by 90°.

The output view itself now centers the ASCII output by default, and you can change the font size to zoom in and out of the view! For changing output width and zoom level, there are now also keyboard shortcuts.

The app has also been translated into Ukranian (by @volkov).

You can get the new release from Flathub. If you already have the app installed, just update, no need to worry about the new app ID.

Fretboard

Look up guitar chords

Brage Fuglseth announces

This week I released Fretboard 3.0, with the following improvements:

  • Refined visuals taking advantage of the new capabilities of GNOME 45
  • Keyboard shortcuts for common actions
  • Italian, Russian, and Dutch translations, making Fretboard available in a total of 4 different languages

If you would like to come with suggestions, report bugs, translate the app, or contribute otherwise, feel free to reach out! The app is now translatable through Weblate, which I hope will make it more straightforward to contribute.

You can get Fretboard on Flathub.

Cavalier

Visualize audio with CAVA.

Nick says

Cavalier V2023.9.0 is here! In this release a feature to set foreground image was added to Cavalier! You can also now set transparency on both background and foreground images. Combine beautiful pictures with gradients to give Cavalier unique look that fits your mood!

Here’s the full changelog:

  • Added ability to set foreground image for Box modes
  • Transparency can now be set both for background and foreground image
  • Cavalier switched back to using PulseAudio by default. You can still switch audio backend to whatever is supported by CAVA using CAVALIER_INPUT_METHOD environment variable
  • Updated translations (Thanks everyone on Weblate!)

GNOME Foundation

Caroline Henriksen announces

This week GNOME 45 was released! Along with assisting with the release announcements, sharing the news on social channels, and updating the website, one of my jobs as Foundation staff is coordinating the GNOME 45 release video. Big thanks to Freehive for their work producing this video!

Reminder: We’re still looking for sponsors for GNOME.Asia 2023! If you or your company would like to sponsor this year’s summit reach out to mwu@gnome.org for more information.

Volunteer Opportunity: We’re looking for people to help manage the Foundation’s social media channels. If you’re interested in getting involved reach out to chenriksen@gnome.org for more information.

That’s all for this week!

See you next week, and be sure to stop by #thisweek:gnome.org with updates on your own projects!

September 21, 2023

On the Usefulness of SO_PEERPIDFD

Kernel 6.5 added a few new pidfd functions: SCM_PIDFD and SO_PEERPIDFD. The idea behind them is the same as SCM_CREDENTIALS and SO_PEERCRED respectively. The only difference is that the PIDFD functions return not a plain, numerical PID but a file descriptor instead.

A plain PID is small number of type pid_t that is incremented for each new process and wraps over when too many processes have been created. This PID is usually used to look up some information about the process via files in /proc/$PID. While a process is looking up some information, it is possible that the process that PID initially referred to has terminated and a new process with this PID has been created. The looked up information is now incorrect, possibly resulting in a security vulnerability.

The pidfd on the other hand always refers to one process and can be queried about the state of the process. This allows one to look up information from /proc/$PID without the race mentioned earlier. The SO_PEERCRED functionality in particular is interesting because it allows a service to query the pidfd of a connected client.

Or so it seems. For flatpak, wayland compositors and D-Bus services also want to authenticate their clients but they do not rely on this functionality. Instead the preferred approach taken here (implemented in wayland as the security-context protocol, still in discussion for D-Bus as the org.freedesktop.DBus.Containers1 interface) is to create a new wayland or D-Bus socket for each application instance and make sure that those sockets are the only way to connect to the services (specifically the “normal” host sockets must not be made available to the application instances). Flatpak is responsible for creating those sockets, attaching some metadata, and then mounting them. Currently the metadata is a triple: the sandboxing engine (flatpak, snap, …), the application id and the application instance. There are plans to extend this for further metadata.

The question is, why add all of this complexity to authenticate a process when the pidfd approach is so much easier. Turns out that the hard part is knowing what to do with the pid, where to look up the information that you need. For flatpak, /proc/$PID/root/.flatpak-info is a file that cannot be changed from processes in the sandbox and contains among other things the flatpak instance-id which can be used to look up more data from $XDG_RUNTIME_DIR/.flatpak/$instance-id. For snap the whole process is very different and other technologies like firejail have basically no way to do this lookup (don’t take my word on it).

(Aside: xdg-desktop-portal does does look up flatpak information from /proc/$PID and I just said it doesn’t use SO_PEERCRED so why is this not broken? Flatpak makes sure there is a process in the sandbox that stays alive the entire time and acts as a proxy between the app and the D-Bus broker to enforce access control. The PID of all connections from inside the sandbox are the PID of the proxy. This is still technically racy but it becomes much harder to pull off an attack. Implementing SO_PEERCRED to get rid of the race entirely in xdg-desktop-portal would be nice.)

Implementing all kinds of different, subtle mechanisms to look up information from different sandbox engines you might not even know exist doesn’t scale and that makes pidfd with SO_PEERCRED much less useful than one would expect in a lot of cases.

Paths in GTK, part 2

In the first part of this series, we introduced the concept of paths and looked at how to create a GskPath. But there’s more to paths than that.

Path Points

Many interesting properties of paths can change as you move along the trajectory of the path. To query such properties, we first need a way to pin down the point on the path that we are interested in.

GTK has the GskPathPoint struct for this purpose, and provides a number of functions to obtain them, such as gsk_path_get_closest_point(), which lets you find the point on the path that is closest to a given point.

Once you have a GskPathPoint, you can query the properties of the path at that point. The most basic property is the position, but you can also get the tangent, the curvature, or the distance from the beginning of the path.

Input

Another interesting question when using paths in a user interface is:

Is the mouse pointer hovering over the path?

You need the answer to this question if you want to highlight a path that the pointer is over, or if you want to react to the user clicking a path.

For a filled path, GTK provides the answer with the gsk_path_in_fill() method.

For a stroked path, it is much more complicated to provide a 100% accurate answer (in particular, if the stroke is using a dash pattern), but we can provide an approximate answer that is often good enough: a point is inside the stroke, if the distance to the closest point on the path is less than half the line width.

Outlook

The next part of this series will look at rendering with paths.

September 19, 2023

Circles do not exist

Many logos, drawings and other graphical designs have the following shape in it. What is this shape?

If you thought: "Ah-ha! I'm smart and read the title of this blog post so I know that this is most definitely not a circle."

Well it is. Specifically it is a raster image of a circle that I created with the Gimp just for this use.

However almost every "circle" you can see in printed media (and most purely digital ones) are not, in fact, circles. Why is this?

Since roughly the mid 80s all "high quality" print jobs have been done either in PostScript or, nowadays almost exclusively, in PDF. They use the same basic drawing model, which does not have a primitive for circles (or circle arcs). The only primitives they have are straight line segments, rectangles and Bézier curves. None of these can be used to express a circle accurately. You can only do an approximation of a circle but it is always slightly eccentric. The only way to create a proper circle is to have a raster image like the one above.

Does this matter in the real world?

For printing probably not. Almost nobody can tell the difference between a real circle and one that has been approximated with a Bézier curve with just four points. Furthermore, the human vision system is a bit weird and perfect circles look vertically elongated. You have to make them non-circular for people to consider them properly circular.

But suppose you want to use one of these things:

This is a laser cutter that takes its "print job" as a PDF file and uses its vector drawing commands to drive the cutting head. This means that it is impossible to use it to print a wheel. You'd need to attach the output to a lathe and sand it down to be round so it actually functions as a wheel rather than as a vibration source.

Again one might ask whether this has any practical impact. For this case, again, probably not. But did you know that one of the cases PDF is being considered (and, based on Internet rumors, is already being used) is as an interchange format for CAD drawings? Now it suddenly starts mattering. If you have any component where getting a really accurate circle shape is vital (like pistons and their holes), suddenly all your components are slightly misshaped. Which would not be fun.

Extra bonus information

Even though it is impossible to construct a path that is perfectly circular, PDF does provide a way to draw a filled circle. Here is the relevant snippet from the PDF 2.0 spec, subsection 8.5.3.2:

If a subpath is degenerate (consists of a single-point closed path or of two or more points at the same coordinates), the S operator shall paint it only if round line caps have been specified, producing a filled circle centred at the single point.

Who is willing to put money on the line that every PDF rendering implementation actually uses circles rather than doing the simple thing of approximating it with Béziers?

Paths in GTK

It is no secret that we want to get rid of cairo as the drawing API in GTK, so we can move more of our drawing onto the GPU.

While People have found creative ways to draw things with render nodes, they don’t provide a comprehensive drawing API like Skia or, yes, cairo. Not a very satisfying state of affairs.

A few years ago, we started to investigate how to change this, by making paths available as first-class objects in GTK. This effort is finally starting to come to fruition, and you can see the first results in GTK 4.13.0.

Paths

So, what is a path? A rough definition could be:

A sequence of line segments or curves that may or may not be connected at their endpoints.

When we say curves, we specifically mean quadratic or cubic Bézier curves. On top of cairo, we also support rational quadratic Béziers (or as Skia calls them: conics), since they let us model circles and rounded rectangles precisely.

This picture shows a typical path, consisting of 4 curves and 2 lines, some of which are connected. As you can see, paths can be closed (like the 4 curves here) or open (like the 2 lines), with a start- and endpoint.

And how are paths useful for drawing? First, you can use a path to define an area (the part that’s inside the path) and fill it with a color, a gradient or some more complex content.

Alternatively, you can stroke the path with various properties such as line width, color or dash pattern.

Paths in GTK

The object that we use for paths in GTK is GskPath. It is a compact, immutable representation that is optimized for rendering. To create a GskPath, you need to use a GskPathBuilder, which has many convenience methods to create paths, either from individual curves or from predefined shapes.

This example creates a path that is a closed triangle:

builder = gsk_path_builder_new ();
gsk_path_builder_move_to (builder, 0, 50);
gsk_path_builder_line_to (builder, 100, 50);
gsk_path_builder_line_to (builder, 50, 0);
gsk_path_builder_close (builder);
path = gsk_path_builder_free_to_path (builder);

And this one creates a circular path with the given center and radius:

builder = gsk_path_builder_new ();
gsk_path_builder_add_circle (builder, center, radius);
path = gsk_path_builder_free_to_path (builder);

Outlook

In the next post, we’ll look at properties of paths, and how to query them.

September 15, 2023

Extending the month to infinity

Greetings! It’s been a long time since my last article.

I’d like to share some recent developments in GNOME Calendar that got some people really excited about: the infinitely scrolling month view.

The Now

Before GNOME 45, Calendar offers two views – week and month – as well as a sidebar with a list of events.

The headerbar offers controls to navigate back and forth in time. The effect of these controls depends on the current view: if you’re in the week view, going forward in time moves you a week ahead; in the month view, it moves you a month.

Both views have evolved to be strictly about their respective time ranges. That means the month view, for example, strictly deals with the current month. Days from other months that sneak into the view are not interactive, and it doesn’t show events on them. This has been a long-standing feature request in GNOME Calendar.

GNOME Calendar 44 (screenshot by @TheEvilSkeleton)

The week view doesn’t really suffer from the same problem, even though it has the same constraint, since weeks are not variable in length like months.

While this approach has served us well enough for more than a decade, it had significant usability issues. One of the primary goals of a calendaring application is to facilitate planning ahead. The static month view made harmed the usability of the application, in particular when in the last days of the month, since you could not see ahead without chaging the month entirely.

Another shortcoming of the static month view was scrolling. Because the view was bound to a particular month, there was no way to transition between months smoothly. I eventually added mouse & touchpad scroll support to it, but it has been a source of bugs and confusion since the view abruptly switches after an apparently random amount of scrolling.

Overall, despite the constant efforts to improve it, it felt like we were hitting the limitations of the current design. To the drawing board we needed to go.

New possibilities with GTK4?

GTK4 introduces a family of data-oriented widgets (GtkListView, GtkGridView, GtkColumnView) that are able to handle virtually endless datasets, we had a promising start with rethinking the month view. We could just stuff weeks into a GtkListView, or days into a GtkGridView, and be done with it, right?

Well, not quite.

There is a fundamental difference between datasets and timelines, which very directly informs the architecture of these GTK widgets: timelines are infinite, datasets are bounded.

No matter how many entries you add to a dataset, or how large you make that dataset, it will still have a countable number of items. Timelines are uncountably infinite, both towards the past and the future. ¹

This, by itself, directly affects the user interface we can present, and prevents us from using GtkListView, or pretty much anything that uses a GtkAdjustment as the underlying controller of position. I originally assumed that the number of workarounds to make a new month work with adjustment would be manageable, but after some weeks of experimentation, it became abundantly clear that this approach cannot work without either a massive number of hacks, and at the cost of maintainability and my own sanity.

Eventually I bowed to the fate, and wrote a completely custom layout for the new month.

Extending the month to infinity

As it happens so often in computing, we don’t necessarily have to implement material infinity to give people the impression of infinity.

The ultimate goal here is to make the month view smoothly scroll between weeks. In principle, the smallest way to create the illusion of infinity is to show current weeks, plus one or more offscreen rows above and below.

Due to the way calendars are stored by evolution-data-server, where data transfers happen over D-Bus, we have to be particularly careful to prefetch more events than visible onscreen, and keep them cached. Scrolling still changes the date range in which the month view operates – every time a week is moved up or down, we need to discard events of weeks that went out of range, and request events of week that are now within range.

All these range changes needs to be carefully managed so that we don’t blow evolution-data-server too much with range changes.

Fundamentally, the architecture of the new month view is based on a row recycling mechanism. Conceptually, the month view is just a list of rows, each row representing a week, laid out vertically given the available width. It’s as if the month view is a partial circular buffer of time. Moving the bottom row to the top, and vice versa, during scroll, allows the month view to keep a static number of row widgets allocated at all times,

After experimenting with different ranges and caching a bit, I’ve arrived at the wonderful number of 10 weeks of wiggle room – so in addition to the 5 visible week rows, there’s 10 weeks above them, and 10 weeks below them. This completely arbitrary value seems enough to cache just enough events for the transitions not to be noticeable.

The new month view can now anchor itself against any particular week, and is no more limited to a single month. On months that take precisely 5 rows, you’ll be able to see events from the previous and next month as well – something that’s been requested over and over for a long time now.

Most of the expected features – dragging events around, creating events, etc – continue to work exactly as they used to.

Touching the limits of infinity

Of course, being able to fetch, cache, layout, and render 25 weeks worth of events pushed Calendar’s backend to its limits. We had to make things fast, real fast, in order to keep up with e.g. scrolling at 60 FPS.

This was a good opportunity to revisit how the month view calculates the layout and overflow of each week. The layout algorithm had to be reworked almost from scratch, and I eventually figured out that it was still violating GTK4 layout rules during the layout phase – it was changing visibility of events during layout, which causes a relayout inside a relayout. These days, GTK4 has some tolerance for that, but after increasing the number of events by a factor of 5x, not even GTK4 could be so complacent with Calendar.

Now we pre-calculate the event layout while building up the list of events, and keep it cached in a size-agnostic way. During layout, it only resolves overflow, and it does so by carefully changing child visibility of the widgetry.

This not only enables the month view to actually render its contents, but we also don’t do any significant calculation at layout time, which makes Calendar smoother overall. Nice.

While developing this new month view, every single potential threading issue in Calendar showed up as a nasty crash as well. Most of them should be fixed now.

This, at last, allows us to have this nice, beautiful, smooth Calendar app:

The new month view in all its glory (credits to @TheEvilSkeleton again)

A story of ups and downs

It is not a secret that calendars aren’t the most exciting applications humankind has ever conceived. In an era where we have to look at hands to detect whether a picture is generated by AI; an era where we watch the dusk of both the eleventh and the 45-billion-dollar Xs; an era where we Linux has become a mainstream gaming platform, and all the problems that come with that; who on actual Earth gives a damn about a niche-of-a-niche calendar app?!

I do.

During the development of GNOME 45, I finally could squeeze some dry drops of free time to dedicate some love to Calendar again. It took some serious effort to make the new month view a reality, but looking at it now, I think it was worth it.

After the original maintainer and the main designer left the project and the community, around 2016, it felt a tad bit lonely. Motivation to continue working on was eroding at a slow but constant pace. Calendar’s issue tracker was out of control. The IRC and Matrix channels were dead. Attempts to get some funding for Calendar development led nowhere – who would be willing to pay people to work on such an uninteresting, unmarketable component of the desktop?! – and of course, there was no reason for day job to give me work time to dedicate to Calendar. The project was, by all measurements, a dying project. This was not healthy.

But lurking in the darkest corners of Canada, silent, and always on the watch, a night elf with personal investment in the project noticed the situation. And acted. Over the last couple of months, Jeff single-handedly tamed the issue tracker again; triaged and labeled every single issue; closed almost 300 issues; made everything actionable again; helped building up a roadmap; and, the most important to me, offered a friendly hand and brought back the fun of developing an open source app.

It made me realize that, contrary to what I believed for too many years, and despite not being exactly what’s advertised as free software culture, it really is all about people. The whole thing. Just being there, talking and discussing and having fun and eventually doing some contribution – that’s the sweet spot of free software culture to me. The reason I’m still involved with other GNOME projects? The people. The reason Calendar didn’t just die? The people.

I’m pretty attached to Calendar as it was the first project I contributed with code on GNOME, and it makes me happy to see that the project is slowly getting back to track, and a community is gathering around once again.

Join us in making this nice calendaring app at Matrix!


¹ – GNOME Calendar is resonably far from dealing with the granularity of Planck time!

Libadwaita 1.4

Screenshot showing a few apps using libadwaita 1.4, front to back: Files, Characters, Epiphany
A few apps using libadwaita 1.4

It’s that time of year again, so let’s look at what’s new.

New Adaptive Widgets

I’ve already talked about them in my last blog post, so I won’t go into details this time.

Breakpoints

Libadwaita 1.4 introduces a breakpoint system, allowing to change UI in arbitrary ways depending on the window size. Breakpoints can be used with AdwWindow, AdwApplicationWindow, or with AdwBreakpointBin if you need more control.

Breakpoints can be used in a fully declarative way from UI files, for example:

<object class="AdwBreakpoint">
  <condition>max-width: 500sp</condition>
  <setter object="split-view" property="collapsed">True</property>
</object>

As a tradeoff, you have to manually specify the window’s or bin’s minimum size and ensure its contents actually fit, same as you do on a small screen.

To help with that, GtkButton, GtkMenuButton, AdwSplitButton and AdwButtonContent now all include a :can-shrink property to enable text ellipsizing, while widgets like AdwBanner automatically enable it for their buttons in order to not get uncontrollably wide.

For breakpoint conditions one can use pixels (px), points (pt) or a new sp unit (scalable pixels, name lifted from Android), which is equivalent to pixels with default text scale, but scales with it: 1sp is equivalent to 1.25px with Large Text enabled and so on. To accommodate different text scale factors better, it is recommended to use sp whenever it’s feasible.

Navigation View

Screenshot of a small window with Page 2 in the title, a back button on the left and an "Open Page 3" button in the middle.

AdwNavigationView is an integrated widget implementing the browsing pattern, replacing AdwLeaflet with can-unfold=false. It provides a navigation stack that can be populated statically (e.g. from a UI file) or dynamically, and automatically provides gestures and shortcuts.

It also provides the navigation.push and navigation.pop actions, allowing to push pages directly from a UI file:

<object class="AdwActionRow">
  <property name="title" translatable="yes">_Details</property>
  <property name="use-underline">True</property>
  <property name="activatable">True</property>
  <property name="action-name">navigation.push</property>
  <property name="action-target">"details"</property>
  <child>
    <object class="GtkImage">
      <property name="icon-name">go-next-symbolic</property>
      <property name="accessible-role">presentation</property>
    </object>
  </child>
</object>

To further simplify using it, AdwHeaderBar can automatically show the correct title for each navigation page, as well as a back button to pop the current page when appropriate.

Automatic back buttons also provide a context menu that allows to pop multiple pages at once:

Screenshot of a back button context menu in Settings, opened on the Night Light page and showing Displays and Settings items. Displays is hovered

This still works with nested navigation views, as well with navigation views combined with split views.

Split Views

Screenshot of a window using a split view with prominent "Sidebar" and "Contents" labels in each pane and nothing else (except for a close button in the upper right corner)

While AdwNavigationView replaces the can-unfold=false case of AdwLeaflet, AdwNavigationSplitView replaces the other one.

It has two children: sidebar and content, and it displays them side by side. When the :collapsed property is set to TRUE, it literally turns into an AdwNavigationView. It doesn’t set it automatically though – you are supposed to do it from your breakpoints as needed.

It also provides a more sophisticated sizing for the sidebar, based on the percentage of the split view’s total width.

Meanwhile, AdwOverlaySplitView is similar, but instead of turning into a navigation view when collapsed, it overlays the sidebar over content, not unlike AdwFlap. As such, AdwFlap is what it replaces.

It has a few extra features compared to navigation split view, such as an ability to move the sidebar to the right and show or hide it even when not collapsed, but the two widgets have extremely similar API.

And, like with AdwNavigationView, AdwHeaderBar can integrate with split views: when put inside one, it will automatically hide redundant window buttons, so there’s no need to show or hide them manually like with AdwLeaflet or AdwFlap.

Toolbar View

The new split view styles really need flat header bars to work well. While we’ve had the .flat style class since libadwaita 1.0, in practice it’s quite limited, especially with scrolling content.

As such, there’s a new widget called AdwToolbarView. It contains a content widget and a number of top and bottom bars (for example, AdwHeaderBar, AdwTabBar, GtkSearchBar, GtkActionBar, or GtkBox with the .toolbar style class). Then it will automatically manage the correct styles for the toolbars, for example making them flat and managing undershoot shadows on scrolling content (though this can be changed using the :top-bar-style and :bottom-bar-style properties), as well as collapsing spacing between them:

Screenshot of GNOME Text Editor 45, showing a header bar and a tab bar inside a toolbar view

It’s recommended to always use it instead of GtkBox when you have header bars or other toolbars, regardless of whether you’re using split views.

Deprecations

With breakpoints and the new widgets, a number of older widgets have been deprecated, namely AdwLeaflet, AdwFlap, AdwSqueezer and AdwViewSwitcherTitle, as well as the old subpage API in AdwPreferencesWindow and the .flat style class for header bars. Refer to the migration guide for how exactly to replace them.

List Rows

There has been a number of boxed list additions this cycle.

Switch Row

Screenshot of a switch row in a group with the title "Switch Rows". The row has a title "Switch Row" and the switch is active.

Joshua Lee added AdwSwitchRow – a simple AdwActionRow subclass containing a GtkSwitch. While it’s easy to implement manually, it’s a very common case and so it’s nice to have a shortcut.

Spin Row

Screenshot of a spin row in a group with the title "Spin Rows". The row has a title "Spin Row" and the value is 50. Plus and minus buttons are both sensitive.

Chris added AdwSpinRow – a list row with an embedded GtkSpinButton, similar to AdwEntryRow.

Property Row

Screenshot of a property row in a group with the title "Property Rows". The row has a title "Property Row" and the subtitle "Value".

While it’s not a widget, the new .property style class, also by Chris, can swap styles on AdwActionRow‘s title and subtitle to emphasize the latter. This can be useful when displaying, say, EXIF properties in an image viewer.

Misc Changes

  • Jamie added adw_about_window_new_from_appdata() to simplify creating about windows.
  • AdwClamp can now scale with text scale factor, via the :unit property, incl. defaulting to the sp unit.
  • Yuri Izmer implemented search in AdwComboRow, matching GtkDropDown.

    Screenshot of a combo row search entry. The row value is "unknown" and you can see cropped "gpl-3-0" and "lgpl-2-1" in the list.

  • Maksym Hazevych added AdwPreferencesPage:description property, allowing to show a description at the top of the page.
  • Corey Berla fixed another bunch of drag-n-drop issues to make sure it works as expected in Nautilus.
  • The way AdwTabOverview handles thumbnails has been significantly reworked to make it work better with WebKitWebView.
  • Xenia added the AdwToast:use-markup property to allow disabling markup in toasts (it’s enabled by default).
  • A lot of accessibility issues throughout different widgets have been fixed – special thanks goes to Lukáš Tyrychtr and Maximiliano.
  • Header bars and other toolbars are now white instead of darker grey in the light variant, while the previous grey is now used for sidebars instead. Header bars set as GtkWindow titlebar now also have a shadow, same as when used in a toolbar view with top-bar-style=raised
  • While default GTK dialogs cannot use the new widgets, they have been styled to look similar anyway.

    Screenshot of a GTK file chooser. The sidebar is dark as if it's using a split view. The file chooser is showing libadwaita source directory, with Adw-1.metadata selected


As always, thanks to all the contributors who helped to make this release happen.

Status update, 15/09/2023

Musically this has been a fun month. One of my favourite things about living in Galicia is that ska-punk never went out of fashion here and you can legitimately go to a festival by the sea and watch Ska-P. Unexpectedly brilliant and chaotic live show. I saw an interview recently where Angelo Moore of Fishbone was asked by a puppet what his favourite music is, and he answered: “I like … I like the Looney Tunes music”. Same energy.

I wrote already this month about my DIY media server and the openQA CLI tool. This post contains some brief thoughts about Nushell and then some lengthy thoughts about the future of the Web. Enjoy!

Nushell everywhere

I read a blog by serial shell innovator JT entited “The case for Nushell”. I’ve been using Nushell for data-focused work for a while and the post inspired me to make it my default shell in a few places.

Nushell is really comfortable to use these days, it’s very addictive the first time you construct a one-liner to pretty-print some JSON or XML, select the fields you want and output a table as Markdown that you can paste straight into a Gitlab issue. My only complaint is the autocomplete isn’t quite as good as the Fish shell yet. (And that you can’t type rm -R… like chown and chmod only accept -R, and now rm only accepts a lower case -r, how am I supposed to remember that guys???)

I have a load of arcane Bash knowledge that I guess I’ll have to hang onto for a while yet, particularly as my job mostly involves SSH’ing into strange old machines from the 1990s. Perhaps I can try and contribute Nushell binaries that run on HP-UX and Solaris. (For the avoidance of doubt, that previous sentence is a joke).

Kagi Small Web

There’s a new search engine on the block called Kagi which is marketed as “premium search engine”, you pay $0.05 per search, and in return the results are ad-free.

I like this idea. I signed up for the free trial 100 searches, and I haven’t got far with them.

It turns out most of the web searches I do, are things I could search on a specific site if I wasn’t so lazy. For example I search “rust stdio” when I could go to the Rust documentation on my local machine and search there. Or I search for a programming problem when I could clearly just search StackOverflow itself. DuckDuckGo has made me lazy; adding a potential $0.05 cost to searches firstly makes you realize how few you actually need to do. Maybe this is a good thing.

Anyway, Kagi. They just launched something named Kagi Small Web, which is announced here:

Kagi Small Web offers a fresh approach by promoting recently published content from the “small web.” We gather new content, published within the last week, from a handpicked list of blogs and surface it in multiple ways:

  • Directly within Kagi search results for applicable queries (existing Kagi members do not need to do anything, this will be automatic)
  • Via the new Kagi Small Web website
  • Through the Kagi Small Web RSS feed
  • Via our Search API, where results are now part of the news enrichment API

Initially inspired by a vibrant discussion on Hacker News, we began our experiment in late July, highlighting blog posts from HN users within our search results. The positive feedback propelled the initiative forward. Today, our evolving concept boasts a curated list of nearly 6,000 genuine websites featuring people with a wide variety of interests.

When I first saw this my mind initially jumped to the problematic parts. Who are these guys to suddenly define what the Small Web is, and define it as a a club of some 6,000 websites chosen by Hackers News? All sites must be in English, so is the Web only for English speakers now?? More importantly, why is my site not on the list? Why wasn’t I consulted??

There’s also something very inspiring about the project. I try to follow the rule “something is better than nothing”, and this project is a pretty bold step forwards, which inspired a bunch of thoughts about the future of The Web.

Google Search is Dying

Since about 2000, when you think of the Web, you think of Google.

Google Search has been dying a slow, public death for about the last ten years. Google has been too big to innovate since the early 2010s (with one important exception, the Emoji Kitchen).

Google Search remained king until now for two reasons: one, their tech for turning hopelessly vague search queries into useful results was better than anyone’s in the industry, and two, as of 2023, almost nobody else can operate at the scale needed to index all of the text on the Web.

I guess there’s a third reason too, which is spending billions of $$$ to be the default search provider nearly everywhere, to the point that the USA is running an antitrust hearing against them, but let’s focus on the technical aspects.

The current fad for large language models is going to bring big changes to the Web, for better or worse. One of those is that “intent analysis” is suddenly much easier than it was. Note, I’m not talking about prompting an LLM with a question
and presenting the randomly generated output as an answer. I’m talking about taking unstructured text, such as “trains to London” and turning it into an actionable query. A 1990’s era search engine would throw away the “to” return any website that contained “trains” and “London”. Google Search shows a table of live departure times for trains heading to London. (With some less useful things above and below, since this is the Google Search of 2023).

A small LLM such as Vicuna can kinda just DO this stuff, not perfectly of course, but its an order of magnitude easier than a decade ago. Perhaps Google kept their own LLM research internal for so long for fear of losing exactly this edge? The “We have no moat” memo suggests fear.

Onto the second thing, indexing all the content on the Web. LLMs don’t make this easier. They make it impossible.

Its now so easy to generate human-like text on the Web using machines, that it doesn’t make sense to index all the text on the Web any more. Much of it is already human-generated generated garbage aiming to game search ranking algorithms (see “A storefront for robots” for fun examples).

Very soon 99% of text on the web will be machine generated garbage. Welcome to the dark forest.

For a short time I was worried about this, but I think it’s a natural evolution of the Web. This is the end of the Olde World Wide Web. What comes next?

There is more than one Small Web

If you’ve read this far, firstly, thanks and well done, in 2023 its hard to read so many paragraphs in one go! I didn’t even put in single video.

Let me share the insight I had on thinking over Kagi Small Web. Maybe it’s obvious and maybe it isn’t.

A search engine of 6,000 websites is small-scale enough that one person could conceivably run it.

Let’s go back a step. How do you deal with a Web that’s 99% machine-generated noise? I imagine Google will try to solve this by using language models to detect if the page was generated by a language model, triggering another fairly pointless technological arms race against the spammers who will be generating this stuff. This won’t work very well.

The only way for humans to make sense of the new Dark Forest Web is to have lists of websites, curated by humans, and to search through those when we want to find information.

If you’re old you know that this isn’t a new idea. In fact, we kinda had all of this stuff in the form of web rings, link pages on other people’s websites, bookmarking services, link sites like Digg, Fark and Reddit, RSS feeds and feed readers. If you look at Kagi Small Web reader site it’s literally a web ring. It’s StumbleUpon. It’s Planet GNOME. But through the lens of 2023, it’s
also something new.

So I’m not going to start reading Kagi’s small web, though it may be great. And I’m going stop capitalising “small web”, because I think we’re going to curate millions of these collectively, in thousands of languages, in infinite online communities. We’re going to have open source tools for searching and archiving high quality online content. Who knows? Perhaps in 10 years we’ll have small web search tools integrated into GNOME.

Further Reading

This year, 2023, is the 25th Year of our Google, and The Verge are publishing a series of excellent articles looking forwards and backwards. I can recommend reading “The end of the Googleverse” as a starting point. Another great one: “Google and YouTube are trying to have it both ways with AI and copyright“.

#113 Numerous Updates

Update on what happened across the GNOME project in the week from September 08 to September 15.

GNOME Core Apps and Libraries

Libadwaita

Building blocks for modern GNOME apps using GTK4.

Alice (she/they) reports

I just released libadwaita 1.4 and wrote an overview of the changes: https://blogs.gnome.org/alicem/2023/09/15/libadwaita-1-4/

GNOME Circle Apps and Libraries

Brage Fuglseth reports

Errands has been accepted into GNOME Circle. This app lets you keep track of your tasks with a carefully selected collection of features, like subtasks, custom colors and a history of previously cleared tasks. Congratulations!

Third Party Projects

dlippok announces

Since the last announcement on TWIG, the Photometric Viewer now has the ability to convert between IESNA and EULUMDAT file formats. Additionally, you can export photometric data in CSV and JSON formats, as well as light distribution curves in PNG or SVG formats. Furthermore, the software now allows you to calculate the annual power consumption of the luminaire and compare its light color temperature to that of common light sources. In addition to English, the application is also available in German and Polish.

Newest version is now available on Flathub and Snap Store.

Tagger

Tag your music.

Nick reports

Tagger V2023.9.1 is here!

This release includes many new features and improvements to Tagger’s music file system. The Music Folder has been replaced with the Music Library allowing users to not only open folders, but to open playlist files as well and manage the files referenced in the playlist all from within the app. We also added support for creating a playlist file from a folder and fixed some issues with synchronized lyrics.

Here’s the full changelog:

  • Added the ability to open, manage, and create playlists within Tagger. As a result, the “Music Folder” has now become the “Music Library” that can be opened from a folder or a supported playlist file
  • Synchronized lyric timestamps will now be shown in the mm:ss.xx format as per the LRC specification. When creating a new lryic, both hh:mm:ss and mm:ss.xx can be specified and Tagger will convert them appropriately
  • Tagger will display headers in the list of music files when sorting to provide a more organized view of files
  • Fixes an issue where lrc files were not importing correctly
  • Tagger’s main window size will be remembered and restored on application restart
  • Updated translations (Thanks everyone on Weblate!)

Parabolic

Download web video and audio.

Nick announces

Parabolic V2023.9.0 was released this week! This release includes some new feature to make Parabolic even more powerful and secure!

First, the Keyring is now backed by the system credential manager, meaning that users no longer need to provide a password to secure their Keyring as it would be handled by system secrets. Of course, we offer a guide inside the app’s help on moving old password-secured keyrings to the new secret-secured ones.

Besides keyrings, we improved Parabolic’s support for subtitles by downloading the correctly supported format for a file type and supporting two letter language with region codes. We also introduced a new preference for choosing to remove identifying media source information from embedded metadata.

Here’s the changelog:

  • Added support for two letter language with region codes
  • Added a preference for removing media source data from embedded metadata (previously it was automatically removed)
  • Parabolic will now use the system’s credential manager (i.e. DBus Secret Service) for securing the Keyring. Please read the Keyring page in Help for information on migrating the old password-secured Keyring to the new secret-secured Keyring
  • Parabolic will no longer ask for a subtitle format but instead use the format supported by the file type
  • Fixed an issue where translated metadata was embedded instead of original media metadata
  • Updated translations (Thanks everyone on Weblate!)

GNOME Network Displays

Stream the desktop to Wi-Fi Display capable devices

Pedro Sader Azevedo says

GNOME Network Displays can now embed your cursor in its video stream. No more guess work trying to figure out where your mouse pointer is!

PS: the glitchy vertical bars are my TV’s fault. It’s a an old TV!

Denaro

Manage your personal finances.

Nick reports

Denaro V2023.9.2 was released! This release includes many bug fixes for the new Currency Conversion tool we introduced in the last release. A huge thanks to @JoseBritto and @UrtsiSantsi who have contributed to finding these issues and creating fixes ❤️

Here’s the full changelog:

  • Fixed an issue where the currency conversion service would provide the wrong conversion rate
  • Fixed an issue where small converted currency amounts would show as 0
  • Fixed incorrect display of amounts for locales that use Cape Verdean escudo as currency
  • Improved UX in Currency Converter dialog
  • Updated translations (Thanks to everyone on Weblate)!

Cavalier

Visualize audio with CAVA.

Nick says

Cavalier got V2023.9.0-beta1 this week! Combine beautiful pictures with gradients to give Cavalier unique look that fits your mood!

Here’s the full changelog:

  • Added ability to set foreground image for Box modes
  • Transparency can now be set both for background and foreground image
  • Cavalier switched back to using PulseAudio by default. You can still switch audio backend to whatever is supported by CAVA using CAVALIER_INPUT_METHOD environment variable
  • Updated translations (Thanks everyone on Weblate!)

Shell Extensions

Just Perfection says

We’ve updated the EGO Review Guidelines and GNOME Shell 45 extensions no longer should import Gtk and Gdk to the shell process.

Weather O’Clock

Display the current weather inside the pill next to the clock.

Cleo Menezes Jr. says

Weather O’Clock version 9 was released with support for GNOME Shell 45 using the new symbolic weather icons. Check it out on EGO.

Just Perfection

A tweak tool to customize the GNOME Shell and to disable UI elements.

Just Perfection announces

Just Perfection Extension now supports GNOME Shell 45. In this version, we have a new animation option (almost none) that can disable GNOME Shell animation without disabling apps animation. App menu feature has been removed since there isn’t any app menu indicator in GNOME Shell 45. Activities button icon feature also got removed since GNOME Shell 45 uses workspace indicators in the activities button. This version is named after German painter Albrecht Durer. https://www.youtube.com/watch?v=rgIm2ggn67I

GNOME Foundation

Rosanna reports

It’s nearing the end of September, which for the GNOME Foundation, means we are nearing the end of our Fiscal Year. It’s sometimes a surprise to folks that fiscal years aren’t always aligned with the calendar year, but it’s common for corporations and non-profits. This always leads to a flurry of activity, between getting travel reimbursements and expenses pushed through so they are reflected this year’s books, working on the books to make sure there’s nothing outstanding, and looking at creating the budget for next year. There’s a lot to do, so please be patient if you are expecting something from me!

That’s all for this week!

See you next week, and be sure to stop by #thisweek:gnome.org with updates on your own projects!

September 14, 2023

GSOC 2023 Final Report

This is the final report for my project. Here i will be explaining about the method we took to find anagrams.

GNOME Crosswords Editor

Although still under development, Editor is a important part of Crosswords application for GNOME. It allows us to create basic crosswords with grids and clues.

Project Information

My project is add anagram-search support for the Crosswords Editor. The data for the searching comes from a word list file. My task is to search for anagrams. It needs to fast enough, so that the user can set the input word, and the search results are displayed instantaneously (without any lag).

The word-list file

To understand my project, you need to know about the data file i.e. word-list file. The file is made up of 3 sections:

  1. WordList: Just a huge list of words.
  2. FilterFragments: A list of word-indices that follow a particular pattern. e.g. A?? is pattern and its list will have index for words that start with A and have length of 3. Similarly for fragment ?B??, the list will have offsets for words that have second letter as B and length of 4.
  3. Index Section: A JSON block which stores indexes for the above sections.

Finding Anagrams

Approach 1

We simulate a trie to find anagrams of a word.

Trie to search for anagrams

Here at each node we maintain a list of words that follow the particular pattern, like AT? will have words having a prefix of AT and length 3. For branches where the list of words becomes empty like AT?, we perform branch culling.

We were successful in finding anagrams for a word using this approach, but this method is a bit slow to be directly used in a user interface. For a 12 letter word like ABBREVIATION, it took 0.3 seconds to find the anagrams. Thus we need another faster method.

Approach 2

Basic Idea

I will try to explain this with an example:

  • Two words: HEART and EARTH
  • Sort these words, we get AEHRT and AEHRT
  • Create a hash for both of them
  • Notice that they will have the same hash as the sorted words are same
  • We use this unique property to find anagrams.

Implementation

We solve this problem in two stages:

Stage 1: At the compile time

Create a new section in word-list file for anagram fragments, it has the following two parts:

  1. anagram word list: This is a list of words that have same hash. We store a gushort (2 bytes) as index for a word.
  2. anagram hash index: Every entry in anagram word list has a corresponding entry in anagram hash index section, where we store the hash (guint — 4 bytes), offset of the the list entry (guint - 4bytes) and the length of the entry (gchar — 1 byte), with the total size of 9 bytes for each index.

Stage 2: At the run time

Here we search for anagrams from the data created in stage 1. The search follows like:

  • Suppose we have a word “BAT”, we sort its letters and hash it, lets call the hash generated as H1 and the length of word i.e. 3 as LE1.
  • Now, search for H1 in the anagram hash index section, this will be a binary search as the section is always sorted, thus will be very fast.
  • Once found, store the offset (the next 4 bytes), called O1 and the length (the next 1 byte), called L1. The offset points to the anagram-indices stored in the angram-word-list section and the length tells us the number of anagrams.
  • Go to O1 and read the next L1 * 2 bytes. Every 2 bytes here are the index of the words that we want.
  • We get the indices, lets say I1 and I2, set a Fragment list of length LE1, and read the words at index I1 and I2. These are the required anagrams.

Thus we have found the anagrams, we can now show them to the user.

What’s done

Code to write data into word list file has been written and pushed into the main repository. The code creates the above discussed anagram-hash-index and anagram-word-list sections.

What’s left to do

We need to read the word-list file in the run time, find the anagrams using the above mentioned approach and display them to the user.

September 13, 2023

Reconstructing an invalid TPM event log

TPMs contain a set of registers ("Platform Configuration Registers", or PCRs) that are used to track what a system boots. Each time a new event is measured, a cryptographic hash representing that event is passed to the TPM. The TPM appends that hash to the existing value in the PCR, hashes that, and stores the final result in the PCR. This means that while the PCR's value depends on the precise sequence and value of the hashes presented to it, the PCR value alone doesn't tell you what those individual events were. Different PCRs are used to store different event types, but there are still more events than there are PCRs so we can't avoid this problem by simply storing each event separately.

This is solved using the event log. The event log is simply a record of each event, stored in RAM. The algorithm the TPM uses to calculate the PCR values is known, so we can reproduce that by simply taking the events from the event log and replaying the series of events that were passed to the TPM. If the final calculated value is the same as the value in the PCR, we know that the event log is accurate, which means we now know the value of each individual event and can make an appropriate judgement regarding its security.

If any value in the event log is invalid, we'll calculate a different PCR value and it won't match. This isn't terribly helpful - we know that at least one entry in the event log doesn't match what was passed to the TPM, but we don't know which entry. That means we can't trust any of the events associated with that PCR. If you're trying to make a security determination based on this, that's going to be a problem.

PCR 7 is used to track information about the secure boot policy on the system. It contains measurements of whether or not secure boot is enabled, and which keys are trusted and untrusted on the system in question. This is extremely helpful if you want to verify that a system booted with secure boot enabled before allowing it to do something security or safety critical. Unfortunately, if the device gives you an event log that doesn't replay correctly for PCR 7, you now have no idea what the security state of the system is.

We ran into that this week. Examination of the event log revealed an additional event other than the expected ones - a measurement accompanied by the string "Boot Guard Measured S-CRTM". Boot Guard is an Intel feature where the CPU verifies the firmware is signed with a trusted key before executing it, and measures information about the firmware in the process. Previously I'd only encountered this as a measurement into PCR 0, which is the PCR used to track information about the firmware itself. But it turns out that at least some versions of Boot Guard also measure information about the Boot Guard policy into PCR 7. The argument for this is that this is effectively part of the secure boot policy - having a measurement of the Boot Guard state tells you whether Boot Guard was enabled, which tells you whether or not the CPU verified a signature on your firmware before running it (as I wrote before, I think Boot Guard has user-hostile default behaviour, and that enforcing this on consumer devices is a bad idea).

But there's a problem here. The event log is created by the firmware, and the Boot Guard measurements occur before the firmware is executed. So how do we get a log that represents them? That one's fairly simple - the firmware simply re-calculates the same measurements that Boot Guard did and creates a log entry after the fact[1]. All good.

Except. What if the firmware screws up the calculation and comes up with a different answer? The entry in the event log will now not match what was sent to the TPM, and replaying will fail. And without knowing what the actual value should be, there's no way to fix this, which means there's no way to verify the contents of PCR 7 and determine whether or not secure boot was enabled.

But there's still a fundamental source of truth - the measurement that was sent to the TPM in the first place. Inspired by Henri Nurmi's work on sniffing Bitlocker encryption keys, I asked a coworker if we could sniff the TPM traffic during boot. The TPM on the board in question uses SPI, a simple bus that can have multiple devices connected to it. In this case the system flash and the TPM are on the same SPI bus, which made things easier. The board had a flash header for external reprogramming of the firmware in the event of failure, and all SPI traffic was visible through that header. Attaching a logic analyser to this header made it simple to generate a record of that. The only problem was that the chip select line on the header was attached to the firmware flash chip, not the TPM. This was worked around by simply telling the analysis software that it should invert the sense of the chip select line, ignoring all traffic that was bound for the flash and paying attention to all other traffic. This worked in this case since the only other device on the bus was the TPM, but would cause problems in the event of multiple devices on the bus all communicating.

With the aid of this analyser plugin, I was able to dump all the TPM traffic and could then search for writes that included the "0182" sequence that corresponds to the command code for a measurement event. This gave me a couple of accesses to the locality 3 registers, which was a strong indication that they were coming from the CPU rather than from the firmware. One was for PCR 0, and one was for PCR 7. This corresponded to the two Boot Guard events that we expected from the event log. The hash in the PCR 0 measurement was the same as the hash in the event log, but the hash in the PCR 7 measurement differed from the hash in the event log. Replacing the event log value with the value actually sent to the TPM resulted in the event log now replaying correctly, supporting the hypothesis that the firmware was failing to correctly reconstruct the event.

What now? The simple thing to do is for us to simply hard code this fixup, but longer term we'd like to figure out how to reconstruct the event so we can calculate the expected value ourselves. Unfortunately there doesn't seem to be any public documentation on this. Sigh.

[1] What stops firmware on a system with no Boot Guard faking those measurements? TPMs have a concept of "localities", effectively different privilege levels. When Boot Guard performs its initial measurement into PCR 0, it does so at locality 3, a locality that's only available to the CPU. This causes PCR 0 to be initialised to a different initial value, affecting the final PCR value. The firmware can't access locality 3, so can't perform an equivalent measurement, so can't fake the value.

comment count unavailable comments

September 12, 2023

Building a NAS

The status quo

Back in 2015, I bought an off-the-shelf NAS, a QNAP TS-453mini, to act as my file store and Plex server. I had previously owned a Synology box, and whilst I liked the Synology OS and experience, the hardware was underwhelming. I loaded up the successor QNAP with four 5TB drives in RAID10, and moved all my files over (after some initial DoA drive issues were handled).

QNAP TS-453mini product photo
QNAP TS-453mini product photo

That thing has been in service for about 8 years now, and it’s been… a mixed bag. It was definitely more powerful than the predecessor system, but it was clear that QNAP’s OS was not up to the same standard as Synology’s – perhaps best exemplified by “HappyGet 2”, the QNAP webapp for downloading videos from streaming services like YouTube, whose icon is a straight rip-off of StarCraft 2. On its own, meaningless – but a bad omen for overall software quality

The logo for QNAP HappyGet 2 and Blizzard's Starcraft 2 side by side
The logo for QNAP HappyGet 2 and Blizzard’s StarCraft 2 side by side

Additionally, the embedded Celeron processor in the NAS turned out to be an issue for some cases. It turns out, when playing back videos with subtitles, most Plex clients do not support subtitles properly – instead they rely on the Plex server doing JIT transcoding to bake the subtitles directly into the video stream. I discovered this with some Blu-Ray rips of Game of Thrones – some episodes would play back fine on my smart TV, but episodes with subtitled Dothraki speech would play at only 2 or 3 frames per second.

The final straw was a ransomware attack, which went through all my data and locked every file below a 60MiB threshold. Practically all my music gone. A substantial collection of downloaded files, all gone. Some of these files had been carried around since my college days – digital rarities, or at least digital detritus I felt a real sense of loss at having to replace. This episode was caused by a ransomware targeting specific vulnerabilities in the QNAP OS, not an error on my part.

So, I decided to start planning a replacement with:

  • A non-garbage OS, whilst still being a NAS-appliance type offering (not an off-the-shelf Linux server distro)
  • Full remote management capabilities
  • A small form factor comparable to off-the-shelf NAS
  • A powerful modern CPU capable of transcoding high resolution video
  • All flash storage, no spinning rust

At the time, no consumer NAS offered everything (The Asustor FS6712X exists now, but didn’t when this project started), so I opted to go for a full DIY rather than an appliance – not the first time I’ve jumped between appliances and DIY for home storage.

Selecting the core of the system

There aren’t many companies which will sell you a small motherboard with IPMI. Supermicro is a bust, so is Tyan. But ASRock Rack, the server division of third-tier motherboard vendor ASRock, delivers. Most of their boards aren’t actually compliant Mini-ITX size, they’re a proprietary “Deep Mini-ITX” with the regular screw holes, but 40mm of extra length (and a commensurately small list of compatible cases). But, thankfully, they do have a tiny selection of boards without the extra size, and I stumbled onto the X570D4I-2T, a board with an AMD AM4 socket and the mature X570 chipset. This board can use any AMD Ryzen chip (before the latest-gen Ryzen 7000 series); has built in dual 10 gigabit ethernet; IPMI; four (laptop-sized) RAM slots with full ECC support; one M.2 slot for NVMe SSD storage; a PCIe 16x slot (generally for graphics cards, but we live in a world of possibilities); and up to 8 SATA drives OR a couple more NVMe SSDs. It’s astonishingly well featured, just a shame it costs about $450 compared to a good consumer-grade Mini ITX AM4 board costing less than half that.

I was so impressed with the offering, in fact, that I crowed about it on Mastodon and ended up securing ASRock another sale, with someone else looking into a very similar project to mine around the same timespan.

The next question was the CPU. An important feature of a system expected to run 24/7 is low power, and AM4 chips can consume as much as 130W under load, out of the box. At the other end, some models can require as little as 35W under load – the OEM-only “GE” suffix chips, which are readily found for import on eBay. In their “PRO” variant, they also support ECC (all non-G Ryzen chips support ECC, but only Pro G chips do). The top of the range 8 core Ryzen 7 PRO 5750GE is prohibitively expensive, but the slightly weaker 6 core Ryzen 5 PRO 5650GE was affordable, and one arrived quickly from Hong Kong. Supplemented with a couple of cheap 16 GiB SODIMM sticks of DDR4 PC-3200 direct from Micron for under $50 a piece, that left only cooling as an unsolved problem to get a bootable test system.

The official support list for the X570D4I-2T only includes two rackmount coolers, both expensive and hard to source. The reason for such a small list is the non standard cooling layout of the board – instead of an AM4 hole pattern with the standard plastic AM4 retaining clips, it has an Intel 115x hole pattern with a non-standard backplate (Intel 115x boards have no backplate, the stock Intel 115x cooler attaches to the holes with push pins). As such every single cooler compatibility list excludes this motherboard. However, the backplate is only secured with a mild glue – with minimal pressure and a plastic prying tool it can be removed, giving compatibility with any 115x cooler (which is basically any CPU cooler for more than a decade). I picked an oversized low profile Thermalright AXP120-X67 hoping that its 120mm fan would cool the nearby MOSFETs and X570 chipset too.

Thermalright AXP120-X67, AMD Ryzen 5 PRO 5650GE, ASRock Rack X570D4I-2T, all assembled and running on a flat surface

Testing up to this point

Using a spare ATX power supply, I had enough of a system built to explore the IPMI and UEFI instances, and run MemTest86 to validate my progress. The memory test ran without a hitch and confirmed the ECC was working, although it also showed that the memory was only running at 2933 MT/s instead of the rated 3200 MT/s (a limit imposed by the motherboard, as higher speeds are considered overclocking). The IPMI interface isn’t the best I’ve ever used by a long shot, but it’s minimum viable and allowed me to configure the basics and boot from media entirely via a Web browser.

Memtest86 showing test progress, taken from IPMI remote control window

One sad discovery, however, which I’ve never seen documented before, on PCIe bifurcation.

With PCI Express, you have a number of “lanes” which are allocated in groups by the motherboard and CPU manufacturer. For Ryzen prior to Ryzen 7000, that’s 16 lanes in one slot for the graphics card; 4 lanes in one M.2 connector for an SSD; then 4 lanes connecting the CPU to the chipset, which can offer whatever it likes for peripherals or extra lanes (bottlenecked by that shared 4x link to the CPU, if it comes down to it).

It’s possible, with motherboard and CPU support, to split PCIe groups up – for example an 8x slot could be split into two 4x slots (eg allowing two NVMe drives in an adapter card – NVME drives these days all use 4x). However with a “Cezanne” Ryzen with integrated graphics, the 16x graphics card slot cannot be split into four 4x slots (ie used for for NVMe drives) – the most bifurcation it allows is 8x4x4x, which is useless in a NAS.

Screenshot of PCIe 16x slot bifurcation options in UEFI settings, taken from IPMI remote control window

As such, I had to abandon any ideas of an all-NVMe NAS I was considering: the 16x slot split into four 4x, combined with two 4x connectors fed by the X570 chipset, to a total of 6 NVMe drives. 7.6TB U.2 enterprise disks are remarkably affordable (cheaper than consumer SATA 8TB drives), but alas, I was locked out by my 5650GE. Thankfully I found out before spending hundreds on a U.2 hot swap bay. The NVMe setup would be nearly 10x as fast as SATA SSDs, but at least the SATA SSD route would still outperform any spinning rust choice on the market (including the fastest 10K RPM SAS drives)

Containing the core

The next step was to pick a case and power supply. A lot of NAS cases require an SFX (rather than ATX) size supply, so I ordered a modular SX500 unit from Silverstone. Even if I ended up with a case requiring ATX, it’s easy to turn an SFX power supply into ATX, and the worst result is you have less space taken up in your case, hardly the worst problem to have.

That said, on to picking a case. There’s only one brand with any cachet making ITX NAS cases, Silverstone. They have three choices in an appropriate size: CS01-HS, CS280, and DS380. The problem is, these cases are all badly designed garbage. Take the CS280 as an example, the case with the most space for a CPU cooler. Here’s how close together the hotswap bay (right) and power supply (left) are:

Internal image of Silverstone CS280 NAS build. Image stolen from ServeTheHome

With actual cables connected, the cable clearance problem is even worse:

Internal image of Silverstone CS280 NAS build. Image stolen from ServeTheHome

Remember, this is the best of the three cases for internal layout, the one with the least restriction on CPU cooler height. And it’s garbage! Total hot garbage! I decided therefore to completely skip the NAS case market, and instead purchase a 5.25″-to-2.5″ hot swap bay adapter from Icy Dock, and put it in an ITX gamer case with a 5.25″ bay. This is no longer a served market – 5.25″ bays are extinct since nobody uses CD/DVD drives anymore. The ones on the market are really new old stock from 2014-2017: The Fractal Design Core 500, Cooler Master Elite 130, and Silverstone SUGO 14. Of the three, the Fractal is the best rated so I opted to get that one – however it seems the global supply of “new old stock” fully dried up in the two weeks between me making a decision and placing an order – leaving only the Silverstone case.

Icy Dock have a selection of 8-bay 2.5″ SATA 5.25″ hot swap chassis choices in their ToughArmor MB998 series. I opted for the ToughArmor MB998IP-B, to reduce cable clutter – it requires only two SFF-8611-to-SF-8643 cables from the motherboard to serve all eight bays, which should make airflow less of a mess. The X570D4I-2T doesn’t have any SATA ports on board, instead it has two SFF-8611 OCuLink ports, each supporting 4 PCI Express lanes OR 4 SATA connectors via a breakout cable. I had hoped to get the ToughArmor MB118VP-B and run six U.2 drives, but as I said, the PCIe bifurcation issue with Ryzen “G” chips meant I wouldn’t be able to run all six bays successfully.

NAS build in Silverstone SUGO 14, mid build, panels removed
Silverstone SUGO 14 from the front, with hot swap bay installed

Actual storage for the storage server

My concept for the system always involved a fast boot/cache drive in the motherboard’s M.2 slot, non-redundant (just backups of the config if the worst were to happen) and separate storage drives somewhere between 3.8 and 8 TB each (somewhere from $200-$350). As a boot drive, I selected the Intel Optane SSD P1600X 58G, available for under $35 and rated for 228 years between failures (or 11,000 complete drive rewrite cycles).

So, on to the big expensive choice: storage drives. I narrowed it down to two contenders: new-old-stock Intel D3-S4510 3.84TB enterprise drives, at about $200, or Samsung 870 QVO 8TB consumer drives, at about $375. I did spend a long time agonizing over the specification differences, the ZFS usage reports, the expected lifetime endurance figures, but in reality, it came down to price – $1600 of expensive drives vs $3200 of even more expensive drives. That’s 27TB of usable capacity in RAID-Z1, or 23TB in RAID-Z2. For comparison, I’m using about 5TB of the old NAS, so that’s a LOT of overhead for expansion.

Storage SSD loaded into hot swap sled

Booting up

Bringing it all together is the OS. I wanted an “appliance” NAS OS rather than self-administering a Linux distribution, and after looking into the surrounding ecosystems, decided on TrueNAS Scale (the beta of the 2023 release, based on Debian 12).

TrueNAS Dashboard screenshot in browser window

I set up RAID-Z1, and with zero tuning (other than enabling auto-TRIM), got the following performance numbers:

IOPSBandwidth
4k random writes19.3k75.6 MiB/s
4k random reads36.1k141 MiB/s
Sequential writes2300 MiB/s
Sequential reads3800 MiB/s
Results using fio parameters suggested by Huawei

And for comparison, the maximum theoretical numbers quoted by Intel for a single drive:

IOPSBandwidth
4k random writes16k?
4k random reads90k?
Sequential writes280 MiB/s
Sequential reads560 MiB/s
Numbers quoted by Intel SSD successors Solidigm.

Finally, the numbers reported on the old NAS with four 7200 RPM hard disks in RAID 10:

IOPSBandwidth
4k random writes4301.7 MiB/s
4k random reads800632 MiB/s
Sequential writes311 MiB/s
Sequential reads566 MiB/s

Performance seems pretty OK. There’s always going to be an overhead to RAID. I’ll settle for the 45x improvement on random writes vs. its predecessor, and 4.5x improvement on random reads. The sequential write numbers are gonna be impacted by the size of the ZFS cache (50% of RAM, so 16 GiB), but the rest should be a reasonable indication of true performance.

It took me a little while to fully understand the TrueNAS permissions model, but I finally got Plex configured to access data from the same place as my SMB shares, which have anonymous read-only access or authenticated write access for myself and my wife, working fine via both Linux and Windows.

And… that’s it! I built a NAS. I intend to add some fans and more RAM, but that’s the build. Total spent: about $3000, which sounds like an unreasonable amount, but it’s actually less than a comparable Synology DiskStation DS1823xs+ which has 4 cores instead of 6, first-generation AMD Zen instead of Zen 3, 8 GiB RAM instead of 32 GiB, no hardware-accelerated video transcoding, etc. And it would have been a whole lot less fun!

The final system, powered up

(Also posted on PCPartPicker)

September 09, 2023

A logo for CapyPDF

The two most important things about any software project are its logo and mascot. Here is a proposal for both for CapyPDF.

As you can probably tell I'm not a professional artist, but you gotta start somewhere. The original idea was to have a capybara head which is wearing the PDF logo much like a bow tie around its ear. The gist of it should come across, though it did look much better inside my brain. The PDF squiggle logo is hard to mold to the desired shape.

The font is Nimbus Sans, which is one of the original PostScript Core Fonts. More precisely it is a freely licensed metrically compatible version of Helvetica. This combines open source with the history of PDF quite nicely.

September 08, 2023

Cambalache 0.14.0 Released!

I am pleased to announce a new Cambalache version.

Cambalache is a new RAD tool for Gtk 4 and 3 with a clear MVC design and data model first philosophy.

Version 0.14.0 brings two new features one of them not even originally supported by Glade.

Release Notes:

    • Add GMenu support
    • Add UI requirements edit support
    • Add Swedish translation. Anders Jonsson
    • Updated Italian translation. Lorenzo Capalbo
    • Show deprecated and not available warnings for Objects, properties and signals
    • Output minimum required library version instead of latest one
    • Fix output for templates with inline object properties
    • Various optimizations and bug fixes
    • Bump test coverage to 66%

Menu Support

The <menu> tag in GtkBuilder is the builtin support for GMenuModel and GMenu classes defined in GIO.

All you need to do is create a “(menu)” model object…

and add items, sections or submenus to it.

People familiar with the GObject type system might have noticed that “(menu)” is not a valid GType name this is intentional and used for built in features like <menu> tag or defining external objects ( “(external)” ) this allows me to model everything in a generic way and have custom code to export this in the right non generic format.

UI Requirements

Just like Glade, Cambalache now let you choose which library version you want to target.

A list of all libraries used in the UI file. will be available in the requires tab of the object editor.

Here you will be able to select which target version you want to use in the UI file and the application will warn you if the object, property or signal is not available in the version you selected.

You can also leave it blank and Cambalache will automatically use the minimum library required for your UI.

Where to get it?

As always you can get the code in gitlab

git clone https://gitlab.gnome.org/jpu/cambalache.git

or download from flathub

flatpak remote-add --user --if-not-exists flathub https://flathub.org/repo/flathub.flatpakrepo
flatpak install --user flathub ar.xjuan.Cambalache

Happy coding!

September 07, 2023

Improvements to my helper tool for VM-based openQA testing

It’s two years since I started looking into end-to-end testing of GNOME using openQA. While developing the end-to-end tests I find myself running tests locally on my machine a lot, and the experience was fiddly, so I wrote a simple helper tool named ssam_openqa to automate my workflow.

Having chosen to write ssam_openqa in Rust, it’s now really fun to hack on, and I somewhat gratuitously gave it an interactive frontend using the indicatif Rust library.

Here’s what it looks like to run the GNOME OS end-to-end tests with --frontend=interactive in the newly released 1.1.0rc2 release (video):

You can pause the tests while running by pressing CTRL+C, or using the --pause-test or --pause-event to pause on certain events. This lets you open a VNC viewer and access the VM itself, which makes debugging test failures much nicer.

I’m giving a couple more talks about end-to-end testing with openQA this year. This month at OSSEU 2023 in Bilbao, I’m filling in for my colleague James Thomas, talking about openQA in automotive projects. And in October, at XDC 2023 in A Coruña, I’ll be speaking about using openQA as a way to do end-to-end testing of graphical apps. See you there!

September 06, 2023

GSoC 2023: Final Report

alt

Hello to everyone.

So this is the final report on the work I completed throughout the Google Summer of Code contribution period (May-September). There's a lot to share and discuss, but I'll try to keep this brief.


Mentors:

Sonny Piers and Andy Holmes

Project:

Make GNOME Platform demos for Workbench


About Workbench:

alt

Workbench lets you experiment with GNOME technologies, no matter if tinkering for the first time or building and testing a GTK user interface.

Among other things, Workbench comes with

  • Live GTK/CSS preview

  • Library of 100+ examples

  • JavaScript, Rust and Vala support

  • Declarative user interface syntax

  • Autosave, sessions and projects

  • Code linter and formatter

  • Offline documentation

  • Terminal output

  • 1000+ icons


Tasks performed:

Created beginner-friendly and easy-to-understand examples/demos for all widgets of GTK 4.10 and Libadwaita 1.3 to help newcomers understand how to use them effectively.

Provided ready-to-use code snippets of the widgets/APIs covered, making it easier for developers to integrate them into their projects.

Covered GLib/GIO and Libportal APIs and create relevant examples to help developers understand how to use them in their applications

Created demos while taking UI and UX design concepts into account to showcase how to make aesthetically pleasing and functional user interfaces.

Covered GNOME HIG Patterns to ensure that the examples and demos follow the GNOME Human Interface Guidelines, making them consistent with other GNOME applications and user-friendly for users

Implemented Search function in Workbench


Further Tasks to be performed:

  • Port and integrate the existing implemented search function from a demo as a feature into Workbench.

  • Adding new demos with the forthcoming GTK and Adwaita releases.

  • Update existing demos as GJS, GIO, and GLib versions are updated and new features are added.


My Contributions:

I was amazed and thrilled to learn that I had written about 3800 lines of code in the previous few months. I have contributed the following:

CodeFind Feature:

Apart from the demo coverage I also implemented a codefind feature as a library entry in the workbench.

  • Implemented a code-search feature similar to GNOME Text Editor

  • Highlights matched search results

  • Supports both case and non-case-sensitive search

  • Implemented move to next search and move to previous search functions

  • Keybindings for ease of access

Currently, this feature is implemented as a library entry in Workbench, which is further planned to be integrated shortly.

Going Beyond:

I was thrilled to have the opportunity to contribute to GJS (GNOME Javascript) because we completed what we had planned a week or two before the last week of the coding period, and my mentor allowed me to concentrate on something apart from the project.

  • Made console.log format GError correctly

    Before:

      gjs> try {imports.gi.Gio.File.new_for_path('/foo').load_contents(null) } catch (err) { console.error(err) }
      (gjs:68852): Gjs-Console-CRITICAL **: 21:22:41.833: {
          "stack": "@typein:7:47\n@<stdin>:1:42\n",
          "fileName": "typein",
          "lineNumber": 7,
          "columnNumber": 47
      }
    

    After:

      gjs> try {imports.gi.Gio.File.new_for_path('/foo').load_contents(null) } catch (err) { console.error(err) }
      (gjs:68852): Gjs-Console-CRITICAL **: 21:22:41.833: Gio.IOErrorEnum: Error opening file /foo: No such file or directory
      @typein:9:47
      @<stdin>:1:42
    
  • Improved console.log output

    Before:

      gjs> console.log(new TextDecoder())
      Gjs-Console-Message: 21:38:35.467: {
          "encoding": "utf-8",
          "ignoreBOM": false,
          "fatal": false
      }
    

    After:

      gjs> console.log(new TextDecoder())
      Gjs-Console-Message: 21:38:35.467: TextDecoder {
          "encoding": "utf-8",
          "ignoreBOM": false,
          "fatal": false
      }
    

More information about this issue can be found here


Post GSoC plans:

  • Dive deeper into GNOME technologies and contribute more, particularly in GJS (GNOME Javascript).

  • Continuing exploration of other Open-Source Communities.

  • Creating more complex projects using what I learned here.

  • Engaging with newcomers more regularly to assist them in breaking into open-source software development


Hardships during GSoC:

  • Keeping up with coding and schoolwork (yes, I attended normal, full-time classes for the majority of the coding period)

  • Keeping up with the time zone differences (my mentors and I are in completely different time zones)

  • Proactively learning and adjusting to new tasks. (Each week, we were given a new topic to demonstrate, ensuring continual and adaptable learning.)


My learnings:

  • Working as a team to resolve issues.

  • Communicating with team members ensuring no blockers

  • Iterative Development

  • Writing clean code

  • Remote software development practices

  • Documentation and technical writing skills

  • Work Discipline

  • Personal growth


Achievement:

alt

I am thrilled to share that Workbench now has over 100+ interactive demos!


THANK YOU!

GNOME , my mentors Sonny Piers and Andy Holmes, my teammates, and everyone who supported me during this period. This summer has been a roller coaster journey for me, from coding to documentation to an international sponsored trip; it's been a blast. I'm looking forward to contributing more actively and learning to the best of my ability.


Help us make GNOME Calendar rock-solid by expanding the test suite!

GNOME Calendar 45 will be a groundbreaking release in terms of UX (more on that later?), performance, and to some extent, reliability (we’ve at least solved two complex crashers recently, including a submarine Cthulhu crasher heisenbug and its offspring)… and yet, I think this might be “just the beginning” of a new era. And a beginning… is a very delicate time.

If you’ve tried to use GNOME Calendar in the past decade or so, you’ve certainly encountered one of the many bugs related to timezones, daylight saving time, and all of that crazy stuff that would make Tom Scott curl up in a corner and cry. But it doesn’t have to be that way, and in fact, there is a way for anyone who knows a bit of C programming to help us build the tools to solve this mission-critical problem.

Today, I’d like to urge you to help in writing some automated tests.
The sooner our test suite can grow, the faster we can make GNOME Calendar rock-solid.

The "Anime girl hiding from Terminator" meme, with the Terminator labelled "The automated tests we need" and the anime girl labelled "The current GNOME Calendar codebase"

As I explain in this epic blocker ticket:

[…] There currently are some unit tests in GNOME Calendar, but not nearly enough, and we need to cover a lot more in order to fix the pile of timezone-related issues […]

We really need to use test-driven development here, otherwise these bugs will be a nightmare to fix and to keep fixed forever. Before we start fixing those bugs, we need as many unit tests as possible to cover spec compliance.

By helping write unit tests to ensure GNOME Calendar complies with standard calendaring specifications, your contribution will make a huge impact in GNOME Calendar’s future reliability, as an improved test suite will make it 100x easier for us to fix existing bugs while preventing introducing new bugs.

Doing this will also help us more confidently expand GNOME Calendar’s featureset to handle meetings planning functionality. […]

Why is this suddenly mission-critical now?

I believe a boosted test suite is now becoming mission-critical, because in my logical ordering of the dependency chain for this new roadmap I devised in 2023, I have determined that timezones/DST/midnight backend+views reliability blocks almost everything left: you can’t expand the UI and UX featureset to encompass meetings & timezones management unless you have a rock-solid, reliable backend and views/representation that you can trust to begin with (why create nice UIs if the users won’t trust the resulting events/data/views?)

So we need to fix those ugly issues once and for all. And there’s pretty much only one way to do this kind of daunting bugfixing work efficiently and sustainably: with test-driven development.

Once we have those tests, I trust that we will be able to make the app pretty much bulletproof.

Having extensive automated test coverage will allow us to confidently/safely fix some of the first “fundamental” compliance bugs there, and I can bet that by doing so, we would incidently solve tons of other issues at once—I wouldn’t be surprised if this would allow us to solve 50 issues or more. This will not only make a ton of existing users happy (and make GNOME Calendar viable for a ton more new users), but also make contributors happy because we will be able to resolve and close a ton of tickets cluttering our view. It’s all part of a flywheel of sustainability here.

“How hard can it be?”

While writing such tests might not be easy to do for a “beginner programmer who does not know the C language”, it is a good “newcomer contributor” task for someone who already knows C programming and are looking to make a lasting quality impact on a popular FLOSS desktop productivity application that many people love.

Maybe you, or a friend or colleague, would be interested by this challenge (while existing GNOME Calendar contributors keep fixing bugs at the same time)? This blog post is retootable if you want to spread the word there, too.

Ready to help us improve reliability?

If the above sounds like an interesting way to help the project, read the detailed ticket for more details, join us on the cozy “#gnome-calendar:gnome.orgMatrix channel (mapped to the #gnome-calendar IRC channel on irc.libera.chat) and get started!

September 04, 2023

GUADEC 2023 in Riga, Latvia

My experience at

Me presenting at the conference
GNOME Intern Lightning Talks

This summer I traveled to Riga to attend my first international conference, Gnome Users And Developers European Conference (GUADEC) 2023 and true to its promise, the experience that unfolded was nothing short of amazing.

Well, it began with me missing the opening ceremony because I got to the wrong venue (happens in a new city), and then took a cab to the correct venue. With a cup of coffee in one hand and an apple in the other, i entered the venue, got my ID card and started attending the talks.

The venue
The correct venue

The talks were great, were on various topics, most about technology and while some others about society and GNOME’s role in it. My favourite talk for the day was about GNOME design. They showed some very interesting things like GNOME mobile shell, new tilling, new activities button etc.

After the talks, it was time to explore the city, and what better way to do so than talking a walk. Went to old town Riga, passing by the most beautiful Daugava River which flows through the centre of the city.

River view

The days that followed were even more exciting which involved meeting new people, sharing experiences and knowledge, with each interaction unveiling fresh perspectives about the history as well the current working of GNOME.

I don’t want to make this blog too long to read.

Here’s a mandatory pic with the standee.

September 03, 2023

Crosswords 0.3.11: Acrostic Panels

Long time, no release.

When I last blogged about GNOME Crosswords, I had a design plan to improve the editing API. It’s been a busy summer since then. The crosswords team rewrote large chunks of code to implement and use this new API:

crosswords: 146 files changed, 10545 insertions(+), 4915 deletions(-)
libipuz:  53 files changed, 8224 insertions(+), 961 deletions(-)

There’s now over 38KLOC between the two codebases — this is starting to look like a real application!

Editor and libpanel

The biggest change this cycle is the implementation of the new editing interface. I started changing the code five months ago, but it took a while to land. We now use libpanel from GNOME Builder to manage the information panels. Libpanel has a lot of the functionality I want, though unfortunately I’m fighting its geometry handling.

New grid editor panel
New grid editor panel
New clues editor panel
New clues editor panel

I really struggled getting the UI design for this to work, and I had a number of regrettable paths along the way. Fortunately, Niko agreed to help out with this, and showed up with some fabulous design work! I’m so much happier with the current approach, and am getting ready to implement more of his designs for the next cycle.

Behind the scenes, implementing this was a challenge. I blogged about those challenges previously, but in a nutshell, mutating a puzzle has a lot of side-effects which can leave you in an invalid state. For example, something simple like adding a block could completely change the numbering of the grid.

Federico and I fixed this by adding a number of functions to enforce heuristics and conventions. I also added ipuz_puzzle_fix_all() which will  get a fully well-formed puzzle regardless of the state. It’s turned out to be a really nice design pattern.

I have now released the Editor as a separate application in flathub. You can download it here.

Acrostics

Another major feature this release is Acrostic Puzzle support. Tanmay worked on that for his GSOC project and did fabulously (details in his blog post.) The end result is gorgeous:

Animation of an Acrostic puzzle being solved
Animation of an Acrostic puzzle being solved

I had a great time mentoring Tanmay for the summer, and we’re already making plans on how to add acrostic support to the Editor.

GUADEC and GNOME Mobile

GUADEC in Latvia was a blast. Riga was fun, the countryside was surprisingly interesting, and the overall feel of the conference was lovely. As always, it was great to meet so many people, new and old.

Stone statue looking like a goomba
A wild Goomba in Riga

GNOME Superstar Martin came to my Crosswords BOF in Riga and got it running on an actual mobile device. The end result was.. actually usable! It seems like all the work we did on adaptive sizing paid dividends. There are some rough corners (and Martin filed a number of bugs) but it worked surprisingly well out of the box.

I also got to meet with my other GSOC student Pratham. His work on anagrams will land next release, so we’ll have more to talk about then.

libipuz.org

The final thing mentioning is that I bought the libipuz.org domain for the library. Unfortunately the main ipuz.org site has been down all summer, I put a mirror of the ipuz spec up there so people can read it.

I also managed to reach Roy Leban by phone. He’s the original author of the ipuz spec, and was able to clarify some questions I had when interpreting it. This led me down a deep rabbit hole around the part of the spec regarding clue directions, as it required a substantial rewrite to avoid hardcoding Across/Down directions everywhere.

Finally, Pranjal showed up completely out of nowhere with an MR to add ipuz_puzzle_equal(). This was a tricky function to write, and one I’ve wanted for a really long time. This has been making all our tests so much better. Hero! Pranjal is interested in adding a sudoku loader/saver to libipuz — maybe there will be more in this space in the future.

Thanks

  • Niko, for massive help with the designs for the Editor
  • Tanmay, for Acrostic support
  • Martin, for testing Crosswords on mobile
  • Pranjal, for ipuz_puzzle_equal()
  • Federico, for testing fixes and overall support
  • Rosanna, for continued advice and crossword support
  • Pratham, for initial anagram support
  • Bart, for whatever magic he did to make libipuz.org work
  • The translators, for keeping us multi-lingual

Until next time!

Mentoring in Open Source

This year, I was invited by Sonny Piers to be a co-mentor for the GNOME Foundation, working on platform demos for Workbench. I already contribute a lot of entry-level documentation and help a lot of contributors, so this felt like a good step in a direction I've been heading for a while.

# Internships

Together, Sonny and I mentored three interns; two by way of Google Summer of Code (GSoC) and a third for Outreachy. Both are international programs providing a great opportunity for newcomers, but differ in some important ways.

# Google Summer of Code

Google Summer of Code has been around for a very long time, although I've been around long enough to remember when it started, and how it helped change the public image of open source. Google's program is well developed, while at the same time being mostly hands-off, leaving most of the planning and direction to the mentors.

Something that did influence our choice of applicants was the requirement that interns be open source beginners, which I feel doesn't account for those who could benefit from more advanced mentoring. Of course, we had plenty of applicants and I have no regrets about our selection.

# Outreachy

Outreachy is a newer program and one that actually started as an initiative of the GNOME Foundation. It is focused on those that are underrepresented in the industry, and fostering a positive feedback loop to address this ongoing issue.

In contrast to Google's program, Outreachy is not limited to students or developers, which is important when you consider the barriers to education and lack of opportunity its applicants often face. The program also includes prompts and goals, both for mentors and mentees, that help make these internships more engaging.

The infrastructure for Outreachy is kind of unassuming at first, compared to Google's sleek and streamlined website. Once you're a few weeks into the program though, you realize they really are a lot more focused on genuine mentorship. Nothing about your interaction with the coordinators is impersonal or superficial.

Marina Zhurakhinskaya deserves special mention here, and even if you haven't heard the name before, you have heard the names of those whose lives she touched. Although I never had the honour of working with her myself, it's impossible to ignore how we all continue to benefit from her contributions.

# People

I should first thank Sonny for the opportunity to participate as a mentor for the GNOME Foundation. This is not something I've been able to do in a formal way since high school when I was allowed extra classes to assist younger students in the electronics program.

We've all been in the position where you really just need someone to answer the question, "Okay, but how does it actually work?". Being a part of the "Oh, now I get it!" moments is an unparalleled experience for me. I'm really grateful I had the chance to be involved in this way again.

# Akshay Warrier

Akshay is going to do great things in open source and I think he really gets community-driven software. He's one of those people that renews your excitement for open source.

Aside from the contributions he's made to Workbench as part of his internship, I was really excited to see him appear at our yearly GNOME Shell Extensions workshop. I know he had a fantastic time at GUADEC and the way he talked about it made it obvious how much importance he places on people.

I think we'll see more of Akshay and, one day, I think he could make a really great mentor himself.

# José Hunter

José comes up with some great ideas, and he's quick to jump in and get them implemented. Given the right circumstances, I think we might see him do some really cool things in the community.

He's blogged about privacy and the encroach of corporate interests, and it's hard not to think this encouraged his involvement in open source. I think he has a natural impulse to form his own opinions about the technology he uses and employs, and an honest interest in projects related to his hobbies like libmanette.

I hope José sticks around after his internship is complete, because he has a personality that has served the community well in the past.

# Sriyansh Shivam

Sriyansh displayed an aptitude for development early on, but also a habit of responding to feedback in a really constructive and professional way. That's a hard thing to do sometimes, and he really has the drive for self-improvement.

We spent a fair amount of time working through a series of demos about the model-view-controller pattern together, and I really enjoyed that. These demos covered everything from GListModel and GtkListBox, to the newer view widgets like GtkColumnView. These are very popular in GTK4 applications, and often quite complex, so I'm quite proud to see the results of his hard work.

I'm not sure we really had a chance to see his full potential, and I hope he gets the opportunity to take on a longer more challenging project.

# The Future

I hope to have the time and opportunity to mentor again, and next time I would like to apply myself in a more thoughtful way. I write a lot of entry-level documentation for free software, but the feedback is far more asynchronous and has to be applied more statistically.

Having spent time with several mentees over an extended period of time was a good way to learn a lot. Sonny and I had some good conversations about what seemed to work in retrospect, which was very enlightening. I would really like to try mentoring again, but next time with a more developed strategy.

I will say that co-mentoring is definitely something we should encourage more as a community. It's difficult to be confident of your take on an interpersonal relationship, and having someone to balance that turned out to be invaluable to the mentorship.

September 02, 2023

GUADEC 2023 Experience

Sveiki visiem!

In this blog, I'm pumped to share my experience attending GUADEC 2023 held in Riga, Latvia.

Let's start :)

During the conference, I presented a joint talk with Pooja Patel on "How to add 16.67% more users and contributors: A guide to creating accessible applications".

The talk was on Day 2 of the conference and we got quite nervous haha. We didn't join the walking tour and had to skip some of the most amazing talks on Day 2.

Fortunately, The journey was made more streamlined due to the extensive support of the staff team. I also want to thank Melissa for doing all the bookings and keeping up with our issues xD

The conference was not easy to attend in any way. Me and Asmit (One of the organisers and friends) decided pretty early on that we would volunteer for the GUADEC Organising team. Unfortunately, a lot of stuff happened and I got late. Then I refrained from joining conferences for some time, but my friends convinced me in the end :D And then another downturn came when my exams got severely delayed and clashed completely with the conference. I wanted to volunteer there but it crashed all the plans.

After severe planning and exploring options, I decided to skip 2 exams and attend the conference either way, hence I had to take the remaining exams the next day on reaching India. Although I will have to give the skipped ones again, I don't regret it even a bit. But it made the trip somewhat hectic as I had the tension of remaining exams throughout the trip which took a toll on my health :(

Anyway, let's proceed with the blog :D

University of Latvia

The touchdown

After reaching Helsinki (Layover for flight), I met the GSoC'23 interns of GNOME. It rejuvenated the memories when we were one and took our first flight. It was amazing to interact with them and share our love for open source.

We then took a Flight to Riga. The sights from the plane were mesmerising, to say the least.

On reaching the hotel all my friends surprised me, it was great to catch up with them and meet them again after GNOME Asia. They are the best ;)

Unfortunately, I didn't arrive on time for the introductory party but well

Latvia Riga airport

The first day of summit

The first day was quite good, got to attend the talks which I was looking forward to like - "Accessibility update: the Python stack, Rust, and DBus" by Federico, "Oxidizing GTK" by Emmanuele (The memes like the Elon one were awesome xD), "Community building and best DEI practices in Free and Open Source Communities" by Anisa was great, I had the pleasure of meeting her during GNOME Asia and it was the same energy and an amazing talk as expected.

During the break, I met many people like the Ubuntu staff (Mauro you were a delight to converse with). Met Matthias and Rosanna again, awesome as always and finally met Melissa :D.

Then I attended "GNOME Design: State of the Union" which was awesome as I got a sneak peek into how GNOME is evolving and saw those beautiful designs. It amazes me how much GNOME has progressed.

After that, we attended the "How GNOME Gets into Ubuntu" and "Keynote: All the little things that keep open source going" which following the overall themes, were again great.

Day 1 random pics 1

Day 1 random pics 2

The second day of summit

On the second day, I mostly attended online due to our talk. We practised it, tried to remove our nervousness and got ready.

And the moment finally came. Anisa and Caroline introduced and motivated us :) In the end, it went quite smoothly, There were things which could have been executed in a better manner, but in the end, what can't.

Aryan and Pooja talk at GUADEC 2023

Then we attended the Lightning talks, and man they were so fun. Probably one of the best moments of the conference. There I got to learn Melissa's awesome stage-handling skills LOL. It was just indescribable.

I also conversed with Regina during the break and it was awesome to discuss various topics. She had some great viewpoints and insights regarding GNOME mentorship and contributor involvement.

After the talks, we explored the city a bit and enjoyed the mesmerising architecture and culture of Latvia.

The third day of summit

Attending "Building Student Communities to Foster OSS" by Hrittik was quite a good experience, he then handed out some swag which I'm using right now to plan the blog haha, so thanks for that.

Then the AGM and group photo session came. The group photo was funny and amazing, I wish that was recorded as well lol.

GNOME Group picture

I certainly enjoyed the "Our logo sucks" Lightning talk just after the "pants of thanks". You need a certain level of courage to bash the logo in front of its creator at a conference organised by the same organisation lol. Jokes aside, it was quite on point and I can relate with most of the stuff mentioned. Even if it is not implemented, I hope it stirs a discussion at least.

GNOMELogo sucks lightning talk

After which we continued to attend other talks and enjoyed the conference end.

Then we attended the GNOME Dinner, it was great! I met Rudolf and we conversed quite a lot. He was awesome and full of energy. I also met Georges Stavracas (feaneron), the person who killed the meme (iykyk) Unfortunately due to my introvertness I wasn't able to converse much with them but yeah, was great to finally meet. There were many more people there whom I wanted to ping but dinner is probably not a good time to do so lol unless sitting together.

GNOME Dinner selfie

I also had a great time talking with Felipe, a person I hoped to meet for quite a long.

Felipe and Aryan selfie

The BoFs

Being a GNOME GSoC'22 Intern, I was looking forward to the "GSoC + Outreachy internships with GNOME" BoF by Felipe, we discussed various points which we hope to get implemented to enhance the contributor experience even more and strengthen our community more.

Felipe GSoC BoF

The Nothern Riga tour

The tour was a great opportunity to check out Riga and interact with people.

GNOME India Group photo

Met Emmanuele there and finally decided to speak to him. He was as awesome as I imagined, Again introvertness killed me but yeah, it is always great to meet people you look up to.

Emmanuele and Aryan selfie

Meeting people

At last, I met many new people and got to learn a lot. Made new friends, got to meet people I look up to and many more. Overall it was amazing. I hope I wasn't that introvert but yeah, slowly getting comfortable around new people :)

The End

Thanks to all the people for making the event so great. It was an experience like no other. I would also like to thank GNOME Foundation for sponsoring the trip :)

I also got accepted as a GNOME Foundation member just recently, which is awesome :D.

Sponsored by gnome badge

Extensions in GNOME 45

By now it is probably no longer news to many: GNOME Shell moved from GJS’ own custom imports system to standard JavaScript modules (ESM).

Imports? ESM?

JavaScript originated in web browsers to add a bit of interactivity to otherwise static pages. There was no need to split up small code snippets into multiple files, so the language did not provide a mechanism for that.

This did become an issue when people started writing bigger programs in JavaScript, so environments like node.js and GJS added their own import systems to organize code into multiple files. As a consequence, developers and tooling had a hard time transitioning from one environment to another.

That changed in 2015 when ECMAScript 6 standardized modules, resulting in a well-defined, widely-supported syntax supported by all major JavaScript engines. GJS has supported ESModules since 2021, but porting GNOME Shell was a much bigger task that had to be done all at once.

So? Why should I care?

Well, there is a teeny tiny drawback: Modules and legacy imports are incompatible in practice.

Modules are loaded differently than scripts, and some statements — namely import and export — are only valid in modules. That means that trying to import a module with the legacy system will result in a syntax error if the module uses one of those statements (about as likely as a pope being Catholic).

Modules also hide anything to the outside that isn’t explicitly exported. So while it is technically possible to import a script as module, it is about as useful as importing an empty file.

What does this mean for extensions?

Extensions that target older GNOME versions will not work in GNOME 45. Likewise, extensions that are adapted to work with GNOME 45 will not work in older versions.

You can still support more than one GNOME version, but you will have to upload different versions to extensions.gnome.org for pre- and post-45 support.

There is a porting guide with detailed information. The two most important changes (that will be enough for many extensions!) are:

  1. Use standard syntax to import modules from gnome-shell:

    import * as Main from 'resource:///org/gnome/shell/ui/main.js';
    
    Main.notify('Loaded!');
    
  2. Export a default class with enable() and disable() methods from your extension.js.

    You may want to extend the new Extension class that replaces the convenience API from the old ExtensionUtils module.

    import {Extension, gettext as _} from 'resource:///org/gnome/shell/extensions/extension.js';
    
    export default class MyTestExtension extends Extension {
        enable() {
            console.log(_('%s is now enabled').format(this.uuid));
        }
    
        disable() {
            console.log(_('%s is now disabled.').format(this.uuid));
        }
    }
    

Last but not least, you can always find friendly people on Matrix and Discourse who will be happy to help with any porting issues.

Summary
  • Moving from the custom import system from GJS to the industry standard ECMAScript 6 will cause every extension to break. The move though will mean we are following proper standards and not home grown ones allowing greater compatibility with JavaScript ecosystem.
  • Legacy imports are still supported on extensions.gnome.org but you will need to upload a pre and post GNOME 45 support in order support both LTS and regular distribtuions.

For GNOME Extension Developers:
There is a active extension community who can help you port to the new import system at Matrix and Discourse who can help you quickly port to the new version.

You can test your extensions by downloading the latest GNOME OS and trying your extension there.

To the GNOME Community:
Please file bugs with your favorite extensions or have a friendly conversation with your extension writers so that we can help minimize the impact of this change. Ideally, you could help with the port and provide a pull or merge request to help maintainers.

Resources

Dev Log August 2023

Didn't really do anything on Niepce in August, so no updates on that front. However I worked on a few other things.

libopenraw

A lot of work done on it. See my other wlog post. The short version is that I should look towards releasing 0.4.0 sometime soon.

Since that last post, I have added a long standing to-do item: code to generate test cases so I can run a non regression test suite on existing files. However I get hit with flaws in the various XML serde crates.

AbiWord

Did some work on AbiWord. Nothing ground breaking, but I have a goal to try to release the next 3.0.x with one or two bug fixes (UI), and then what I call 3.next (likely 3.2) that will focus on internal changes (stability) and UI fixes (anything back-portable to 3.0.x will get back-ported).

Done

  • Some code cleanup.
  • Removed the use of boost in many places where the standard C++ library would work.
  • Use C++17 more.
  • Making the build system less recursive. It's less complex and should be more parallel.
  • Removed some Gtk3 construct that are deprecated in provision of a future Gtk4 port.
  • More removal of the legacy containers.
  • Rewrote the toolbars to use plain buttons and a GtkBox instead of GtkToolItem. The look is slightly different.
  • Fixing some Gtk3 bug: the table creation widget was not behaving properly. Will be on stable too (3.0.x)
  • Fixing some bugs with styling. (also on 3.0.x)

In progress

I am introducing Gtkmm into the code base.. This can be done progressively, but it will make Gtk work so much easier. The initial bootstrap involve using it for the GtkApplication object. This ensure it's properly initialized and everything can trickle down from there.

Porting to Gtk4.

I try to anticipate some of the changes by trying to do some of the porting on Gtk3.

Porting the menus to GMenu as the first step. It almost work, but the main problem that the menu machinery doesn't poll the GMenuModel nor the actions. Before that we created the GtkMenuItem directly, and connected to the map and unmap signal to refresh the item. I look into reproducing this with GMenu based code. It not that easy. The API lack: I can't get the action or the GMenuItem from a GtkMenuItem (or vice-versa) and I currently rely on the GtkMenuBar structure. This will break on Gtk4 but it least it is more limited.

The goal is to have it work the same way for 3.next.

GOffice

AbiWord uses libgoffice. I am working on fixing some bugs that happen to hit both Gnumeric and AbiWord (Gtk3): in the GOComboBox the popup isn't set transient to the GtkWindow which is a no-no on Wayland, so it's never positioned right. But then fixing this cause a conflict with tool-tip: if the user click on the control while the tool-tip is showing, it will fail. And GtkComboxBox just use some internal API for that:

_gtk_tooltip_hide (GTK_WIDGET (combo_box))

So I haven't figured out yet how to hide the tool-tip.

See draft merge request 10.

Future

Past 3.next, that will have the UI almost unchanged, I intend to think of a slight redesign. The challenge is to find the proper balance between improving the UI, not getting users lost, and how easy we can do it. It's a lot of work, but the current UI design is still stuck in the early 2000s as a "clone" of Microsoft Word™, and has become a bit messy over time.

August 31, 2023

Fedora Silverblue Development Utils

In the previous post we explored all the different ways to develop software on Fedora Silverblue when Toolbx and Flatpak are not enough. Some of the ideas there are interesting, some are dead-ends and some are extremely useful.

I’ve extracted all the useful parts to a small script I’m calling silverblue-devel-utils.

$ silverblue-devel-utils 
Utils for developing on a Silverblue system

Syntax:
silverblue-devel-utils make_mutable [--temporary]
silverblue-devel-utils unmake_mutable
silverblue-devel-utils cleanup_mutable
silverblue-devel-utils rebase_image <IMAGE>

Let’s go over the functionality: make_mutable makes /usr writable and installs dnf. By default the mutable changes persist until unmake_mutable is called and the machine is rebooted. If you want the changes to automatically revert on the next boot you can use make_mutable --temporary. When you have booted into the previous state, you can call cleanup_mutable do get rid of the deployment containing your changes.

Last but not least, rebase_image can rebase your Silverblue install to any image in your local repository. This can be useful if you plan on keeping your changes for a long time, if creating all changes as rpms is hard or you just want to make very specific changes. Using a Dockerfile with the Silverblue OCI image as a base is the easiest way to create such images.


Let’s see how we can actually use the script! First of all, let’s install it:

curl -o ~/.local/bin/silverblue-devel-utils https://gist.githubusercontent.com/swick/968453815a1e918f0f0f56faff310a80/raw/53db4dcfa2480c1f34ede507add533ac801973e5/silverblue-devel-utils && chmod +x ~/.local/bin/silverblue-devel-utils

Now, let’s try temporarily making our system mutable:

$ silverblue-devel-utils make_mutable --temporary
$ sudo dnf install -y gdb
$ sudo dnf debuginfo-install -y mutter gnome-shell
$ # do some serious debugging
$ dbus-run-session gdb --args gnome-shell --wayland --nested
$ systemctl reboot
...
$ # and we're back on our old system!

Let’s do the same, but this time with the changes persisting across boots:

$ silverblue-devel-utils make_mutable
$ sudo dnf builddep -y mutter
$ sudo dnf install -y libei-devel libeis-devel
$ meson setup mybuild --prefix=/usr
$ ninja -C mybuild/
$ sudo ninja -C mybuild/ install
$ # our new mutter is here!
$ mutter --version
mutter 45.beta.1
$ systemctl reboot
...
$ mutter --version
mutter 45.beta.1
$ # still the version we build!
$ silverblue-devel-utils unmake_mutable
$ systemctl reboot
...
$ silverblue-devel-utils cleanup_mutable
$ # our old version again
$ mutter --version
mutter 44.3

And finally, let’s build our own OS image based on Silverblue and boot into it:

ARG IMAGE_NAME="${IMAGE_NAME:-silverblue}"
ARG SOURCE_IMAGE="${SOURCE_IMAGE:-silverblue}"
ARG BASE_IMAGE="quay.io/fedora-ostree-desktops/${SOURCE_IMAGE}"
ARG FEDORA_MAJOR_VERSION="${FEDORA_MAJOR_VERSION:-38}"

FROM ${BASE_IMAGE}:${FEDORA_MAJOR_VERSION} AS base

ARG IMAGE_NAME="${IMAGE_NAME}"
ARG FEDORA_MAJOR_VERSION="${FEDORA_MAJOR_VERSION}"

# setup dnf
RUN rpm-ostree install -y dnf
RUN dnf install -y 'dnf-command(builddep)'

# install development packages
RUN dnf groupinstall -y 'Development Tools'
RUN ln -s /usr/bin/ld.gold /usr/bin/ld # ???
RUN dnf install -y meson strace gdb valgrind sysprof

# clean and commit
RUN rm -rf /tmp/* /var/*
RUN ostree container commit
RUN mkdir -p /var/tmp && chmod -R 1777 /var/tmp
$ podman build -f Dockerfile -t my-very-own-os-image
$ silverblue-devel-utils rebase_image my-very-own-os-image
$ systemctl reboot
...
$ # our OS has strace installed!
$ strace test

The system will stay on this image until another one is installed with silverblue-devel-utils rebase_image (every now and then, you might want to podman pull the base image and rebuild your own image to receive updates) or you manually rebase to another origin with rpm-ostree rebase.

Thanks to Jordan Petridis for his great blog post on systemd-sysext which made me explore all the available tools! Thanks to Ivan Molodetskikh and Tomáš Popela for the feedback on the previous post!

I hope you find this tool useful!

August 30, 2023

Viewing Images in GNOME: Loupe and Glycin

Loupe is GNOME’s new Core app for viewing images. Starting with the GNOME 45 release, you might find it as Image Viewer on your system. It replaces the previous image-viewing app Eye of GNOME. In honor of this historic occasion, I wanted to give a bit of insight into the making and technology of Loupe.

Why Start from Scratch

The first documented commits to Eye of GNOME (EOG) are from September 1999 by Federico. Some of this code from back then withstands the test of time until today. Likewise, the image loading was already powered by GdkPixbuf, which is still GNOME’s image loading library today. So why start replacing such a well-tried set of software now?

App window with a minimalist headerbar on top and an image of the Carina nebula below.
Loupe provides more than just a refreshed user interface.

Original experiments for writing a new image viewer go back to 2020. There wasn’t a complete technical plan for Loupe, but rather a gut instinct with everyone involved that for the goals of Loupe, a rewrite would be the better solution. In retrospect, this was the right decision. With the port to GTK 4, a new interface design, gestures and animations, sandboxed image loading, and using more upstream projects for decoding tasks, the code overlap wouldn’t have been too large to begin with. On top, the Rust language is a good fit for handling media data since it provides memory safety and performance at the same time.

Hidden Features

On the surface, Loupe looks simple. And that’s what we want it to feel like. Behind this, however, are a lot of features that make this possible. Here are a few examples that highlight the differences to EOG. The image rendering is now happening on the graphics card. That allows viewing quite large images while retaining a responsive interface, including animations and direct visual feedback for touchpad and touchscreen gestures. It also makes Loupe much more snappy on high-resolution screens, especially noticeable with higher refresh rates. To keep image navigation for SVGs smooth as well, Loupe uses “tiled rendering,” only rendering relevant parts of the SVG and providing those tiles to the graphics card for combining them. This allows Loupe to render large SVGs while keeping the interface responsive. Loupe also supports High DPI screens better in the way that a 100 % image size is oriented on the physical amount of pixels. For transparent images, Loupe analyzes the image’s content and chooses a different background color to not end up drawing black content on a black background and alike.

Dialog with layout options to the left an a print preview to the right showing a Pika.
Not all features new to Loupe are as apparent as the print layout dialog.

Last but not least, I wrote a new wrapper around the image decoding process. The project is named Glycin after a photographic developing agent. Glycin sandboxes the image loading, which potentially¹ adds security and allows a flawed image loader to crash without pulling Loupe down with it.² Additionally, most decoders Glycin uses are written in Rust, adding an additional line of defense via memory safety.

Rust Doesn’t Solve Everything

Duh! I know. But sometimes I feel like there is a tendency that one might forget that besides all the struggle with programming languages and build systems, programming in itself also remains hard. No matter the tools one uses. While the codebase at hand is still relatively small – about 8000 lines of Rust code – there were a bunch of logic and design challenges that required a day without brain fog³. One example is the combination of many input methods and animations and their potential interactions and always getting them represented correctly on the screen.

Another big part of this project was, as so often, working with upstream projects that make all the magic possible in the first place. The GTK maintainers took on the battle with OpenGL to add mipmapping support for better texture rendering. I even dipped my toes into GTK’s texture code myself to add support for gray-scale textures, apart from some smaller additions. We also discussed future GTK APIs for HDR images and ICC profiles in a call. Libadwaita also got some features needed for Loupe. The image decoding crates got a bunch of commits to add support for ICC color management profiles. For the libheif bindings, the maintainer took care of most of the needed changes.

I learned a bunch of new things during the development. About GTK internals, custom widgets, the dreaded OpenGL, image formats, color spaces, memfd, and more.

What will Break

Now, with replacing and changing so many bits an pieces, what will break? First of all, there are some limitations when using Loupe via Flatpak. This, however, was already the case with EOG. Using hardware acceleration also means that your graphics drivers have to work. In some rare occasions, like with the PinePhone (non-pro), this seems to be an issue. Switching to software rendering in those cases is possible, however. The tiled rendering code is not working as well as I would like it to. This can, for example, result in SVGs being rendered slightly blurry while using a pinch gesture to zoom. There can also be a short delay in animations for very large images or on older hardware. I did not have the time to properly address the issue for this release.

We are also replacing many of the image decoders with new ones. There are cases where the new image loading has better support for formats, and there are images that can’t be opened at all but work in EOG. The reason is mostly that many image standards have parts that have rarely been used in practice, and therefore, almost no image decoder implementation is 100 % complete. I’m sure that we will find some formats that are used in practice, and support has to be added in future versions. Maybe the fax encoding standards for TIFF?

We are shipping Loupe in its current state as a Core component since we are convinced that the benefits will outweigh the minor regressions we might hit. We also had the chance to confirm this through the feedback we got over the complete development and incubation period.

Closing Thoughts

Most of Loupe has been written over a timespan of nine months, with me also having to move flats during this time. None of the people who directly contributed to Loupe were paid for their work. On one hand, it is promising that we can pull such a project off as a community effort. On the other hand, considering how many companies will rely on such GNOME Core components, the balance does look off to me.

Big thanks go out to everyone who helped make this project happen: Chris, who started the whole project. Allan, for all the design work. Alice, Ivan, Zander, for a huge amount of help in all areas, especially touch gestures, and lower-level plumbing, and … even more gestures! The contributors to all the countless projects Loupe depends on. And everyone who helps with testing and is reporting issues.

The development of Loupe hasn’t been without friction. With the release of GNOME 45, I will be stepping down from maintaining Loupe to not stand in the way of the original ideas of how to run this project.


¹ While decoder processes don’t have write access to files and no network connection, I still consider the hardening of the sandbox in its infancy.

² For those who are interested in the technical details: Glycin spawns one sandboxed process per image file via bwrap or flatpak-spawn. The communication happens via peer-to-peer D-Bus over UNIX socket. The file data is read from a GFile and sent to the sandbox via a separate UNIX socket. The texture data is provided from the sandbox via a memfd that is sealed afterward and given as an mmap to GTK. For animations and SVGs the sandboxed process is kept alive for new frames or tiles as long as needed. For an introduction to this structure, see Christian Hergert, “Designing Multi-Process Application Security,” GUADEC 2019.

³ Developing software with chronic illness and disability would probably be worth a blog post on its own.

August 29, 2023

rpmlint updates (August 2023)

We are at the end of the summer and this means that this year Google Summer of code is ending.

The recent changes applied now in the main branch include:

  • Remove usage of pkg_resource because it's deprecated.
  • Fix elf binary check with ELF files with a prefix.
  • New check for python packages with multiple .pyc files for different python versions.
  • Improve the testing framework (merged the work done during the GSoC 2023)

Summer of Code 2023 updates

The summer of code is ending and the work done by Afrid was good enough to be merged, so I merged it the past week.

I'm really happy with the work done during the GSoC program, now we've a more simple way to define tests for rpmlint checks mocking the rpm, so it's not always needed to build a fake rpm binary for each new test. This will make a lot easier to create simple tests, so I hope that we can increase the code coverage using this new framework.

During this time Afrid has extended the FakePkg class, so it's possible now to define fake metadata and files with fake tags and attributes. It's not complete and it's not a simple task to replace all the rpm binaries used for tests, because the Pkg class and RPM tags is a complex thing, but the current state allow us to replace a lot of them. Afrid has replaced some of the tests that uses binaries, but in the following months we can continue working on this and replace more.

After this work, we can now start to use more the FakePkg class in tests, so another task that we can do is to provide some common fake pkgs to use in different tests and new checks, so now it's possible to create fake packages with dynamic random data, so we can extend tests with fuzz testing and maybe this will help to improve the tool reliability.

Conclusion

I've participated as mentor several times now in the summer of code, and outreachy, and almost always was a good experience. With the gnome foundation in previous programs and this year with opensuse. These two communities are very open to collaboration and makes the whole process really simple, for me as mentor, and also for the intern.

I want to congratulate Afrid, because it was nice to work with him during this summer, he has done a great work, not just technically, but communicating, asking and finding his own solutions without requiring a continuous guidance.

He is very passionate and looks like a nice person, so I hope that he will continue around the open source, it could be opensuse, rpmlint or any other community, but this kind of people is what you want to find in any community.

After many years collaborating with different free software communities, it's amazing that there are so many great people in every project, of course you can find toxic communities and people, but in my experience, that's usually just noise, there are a lot of nice people out there, doing a great work, and I'm happy that young people like Afrid can be part of the free software movement, because this is what makes the free software great, the people that is working on it.

So Thanks a lot to Google for another summer of code, thanks to SUSE for letting me, and encourage me, to mentor, and thanks to all the free software developers that are out there.

I encourage everyone to participate in this kind of programs, for interns, it's a good opportunity to learn and to make some money working on free software, for mentors it's an opportunity to get some help in your project and help newcomers to be part of the community.

Have a lot of fun!

Portfolio 1.0.0

I am happy to announce the release of Portfolio 1.0.0! This new release is the first step in the modernization process to GTK 4 and Libadwaita. It’s also a continuation to my efforts of bringing a minimalist file manager to the mobile Linux community, with a few important bug fixes.

As a starting point for the modernization process, this new version of Portfolio preserves the exact same design, in a GTK 4 flavor. A few reasons for that.

First, although I wish everyone was distributing applications using Flatpak, I want to reduce friction for the mobile Linux distributions, by sticking to currently available APIs. Second, I want to spend more time experimenting with newer Libadwaita widgets, specially with the ones from the upcoming 1.4 release, as these could require redesigning a few aspects of Portfolio’s graphical interface.

On the maintenance side of things, I fixed a major issue which prevented some files to be sent to the trash and another that prevented displaying progress while moving files, among other minor fixes. Porting widgets to Libadwaita fixed one annoying issue which prevented swapping back, so that’s a big free win.

Even before all of this work ever happened, I spent some time refactoring the project into smaller, more modular, pieces. This helped significantly to the porting process and it will ensure an easier maintenance move forward. So expect more refactors to come.

Unix sockets, Cygwin, SSH agents, and sadness

Work involves supporting Windows (there's a lot of specialised hardware design software that's only supported under Windows, so this isn't really avoidable), but also involves git, so I've been working on extending our support for hardware-backed SSH certificates to Windows and trying to glue that into git. In theory this doesn't sound like a hard problem, but in practice oh good heavens.

Git for Windows is built on top of msys2, which in turn is built on top of Cygwin. This is an astonishing artifact that allows you to build roughly unmodified POSIXish code on top of Windows, despite the terrible impedance mismatches inherent in this. One is that until 2017, Windows had no native support for Unix sockets. That's kind of a big deal for compatibility purposes, so Cygwin worked around it. It's, uh, kind of awful. If you're not a Cygwin/msys app but you want to implement a socket they can communicate with, you need to implement this undocumented protocol yourself. This isn't impossible, but ugh.

But going to all this trouble helps you avoid another problem! The Microsoft version of OpenSSH ships an SSH agent that doesn't use Unix sockets, but uses a named pipe instead. So if you want to communicate between Cygwinish OpenSSH (as is shipped with git for Windows) and the SSH agent shipped with Windows, you need something that bridges between those. The state of the art seems to be to use npiperelay with socat, but if you're already writing something that implements the Cygwin socket protocol you can just use npipe to talk to the shipped ssh-agent and then export your own socket interface.

And, amazingly, this all works? I've managed to hack together an SSH agent (using Go's SSH agent implementation) that can satisfy hardware backed queries itself, but forward things on to the Windows agent for compatibility with other tooling. Now I just need to figure out how to plumb it through to WSL. Sigh.

comment count unavailable comments

Misconceptions About Immutable Distributions

What Is an Immutable Distribution?

“Immutable” is a fancy way of saying unchangeable. So, an immutable distribution is essentially an unchangeable distribution, i.e. read-only. However, in most cases, the entirety of the distribution is not immutable. Some directories, like /tmp are writable, as some directories need write capabilities to actually function. Some distributions may even have other directories, such as /etc, writable for extra functionality.

To redefine, an immutable distribution is when, at a minimum, only the core of the distribution is read-only, and the rest is read-write.

Note: The more accurate terms are “reprovisionable” and “anti-hysteresis”. However, since “immutable” is more commonly used, I’ll use it instead throughout this article. Further reading: “Immutable” → reprovisionable, anti-hysteresis by Colin Walters.

Technical Definitions

There are a few technical definitions that should be understood before reading the rest of the article:

  • Rollback (rolling back): The process of undoing a change or update. For example, you upgrade your system to the latest version, which introduces bugs. You can then revert that update and go back to the version that worked well.
  • Reproducibility: The process of delivering identical software regardless of the time or system used. Suppose you install a distribution on hundreds of laptops and desktops over the course of a week. The distribution you install on the first desktop (on the first day) will be exactly the same as the last laptop (on the last day). That is, all systems will be identical. You can think of it as a perfect clone.
  • Atomicity (atomic updates): The process of ensuring that a change (update, downgrade, etc.) is either fully applied or aborted. For example, your system shuts down in the middle of an upgrade due to a power outage. Instead of having a half-updated system, the update is fully aborted, meaning that your system was unchanged, just like before you started the update.

Concept Versus Implementations

It’s really easy to confuse the concept and the implementation when discussing immutability.

As we noted above, an immutable distribution is when the core of the distribution is read-only — that’s it. You (or someone else) decide to implement that concept as you see fit. You could have a locked down and inflexible system, like GNOME OS,1 where there isn’t even a system package manager; or you could have something that is designed to be flexible, like NixOS, where you can change the desktop environment, the kernel, the boot loader, and so on.

You can think of an immutable distribution as any traditional distribution. Some traditional distributions are locked down and inflexible, such as Fedora Workstation, where removing or even disabling SELinux can cause problems. Fedora Workstation also prevents you from accidentally removing core packages like sudo. Meanwhile, some traditional distributions are designed to be flexible, such as Gentoo Linux, where you can change almost anything in your system.

Misconceptions

There are several misconceptions both for and against immutable distributions that I would like to address.

I’m going to use pseudo quotes, which means I will be quoting and refining arguments I’ve interpreted from debates and media.

“Needing to Reboot”

Due to the read-only nature of immutable distributions, applying changes (updates, installs, etc.) absolutely requires a reboot.

The need of reboots varies per implementation. With Endless OS, you must reboot to apply updates. On Fedora Silverblue, you can run sudo rpm-ostree apply-live in the terminal to apply changes while the system is booted. On NixOS, you don’t need to reboot at all.

Even then, rebooting your system after updating it is really important to keep it running optimally. As the article Restarting and Offline Updates explains: “The Linux Kernel can change files without restarting, but the services or application using that file don’t have the same luxury. If a file being used by an application changes while the application is running then the application won’t know about the change. This can cause the application to no longer work the same way. As such, the adage that “Linux doesn’t need to restart to update” is a discredited meme. All Linux distributions should restart.

Many immutable distributions focus on reboots when performing system-level updates, because in many cases a reboot is crucial for stability. That’s why Android, iOS, macOS, Windows, ChromeOS, etc. require a reboot.

“Difficult to Use”

Since immutable distributions are locked down, inflexible, and often need workarounds for various issues, they’re more difficult to use.

Difficulty varies per implementation and users’ priorities. For me, Fedora Silverblue is really easy to use, because it provides a great experience by default;2 I want something that is reliable, gets out of my way as much as possible, and follows closest with my political views (I value free software, inclusiveness in the community, and progressing the Linux desktop), which Fedora Silverblue is one of the few that provides.3 However, NixOS isn’t for me, because I’m not a power user.

Endless OS is used by many educational institutions, because these institutions typically need a system that is simple, accessible and secure. On the contrary, NixOS is a powerful system typically used by power users and organizations that absolutely need the features it offers.

However, it’s worth noting that, no matter the distribution, immutable or not, workarounds may be needed. Nothing is perfect, but there are many cases when external developers refuse to collaborate. For example, many drivers are unavailable in the Linux kernel, so we’re sometimes forced to inconveniently install these drivers manually. While a distribution like Arch Linux has the AUR, most other traditional distributions do not have the luxury of installing obscure drivers. In short, this isn’t a fundamental problem with immutable distributions; it’s a problem that you have with every distribution.

“Inflexible by Nature”

Due to the read-only nature of immutable distributions, users cannot tinker with their system.

Again, this heavily depends on the implementation. If we look at traditional distributions, Ubuntu is not designed to be tinkered with, nor is Fedora Linux. As such, these distributions are prone to breaking after a significant amount of modification.

Meanwhile, NixOS, an immutable distribution, empowers the user and allows them to change many aspects of their system. Don’t like GRUB? Use systemd-boot, rEFInd or something else. You want to add a Mandatory Access Control? Use SELinux. Don’t like SELinux? use AppArmor or something else. Don’t like using GNOME? Use Plasma, XFCE, MATE, Cinnamon, Sway or any desktop environment/window manager. Don’t like the fact that you’re using an LTS version of NixOS? Just switch to an unstable or even a bleeding edge channel. What’s that, you want to go back to LTS? You can. Want to mix and match packages from different channels? Go for it… You could view NixOS as Arch Linux on steroids.

Really, some traditional and immutable distributions are locked down and inflexible, meanwhile some others are flexible. This heavily varies per implementation.

“Immutable Distributions Are More Secure”

Immutable distributions provide better security, because […]

I truly believe that statements like these create a false sense of security because “security” can be interpreted differently by different people. Allow me to identify the meanings of “security” and address each of them:

“Security as a Safety Net”

You can roll back or prevent damage in case an error occurs. For example, your system shuts down in the middle of an update due to a power outage. However, thanks to the system’s atomicity, nothing was changed on the system, thereby allowing you to boot back into the system without a problem.

Atomicity could be replicated on traditional distributions if someone put in the effort.

If the system updates but runs into problems, you can roll back from your bootloader.

Rollbacks can be implemented on traditional distributions. Linux Mint and openSUSE Tumbleweed/Leap are some implementors of this feature.

“Security as a Measure to Prevent Malicious Actors From Compromising Your System”

Thanks to the read-only nature of immutable distributions, they prevent malicious actors from doing more damage than traditional distributions.

While immutable distributions might prevent malicious actors from doing certain things on your system, the bigger problem is that your personal data is at risk. Whether the actor can use sudo is less of an issue than the fact that your GPG keys, personal media and documents, or other sensitive data are easily accessible once compromised.

Whether your system is immutable or not, there are still really important steps you should be taking to make your system more secure. For example, you can use LUKS2 to encrypt your drive(s) to protect your data from physical access, and you can use Flatpak (with Flatseal) to make sure that your data is better protected from malicious/compromised apps. These steps are highly beneficial and should not be taken lightly!

“Security as a Measure to Prevent Users From Breaking Their Own Systems”

Thanks to the read-only nature of immutable distributions, even the most destructive actions, like sudo rm -rf --no-preserve-root /* won’t destroy the system.

While running such commands might prevent total destruction, they can still do enough damage to make the benefit meaningless. In this example, running sudo rm -rf --no-preserve-root /* will delete your $HOME (user) directory if you wait long enough, since the user directory is mutable; depending on the implementation, it could delete your boot partition as well. These destructive commands are usually destructive in other ways on immutable distributions, as the vulnerable bit is the mutable places. Only the immutable parts of the systems are unaffected, but the rest are just as vulnerable as any vulnerable place in traditional distributions.

Immutability Is an Implementation Detail

Immutable and traditional distributions can literally share the same benefits and drawbacks, because, in reality, whether the core of the distribution is read-only or read-write is a minor detail. An immutable distribution could be as unstable as Manjaro and rely on distribution packages, meanwhile a traditional distribution could be as stable as Endless OS and fully embrace containers, such as Flatpak. A traditional distribution could be as locked down and inflexible as GNOME OS, meanwhile an immutable distribution could be as flexible as Gentoo Linux (theoretically).

Immutable distributions are different from each other, the same way traditional distributions are different from each other as well. For example, Vanilla OS utilizes ABRoot, whose concept is inspired from Android’s A/B system updates. Fedora Silverblue and Endless OS utilize OSTree, which is similar to git in paradigm. MicroOS utilizes btrfs’s features. NixOS does, uhh… yes.

While immutable and traditional distributions are presented as different in categories, they have very little difference in nature. It’s best to treat immutable distributions like traditional distributions, rather than a whole different paradigm. It’s up to the maintainers to decide what they want to do with their distribution.

Immutability Relieves Maintenance Burden

Okay, as explored above, if the “benefits” are not benefits that are exclusive to immutable distributions and can be applied to traditional distributions, then why do immutable distributions exist in the first place? Why not just implement atomicity, reproducibility, and rollback functionalities to a traditional distribution?

Simply put, immutable distributions make it much easier for maintainers to focus on their high-level goals, because immutability eases the burden of maintenance.

The benefits are indirect for users, but direct for maintainers. Maintainers have fewer things to worry about than in traditional distributions, which allows them to spend their precious time and resources on the things they actually want to worry about, such as stability, reproducibility, providing a safety net, and so on. As a result, users get a more robust system.

This is why immutable distributions are generally considered “safer” than traditional distributions, even though the concept is not. So far, maintainers have put more effort into making their immutable distribution robust, as opposed to traditional distributions, which usually have to worry about the whole mutable aspect. This benefits immutable distributions greatly, because they don’t have to worry about solving the problems that traditional distributions typically run into.

Similarly, many developers prefer to officially support Flatpak over traditional packaging formats, because they only have to worry about one format as opposed to hundreds. As a result, they have more time and energy to focus on achieving their real goals, instead of focusing on tedious repetitive tasks and supporting a wider range of combinations (packaging format, dependencies version and patches, etc.).

All of these features (reproducibility, atomicity, and rollback) can also be implemented in traditional distributions. However, these features are typically treated as first-class features in immutable distributions, even though they are not mandated, whereas they are second-class features in most traditional distributions that support them. Immutability greatly reduces the maintenance overhead and leaves more room to improve the things maintainers want to improve. Immutable distributions have recently become popular, and are already used and popular in educational and personal systems.

Conclusion

To summarize, an immutable distribution is when the core of the distribution is read-only. Anyone is free to grab that concept and implement a distribution with it. The “benefits” and “inconveniences” that we tend to see with immutable distributions could be applied on traditional distributions, but immutable distributions tend to be easier to accomplish the high-level goals, because they are easier to maintain.

In the end, immutability should be seen as an implementation detail and philosophical/political decision, rather than a purely technical one, because traditional and immutable distributions can literally share the same benefits and drawbacks.

Footnotes

  1. GNOME OS is used for development purposes, so it doesn’t need a package manager. 

  2. Except its downsides

  3. I’m currently looking into Vanilla OS and think about switching to it once 2.0 is released

August 28, 2023

An update on GNOME Settings

There’s no question that GNOME Settings is important to the overall GNOME experience and I feel flattered to share the responsibility of being one of its maintainers. I have been involved with Settings for almost a decade now but only in the last few months I have  started to wear the general maintainer hat “officially”.

That’s why I am compelled to update our community on the current state of the project. Settings is also co-maintained by Robert Ancell who has been doing great work with reviews and also helping us improve our code readability/quality.

The last general update from Settings you might have heard of was Georges’ Maintainership of GNOME Settings Discourse post. Some of what’s written there still holds true: Settings is one of the largest modules in GNOME, and being this hub connecting the Shell, the settings daemons, network manager, portals, cups, etc… it needs more maintainers. It needs especially maintainers with domain expertise. We have a handful of active contributors doing great frontend/UI work,  but we lack active contributors with expertise in the deep dungeons of networking or color management, for example.

To tackle this issue, one of my goals is to improve the developer experience in GNOME Settings to attract new contributors and to enable drive-by contributors to post changes without struggling much with the process. For that, I kickstarted our Developer documentation. It is in an early stage now and welcoming contributions.

I also have been invested in fixing some of our historical UI consistency problems. A lot has been done in the gnome-44 and gnome-45 cycles to adopt the latest design patterns from the GNOME Human Interface Guidelines with libadwaita and modern GTK. Alice Mikhaylenko and Christopher Davis did an outstanding job with the ports to modern Adwaita navigation widgets. We also gained a new “About” panel that can condense more information that is useful especially for debugging/supporting issues. There’s still work to be done on this front especially with certain views that are currently looking a bit out of place in comparison to modern views.

Screenshot of the new "About" panel.

The new Privacy hub is a new “hub” panel introduced by Marco Melorio in gnome-45 that is our initial step towards reducing the overall number of panels.Screenshot of the new "Privacy" panel.For GNOME 46 we want to introduce a new “System” hub panel, developed by our Google Summer of Code intern Gotam Gorabh, as well as introduce a new “Network & Internet” panel that is being already worked on by contributor Inam Ul Haq. These are two epics that involve reworking some complicated panels such as the Wifi/Network and User Accounts ones. These are panels that should also see a big frontend rework in the gnome-46 cycle and that I plan to work on myself.

Also a big thank you to Allan Day, Jakub Steiner, Tobias Bernard, Sam Hewitt, and other folks doing outstanding design and UX work for Settings.

GNOME 45.0 (stable) will be released in September, shipping plenty of new stuff and bugfixes. It would be extremely helpful if you could test the latest changes and report issues and regressions in our issue tracker. GNOME Settings 45.rc has been released and should be available soon in GNOME OS and unstable/development distro releases such as Fedora Rawhide.

If you want to get involved, feel free to join our Matrix chat channel and ask questions there. I also monitor the “settings” Discourse tag, where you can ask support questions and suggest features.

GSoC 2023: Rust and GTK 4 Bustle Rewrite (Final Overview)

Over the summer, for 12 weeks, I worked on rewriting Bustle in Rust and GTK 4 as part of the Google Summer of Code (GSoC) 2023 program. This post is an overview of the work done and the future plans for the project.

About Bustle

Bustle is a graphical application used to analyze D-Bus activities. It uses sequence diagrams to present signal emissions and method calls messages.

Old Bustle

Bustle represents these messages using a row-based layout. Each row is dedicated to displaying the following information: the elapsed time since the first message, the message path, and the member, which could be the name of the emitted signal or invoked method. On the other hand, individual columns correspond to distinct D-Bus services. Bustle draws arrows that transverse these columns to visualize the communication between services, while arcs represent method invocations and returns. This visual paradigm is valuable as, for instance, it can be used to see which services a D-Bus application talks to and how often, which can be handy in debugging, enhancing security, and optimizing performance within the application.

Project Goal

The ultimate goal of this project is to port Bustle to GTK 4 and rewrite it in Rust. Although the current implementation of Bustle in Haskell and GTK 3 is functional, there are compelling reasons to consider a rewrite in Rust. This includes enabling the tool to take advantage of a range of ergonomic libraries, including zbus, gtk4-rs, and pcap-file, that would ease the burden in maintenance. Furthermore, the growing Rust community and the availability of the Rust SDK in Flathub would make the tool more accessible to potential contributors and simpler to distribute to users.

Porting the tool to GTK 4, on the other hand, would offer several benefits, such as access to newer and more performant widgets and drawing APIs like GtkListView and GskPath. This would allow Bustle to benefit from the latest developments in the platform and remain current with evolving standards.

Altogether, a rewrite of Bustle in Rust and GTK 4 would provide advantages that can enhance the tool's functionality, maintainability, longevity, accessibility, and possibly efficiency.

Work Done

Most of the tasks in the original proposal have been completed. This includes having an initial MVP, porting GDBus usage to zbus, implementing file loading and saving, completing the recording functionality, and porting the UI, which comprises the diagram, details view, and services filter, to GTK 4.

For more details on the work done, there are also bi-weekly updates on my blog.

Snapshots

Diagram and Details View

Diagram and Details View

Recording and Services Filter

Recording and Services Filter

Code Merged

These are the summary of code that has been merged during the GSoC period.

Bustle

Most of the work done is in the Bustle GNOME GitLab repository, where the following pull requests have been merged:

zbus and gtk4-rs

While the project is focused on Bustle, some changes are necessary to be upstreamed to other projects. This includes changes to zbus and gtk4-rs, where the following pull requests have been merged:

zbus

gtk4-rs

Code to be Merged

Due to time constraints and unexpected issues, some of the pull requests are still pending review:

Future Plans

While most of the tasks in the proposal have been completed, there are still a few things that need to be done, including the following:

  1. Optimizing performance
  2. Hunting and squashing bugs:
    • Improving name owner changed signal handling
    • Drawing method call arc regardless if the row is not drawable
    • Properly killing the dbus-monitor process
  3. Adding more features:
    • Adding a button that scrolls to the method call message of a method return message or vice-versa
    • Adding a way to open multiple diagrams at once via tabs and multiple windows
  4. Fixing regressions and releasing the application on Flathub
  5. Continuous involvement and contribution to open-source

Final Words

I have to say this is, so far, the most challenging part of my software development journey. It felt like a hackathon since I had to understand things and create something that works quickly. While it was challenging, it was also rewarding as I was able to learn a lot of things and create something that I am proud of. There were moments of time pressure and frustration, but with experimentation, collaboration, and a lot of reading, I was able to overcome these challenges and make meaningful progress. The complexity of the project pushed me out of my comfort zone, forcing me to delve into unfamiliar areas of programming and technology.

Lastly, I would like to express my gratitude to my mentors, Bilal Elmoussaoui and Maximiliano Sandoval, for tirelessly reviewing my pull requests and guiding me. I would also like to thank the GNOME Foundation and the community, especially the GNOME GSoC admin, Felipe Borges, for giving me the opportunity to work on this project. I would also like to extend my appreciation to zbus maintainer, Zeeshan Ali Khan, for their help in getting my pull requests merged against zbus and to the original Bustle developer, Will Thompson, for their guidance. Finally, I would like to thank my family and friends for their support and encouragement.

August 27, 2023

GSoC 2023 Final Project Report

Project Title

Make GNOME Platform demos and create offline documentation viewer for Workbench

Mentors

Sonny Piers, Andy Holmes

About The Project

Workbench is an application that lets you experiment and tinker with GNOME technologies. It’s aimed at beginners who want to get into GTK development or developers who want to prototype a feature for their apps.

Workbench comes with a “Library” which is a collection of demos, made to showcase the widgets, APIs, and design patterns of GTK. The demos are made in such a way that they are easy to understand for beginners and highlight only commonly used parts of a widget or an API. The demos can be a good reference for developers who simply want to look up the usage of a widget without going through piles of documentation and directly copy code snippets from the demos.

The main goal of my GSoC project was to create as many demos as possible for Workbench. Since there were a total of 3 students working on the project (2 GSoC and 1 Outreachy), we later decided that our goal was to try and reach 100 demos in the Library. We focused on making demos in GNOME JavaScript and they can be later ported to other languages.

I also worked on creating a Documentation Viewer for Workbench.

The Demos

I made 31 demos for Workbench during the entire GSoC period. The demos cover widgets and APIs from various GNOME libraries — Gtk, Adwaita, Gio, Xdp, and Shumate. I have listed all the demos with links to the PRs and the widget/API they cover below.

I would love to talk about each of these demos but to keep it short, here are some notable ones for me.

Avatar

While this was a very simple demo, it was one of my first contributions. I learned a lot about GTK and Blueprint from this one simple demo. The “Image…” button lets the user select an image locally using GtkFileDialog which was new at the time. It makes use of the Gio._promisfy pattern to present the dialog asynchronously.

Box

GtkBox is a very commonly used widget, it’s a container that arranges its children into a single row or column. It can be hard for beginners to understand or visualize a box, especially with the various alignment properties. We decided to create an interactive demo for this so that users can play around with the widget and learn.

Web View

WebView provides an easy way to load HTML pages and websites in your apps. Making this demo I learned a lot about WebView, and trying to showcase it in a neat way (both in code and UI) was a little challenging but fun.

Animation

AdwAnimation is a simple API that lets you create timed and spring animations. This demo turned out challenging because I had some trouble working with Adw.CallbackAnimationTarget which is used to animate a custom property by updating it using a callback function. I managed to figure it out in the end, and the result was very satisfying.

Map

libshumate is a library that is used to display and render maps. I had never worked with it before so it did take me some time to learn. It was one of the first demos that covered a library outside of org.gnome.Platform.

Actions and Menus

GAction is a very useful API that’s used throughout GTK and GMenu is a way to visually present Actions to the user. Actions can also appear as different types and it was important the demo made the distinction between those. Since Menus and Actions are very closely connected, it was hard to separate those out into two different demos. I brainstormed some ideas with Andy and I am happy with the way the demo turned out. Andy also updated the GJS Guide to talk about Actions and Menus as a way to complement the demo.

Flow Box

GtkFlowBox is a container that arranges its child widgets into a grid, and when the container is resized it’ll readjust the number of columns of the grid. We got a little creative with this demo ;)

Network Monitor

GNetworkMonitor is a simple Gio API that monitors your network status. This demo was quite tricky to make because it was hard to test the demo and we had to find a way to inform the user on how to do that. We came up with the idea of including instructions in the demo and hiding them away in a popover.

After continuous 3 months of effort, I am very happy to say they we hit our combined goal of reaching 100 demos in the Library 🎉

https://medium.com/media/a1016d86527c097764ac39311d897bf9/href

Documentation Viewer

Halfway through GSoC, Sonny gave us an opportunity to take a break from making demos and work on some features for Workbench. I decided to work on providing offline documentation for Workbench. This can be useful because jumping back and forth between Workbench and documentation can be annoying sometimes and there may be times when you don’t have access to a good internet connection.

To get the documentation, it made sense to make use of the SDK that GNOME provides for documentation which is org.gnome.Sdk.Docs. The first iteration of the Documentation Viewer was very simple, it showed all the available namespaces of docs, and clicking on any item on the sidebar loaded the main page of the documentation in a WebView. Since the sidebar was just a flat list for now and only had a few items, we decided to use a ListBox to display the items.

This was a great start but it wasn’t very useful because it was difficult to navigate and browse through the docs.

Add Documentation Viewer by AkshayWarrier · Pull Request #358 · sonnyp/Workbench

Then there were some follow-ups that made minor improvements to the Documentation Viewer, for example, filtering out irrelevant and outdated docs that weren’t needed and including some docs that were initially missing from the sidebar.

After this, I made a massive improvement to the Documentation Viewer in a follow-up PR.

Manuals: Improve design and use TreeExpander by AkshayWarrier · Pull Request #457 · sonnyp/Workbench

  • I ported the Documentation Viewer to GNOME 45 and improved the design by using the new libadwaita 1.4 widgets such as the NavigationSplitView for the sidebar.
  • I switched the ListBox for a ListView and made use of a GtkTreeListModel and GtkTreeExpander to show all the documentation under a given namespace as different sections using collapsible expanders.
  • We also decided to rename the Documentation Viewer as “Manuals”.

Finally, I worked on a basic searching feature inside Manuals to be able to search for specific documentation. While the search is not very good, it’s good enough for now. I’m relying on a GtkFilterListModel to filter out all the docs, it simply checks if the search term is a substring in any of the items or not. I also refactored a lot of code in this PR, to simplify the loading of docs and cleaned up code.

Manuals: Add searching of docs by AkshayWarrier · Pull Request #521 · sonnyp/Workbench

There is still work left to be done for Manuals such as optimizing the loading of documentation, implementing better search, and synchronizing the item selected in the sidebar and the doc shown in the webview. We are also planning to make this into a standalone app in the future.

Acknowledgments

I would like to thank my mentors, Sonny Piers and Andy Holmes, for their constant guidance and support throughout the project.

I would also like to thank GNOME for sponsoring my first overseas trip to attend GUADEC 2023.

August 26, 2023

Acrostic support added to GNOME Crosswords

It has been quite a while since the last update, and we’ve accomplished a lot since then. I’ve also completed finish my GSoC project, which adds Acrostic puzzles support to GNOME Crosswords.

After implementing cell selection for ClueGrid, I added other signals ie. guess and other command actions.

The next step was to finalize the layout and Implement the Navigation as outlined in the design document.

Regarding navigation, it involved two parts. Firstly, when focusing on the main grid, it should use ‘quote_clue’ to progress to the next cell. Secondly, when focusing on the clue grid, it should move to the next cell within the corresponding clue.
Also, we spent a lot of time discussing on how to differentiate the focus between the main and clue grid. We tried different mockups. @bazylevnik0 came up with some good visualization ideas that we applied.

We then spent a whole week resolving bugs, including issues like ‘undo’ not working.

This is how a loaded Acrostic puzzle appears now.

Yay 🎉

Finally, I added some tests to test out my changes.

We’ll be releasing this soon and would greatly appreciate having more Acrostic puzzles. If you’d like to discuss or suggest any ideas, you can join the chat room here.

Related Merge Requests:

  • Add signals to ClueGrid !115
  • Add quote_clue to MainGrid !116
  • Advance to next cell after guessing !118
  • Use quote_clue to advance on MainGrid !119
  • Acrostic Visualization !120
  • CSS improvements !121
  • Fix Undo bug !122 (by jrb)
  • Add tests for acrostic !123

Thanks for reading!

August 25, 2023

One Device to Do it All

In this article I’ll present how I used to organize myself, mainly using my smartphone for that, and why and how I started changing this.

Dependency

On January 1st 2023 at 00:30, my Android smartphone died, and it made me realize how dependent on that device I was. I used it for many aspects of my life: to stay organized, to be informed, to be entertained, or to move around.

Luckily, at that time I was working for Purism and I had a Librem 5, that was the perfect opportunity to try it out in real conditions, as the device I depend on. I started by listing all I used my previous smartphone for, and tried to fulfill the same needs with the Librem 5. It didn’t go well. The device has some great upsides and interesting features, for sure, but the Linux smartphone ecosystem was very far from something I can depend on, especially given how much I was asking from my smartphone.

I ordered a new smartphone in mid-February 2023, and being back on Android after 1½ months not using it helped me discover things I wasn’t expecting.

Infinite Scrolling

Going back on Android after a forced withdrawal period helped me notice how much I was bombarded with notifications from YouTube, from Twitch, from Discord and more. Every time I need my phone to do something productive, they are there to tempt me, right from the lock screen so even looking at the clock isn’t safe.

These notifications are external unsolicited distractions that are at all time close at hand in my pocket or next to me, if not already in my hands given I need my phone for everything. Even without notifications, these distractions are right there, these apps are just a few centimeters away from the app I need to reach.

When I yield to the temptation, I end up on YouTube and its autoplay, on Twitch and its FOMO and raids, on infinitely scrolling social, or on an endless flow of short videos. The distraction is endless and it can be hard to go back to what I initially needed to do.

When I was using the Librem 5, granted I didn’t have access to these applications on my phone, but I was feeling more at peace, I was able to keep my focus on what I am doing more easily, getting things done was easier as I was less disturbed. Using an Android smartphone is like sailing on the Sea of Entertainment, having to be very careful not to fall for the haunting notifications of the sirens, at the risk of drowning in the depths of the Bay of Infinite Scrolling.

Looking for Better Oganization

A smartphone is a full package, so as long as I use one for everything, I can’t bring my calendar and to-do list with me without also bringing YouTube and Twitch along. Because they are on the same device and as I need my productivity tools to always be at hand, distractions are always at hand too. I need to find some way to not bring nagging distractions with me when I want to work, or at the very least I need to keep them away from the tools I need to work.

I decided to reduce my dependency on my smartphone, which implies fulfilling some of its use-cases with tools that don’t subject me to unsolicited distractions. The more I have other ways to do what I use my smartphone for, the less I’m likely to fall into its attention traps. Another benefit would be that if one of my tools fails, there’s only one task I can’t perform anymore. If I depend on my smartphone for everything, it’s a single point of failure and I’ll be screwed when it will inevitably fail.

If I can do things without having to bring my smartphone with me, that’d be perfect. My goal isn’t to get rid of my smartphone, but to stop depending on it for everything, as I currently don’t have the choice to not use it. Having a single device that’s just a thin and light slab that fits in a pocket is extremely comfortable, so if I replace it with many other dedicated tools, I need these to be as small, light, simple, sober, comfortable and to the point as possible. They must do one thing and do it well.

Moreover, distractions and stimulations are important as they help you recharge your batteries and to remain focused longer. I don’t want to remove distracting applications from my smartphone as they are welcome on some occasions, I do need to find ways to rest and to have fun without my smartphone. Distractions should be solicited by me, not soliciting me, and they should be either finite or easy for me to drop out to go back to work.

Planning

While I disliked the idea of depending on a Google service, I was somewhat satisfied with how I kept track of events and tasks on Google Calendar. I thought about using a planner but I never was super satisfied by them, they mostly bring bad memories from high school, but a few months ago I heard of bullet journaling, and it piqued my curiosity.

I decided to replace Google Calendar with a bullet journal. I took some base ideas from the original bullet journal method and tweaked it to better match how I was planning on Google Calendar to make the transition smoother. In practice, I dedicate half a page to the tasks I need to perform during a given week whenever I find the time for it, then I dedicate half a page for each day of the week for tasks I want to perform on a specific day. I rinse and repeat until I reached a month’s worth (so 4 or 5 weeks) and dedicate two pages for tasks that should be performed after the already written weeks, and I only add a month or two at a time.

I’ve been using this method with an 250 pages A5 hardcover notebook since the end of July 2023, it’s a bit early to judge if it worked but so far this new system isn’t much worse than the previous one. The main downside is that having to carry such a large notebook with me at all time is quite inconvenient, and I end up letting it at home when I’m going out, and even at home I tend to forget about it. If I want such a journal to replace my previous planning method, I need to make it easy to carry around at all time, even when I’m at home. I’m considering switching to an 80 pages pocket-sized journal, because I don’t need to bring a year’s worth of planning around with me, I only need to plan a few weeks ahead, and there probably are better methods to plan further ahead.

Another downside I’ve noticed is that I used to plan with time slots in mind, but now my to-dos are tied to a day. I need finer granularity and flexibility, I need to find some way to easily sort when I will perform a task and for how long, and ideally I should lay the tasks down in a graphical way that will help me get a sense of how the day will go. Without that, my days are just a mishmash of to-dos that I end up not doing at all. The good thing about bullet journals is that at heart, they just are DIY planners. I can let my system evolve as I see fit, and if I end up switching to smaller journals I will be able to make them evolve more quickly as each one should last 2 months and not a year.

Knowing the Time

Then I did something that may sound pretty straightforward as I got myself a wristwatch, but it actually wasn’t that simple as I never really liked them. Every time I saw one, either they were large, bulky and heavy, or they had an uncomfortable wristband that just made them annoying to wear, or they were littered with useless features bloating the dial, or they had virtually no notation that could make the time readable. If not all of these at once.

All I wanted is a simple, sober, small and light watch that just showed the time and did it well, which is what I ended up finding with Casio’s MQ-24 series which is priced at around 20€–25€. You can find some really sober and well designed models from Swatch too, though while they seem slightly more polished they will cost you around 60€–75€, which is a non-negligible difference.

I bought a Casio MQ-24-7B2L mid-August 2023, I’ve been using it for a week now and it turns out I like it a lot! To my surprise, I actually like wearing wristwatches, I just needed to find the right model.

Listening to Music

I use my phone to listen to music as I’m walking, biking, or working. While when listening to music on my phone when I’m walking or biking I can’t be distracted as easily by notificiations, that’s still a reason for me to bring it with me, so I’d rather have a dedicated way to listen to music.

As for the wristwatch, I want a simple, sober, small, light and long lasting device that just plays music and do it well. High end devices are large, bulky, expensive, and full of features I don’t care about, so they aren’t what I’m looking for. Low end devices tend to be pretty simple and straightforward, but they all have a display that takes space, add weight, reduces their battery life and are so low quality that you could just as well not have them. So, I started looking for small music players with no display, rapidly leading me to the iPod Shuffle series and devices inspired by it. Such devices have the added advantage of forcing me to not care about choosing what I’m listening to, letting me focus on what I’m doing.

I think my perfect device to listen to music would be similar in form and function to an iPod Shuffle G4, have a good build quality and UX, use mass storage and microSD cards, offer 20 hours of audio playback, feature VoiceOver, feature Bluetooth, use USB C (though using the 3.5mm jack is fine too), have some good color options, and have no internet connection whatsoever. No device matches all of these, and I could be satisfied with less. I found a few devices offering some of these features, but ultimately I settled for a second hand G4 as their build quality is top notch and you can find some for the price of new similar devices of lesser quality. I may have to replace its battery soonish, but they are fairly affordable and the operation doesn’t look too hard.

I’m not fond of having to use some special software to load music into the iPod, but at least there are several FLOSS options. Rhythmbox and Clementine from Flathub didn’t work, the former didn’t recognize the iPod at all and the latter couldn’t use because it wasn’t built with libgpod. Then I’ve tried this Python script which worked just fine and even supported VoiceOver, but didn’t handle the database very cleanly. Later I tried Rhythmbox from Fedora, and it let me easily handle the music on the device is a clean way, with the downside that it doesn’t support VoiceOver.

I filed the device with puzzle game music, contemplative game music, ambient music, jazz, trip-hop, electronic music… anything goes as long as it’s lyrics-free and calm to help me focus. The first two are great as they are designed specifically with focus in mind.

I ended up liking the iPod a lot. Having only purposefully chosen songs on it, coupled with its shuffle feature and it’s great design and build quality make it the perfect tool to help me focus. The music is always adapted to the situation and never distracts me, I don’t have to bother selecting a song, which means I don’t have to touch the device, a device so light and tiny I tend to forget it’s there at all! I like it so much and they are so cheap nowadays (well, if you know where to search) that I actually ordered a second one I’ll fill with energetic music that helps me focus.

Welcome Distractions

Distractions, as long as they are chosen, are not only welcome but necessary to relieve stress and produce dopamine, which help remaining focused.

I used my smartphone a lot to fidget. It may sound ridiculous, but very regularly I spun my smartphone in my hands, and I juggled with it. It turns out playing with something with my hands helps me stay focused, it’s a need I was fulfilling with my smartphone only because it always was at hand. Now that I understand that, I can accept it’s a need and get myself some fidgets before I break a nearly 400€ device. Ideally, I need something small, silent, that can easily be carried around in a pocket, and that can be used with one hand only. I don’t know much about what exists, but it’s pretty easy to find affordable ones on the second hand market.

I used my smartphone to get the news and to study various topics. I could easily fulfill these use cases with newspapers and books when I’m out, and still get news from my usual online sources when I’m home or when bringing my phone with me is fine.

Looking for highly portable ways to be distracted made me remind of my old Game Boy Micro, and I decided to pick it up ad play with it. I forgot how incredibly tiny that device is, and yet it’s still very comfortable to use! Another advantage of Game Boy Advance games is that they are honnest. They don’t show ads, they don’t use psychological strategies to make you pay… they are are self-contained gaming experiences. And because they are designed for a portable system, they allow short play sessions. On top of that, flash cartridges are fairly affordable and allow to bring a few games along in a single cartridge. I don’t play games on smartphones, I don’t play video games at all actually, but I think I will really enjoy taking a habit of taking my Game Boy Micro with me. 😄

Going Further

Thanks to the iPod and the watch, I’ve been able to go out a few times without bringing my smartphone with me, yet without feeling like I was missing something important. I still need my smartphone for communication, for navigation, for its camera, for its torchlight, and much more, but while having it at all with me is a risk, it’s still a lower risk than having it in my pocket. I can still carry the smartphone around in my backpack when I know I’ll need it for something or when I don’t mind risking to be distracted. So don’t think I need to replace it further, and if in the end this isn’t enough and I still get disturbed by my phone, I can just add other tools to replace some of its use cases.

Is this new organization method in search for fewer external distractions a fad, or will it stick? Only time will tell, but so far it’s pretty promising. On a more lighthearted note, now every time I need to leave, instead of having to look for a single tool that’s not too far away because I definitely used it recently, I need to look for several tools scattered all around my apartment, including the smartphone itself. 🙃 Replace “standards” with “tools” and this classic xkcd comic will perfectly describe the situation I’m in.

Fedora meets RHEL: upgrading UBI to RHEL

Six years ago

As part of our efforts to make Fedora Workstation more attractive for developers, particularly those building applications that would be deployed on Red Hat Enterprise Linux, we had made it easy to create gratis, self-supported Red Hat Enterprise Linux virtual machines inside GNOME Boxes. You had to join the Red Hat Developer Program by creating an account on developers.redhat.com and with that you not only had gratis, self-supported access to RHEL, but also a series of other products like Red Hat JBoss Middleware, Red Hat OpenShift Container Platform and so on.

Few years later

Fedora Silverblue became a thing, and so did Toolbx. So, we decided to take this one step further.

Toolbx already had support for Red Hat Enterprise Linux for the past two and a half years. It means that on RHEL hosts, Toolbx sets up a RHEL container that has access to all the software and support that the host is entitled to. On hosts that aren’t running RHEL, if you want, it will set up a gratis, self-supported container based on the Red Hat Universal Base Image, which is a limited subset of RHEL:

$ toolbox create --distro rhel --release 9.2

However

This works well only as long as you are running a Red Hat Enterprise Linux host. Otherwise, you quickly run into the limitations of the Red Hat Universal Base Image, which is designed for distributing server-side applications, and not so much for persistent interactive CLI environments. For example, you won’t enjoy hacking on GNOME components like GTK, Settings, Shell or WebKitGTK in it because of the sheer amount of missing dependencies.

Today is glorious

You can now have gratis, self-supported Red Hat Enterprise Linux Toolbx containers on Fedora hosts that have access to the entire set of RHEL software, beyond the limited subset offered by UBI.

As always, you need to join the Red Hat Developer Program. Make sure that your account has simple content access enabled, by logging into access.redhat.com and then clicking the subscriptions link at the very top left.

Install subscription-manager on your Fedora host to register it with your Red Hat Developer Program account, create a RHEL Toolbx container as before, and you are off to the races!

Thanks to Pino Toscano for helping me iron out all the wrinkles in the pipeline to get everything working smoothly.