The GNOME Infrastructure has been hosted as part of one of Red Hat’s datacenters for over 15 years now. The “community cage”, which is how we usually define the hosting platform that backs up multiple Open Source projects including OSCI, is made of a set of racks living within the RAL3 (located in Raleigh) datacenter. Red Hat has not only been contributing to GNOME by maintaining the Red Hat’s Desktop Team operational, sponsoring events (such as GUADEC) but has also been supporting the project with hosting, internet connectivity, machines, RHEL (and many other RH products subscriptions). When the infrastructure was originally stood up it was primarily composed of a set of bare metal machines, workloads were not yet virtualized at the time and many services were running directly on top of the physical nodes. The advent of virtual machines and later containers reshaped how we managed and operated every component. What however remained the same over time was the networking layout of these services: a single L2 and a shared (with other tenants) public internet L3 domains (with both IPv4 and IPv6).
Recent challenges
When GNOME’s Openshift 4 environment was built back in 2020 we had to make specific calls:
We’d have ran an Openshift Hyperconverged setup (with storage (Ceph), control plane, workloads running on top of the same subset of nodes)
The total amount of nodes we received budget for was 3, this meant running with masters.schedulable=true
We’d have kept using our former Ceph cluster (as it had slower disks, a good combination for certain workloads we run), this is however not supported by ODF (Openshift Data Foundation) and would have required some glue to make it completely functional
Migrating GNOME’s private L2 network to L3 would have required an effort from Red Hat’s IT Network Team who generally contributes outside of their working hours, no changes were planned in this regard
No changes were planned on the networking equipment side to make links redundant, that means a code upgrade on switches would have required a full services downtime
Over time and with GNOME’s users and contributors base growing (46k users registered in GitLab, 7.44B requests and 50T of traffic per month on services we host on Openshift and kindly served by Fastly’s load balancers) we started noticing some of our original architecture decisions weren’t positively contributing to platform’s availability, specifically:
Every time an Openshift upgrade was applied, it resulted in a cluster downtime due to the unsupported double ODF cluster layout (one internal and one external to the cluster). The behavior was stuck block devices preventing the machines to reboot with associated high IO (and general SELinux labeling mismatches), with the same nodes also hosting OCP’s control plane it was resulting in API and other OCP components becoming unavailable
With no L3 network, we had to create a next-hop on our own to effectively give internet access through NAT to machines without a public internet IP address, this was resulting in connectivity outages whenever the target VM would go down for a quick maintenance
Migration to AWS
With budgets season for FY25 approaching we struggled finding the necessary funds in order to finally optimize and fill the gaps of our previous architecture. With this in mind we reached out to AWS Open Source Program and received a substantial amount for us to be able to fully transition GNOME’s Infrastructure to the public cloud.
What we achieved so far:
Deployed and configured VPC related resources, this step will help us resolve the need to have a next-hop device we have to maintain
Deployed an Openshift 4.17 cluster (which uses a combination of network and classic load balancers, x86 control plane and arm64 workers)
Deployed IDM nodes that are using a Wireguard tunnel between AWS and RAL3 to remain in sync
Migrated several applications including SSO, Discourse, Hedgedoc
What’s upcoming:
Migrating away from Splunk and use a combination of rsyslog/promtail/loki
Keep migrating further applications, the idea is to fully decommission the former cluster and GNOME’s presence within Red Hat’s community cage during Q1FY25
Introduce a replacement for master.gnome.org and GNOME tarballs installation
Migrate applications to GNOME’s SSO
Retire services such as GNOME’s wiki (MoinMoin, a static copy will instead be made available), NSD (authoritative DNS servers were outsourced and replaced with ClouDNS and GitHub’s pipelines for DNS RRs updates), Nagios, Prometheus Blackbox (replaced by ClouDNS endpoints monitoring service), Ceph (replaced by EBS, EFS, S3)
Migrate smtp.gnome.org to OSCI in order to maintain current public IP’s reputation
And benefits of running GNOME’s services in AWS:
Scalability, we can easily scale up our worker nodes pool
We run our services on top of AWS SDN and can easily create networks, routing tables, benefit from faster connectivity options, redundant networking infrastructure
Use EBS/EFS, don’t have to maintain a self-managed Ceph cluster, easily scale volumes IOPS
Use a local to-the-VPC load balancer, less latency for traffic to flow between the frontend and our VPC
Have access to AWS services such as AWS Shield for advanced DDOS protection (with one bringing down GNOME’s GitLab just a week ago)
I’d like to thank AWS (Tom “spot” Callaway, Mila Zhou) for their sponsorship and the massive opportunity they are giving to the GNOME’s Infrastructure to improve and provide resilient, stable and highly available workloads to GNOME’s users and contributors base. And a big thank you to Red Hat for the continued sponsorship over more than 15 years on making the GNOME’s Infrastructure run smoothly and efficiently, it’s crucial for me to emphatise how critical Red Hat’s long term support has been.
I’ve participated in two internships this year, and interns — who are usually busy full-time students — often ask “How do you get time to contribute to open source?”.
And the truth is that there’s no secret formula. It’s tricky to get paid to work on something that you give away for free, isn’t it? Mostly I contribute to open source in free time, either after work hours, or occasionally during periods of downtime.
To my complete surprise I managed to buy a house this year and so I suddenly don’t have any time after work. During the day most of my time is spent on proprietary customer-specific work, and after work I go to look at the house and try to figure out where to start with the whole thing. (By the way, does anyone around Santiago need a load of 1980s-style furniture made from chipboard?).
I’ll still be participating in GNOME around desktop search and the openQA tests, answering questions and triaging bug reports, but I won’t be driving any new stuff forwards.
Anyway, why is it interesting to blog about things I’m not doing?
Make it easy toquit – Actively celebrate people who step back from maintainer positions. Celebrate what they accomplished and what they are moving on to. Don’t punish or otherwise shame quitting. This also incentivizes other people to step up, knowing that they don’t necessarily have to do it forever.
At least in GNOME, we often don’t do this. We don’t celebrate what people *have achieved*, with I think one exception (the legendary “Pants of Thanks” ceremony).
We should do better at this. It’s not that we don’t appreciate each others work. But mostly we require the person doing the work to also be the one shouting loudly about it, before we notice. Is there a better way?
Another thing we don’t do, by the way, is celebrate corporate participation. The great exception to this is the STF grant, and everyone involved in that did an excellent job of highlighting work which the STF grant enabled. We’re less good at crediting all the work that happens thanks to paid engineers from Red Hat, Endless, Canonical, SUSE, and so on.
Another quote from this article:
Each generation of a project (ie open source but not only open source) is responsible for mentoring the next generation. When you mentor someone, spend time emphasizing that it’s their job to mentor the next person, otherwise they will assume that it’s your job. A failure to commuincate this will result in the eventual attrition and death of the community.
I quite like giving conference talks and I’ve been wondering what I could speak about, if I’m not driving any new development myself.
We now have 25 years of history in GNOME and it would be nice to give some talks about “How $thing works.” Desktop search comes to mind here, of course. I also learned (against my will) a lot about initial-setup this year. So I might propose some talks along these lines. It seems like also a nice way to look back at work that’s been done over the years, and give credit the people who have worked on these things over time, doing stuff that’s often invisible.
On that topic, I want to highlight the excellent work done over the summer by our two GSoC interns Divyansh Jain and Rachel Tam, adding a web-based IDE to TinySPARQL that can run queries against the GNOME search database. You can read more about that both on Rachel’s blog and on Demigod’s blog. The idea behind this was making it easier to visualize how the LocalSearch index actually works, what is stored there, and what you can do with it. Hopefully this can lead into some interesting talks about search!
If you like this post, please leave a comment! You use the form below, or reply on the Fediverse to @samthursfield.wordpress.com@samthursfield.wordpress.com. I’m also on LinkedIn.
Last weekend I went to Prague to represent Fedora at LinuxDays 2024. It’s the biggest Linux event in the country with more than a thousand attendees and the Fedora booth is busy there every year.
Like last year the Fedora booth was colocated with the Red Hat booth. It made sense not only because there is a relationship between the two, but it had very practical reasons: I was the only person representing and staffing the Fedora booth and I appreciated help from my colleagues who watch over the Fedora booth when I took a break to have a meal or give a talk.
The biggest magnet at our booth was again a macbook running Fedora Asahi Remix. I gave a talk about it which was only 20 minutes long and was intended as a teaser: here is an overview of the project and if you’d like to know and see more, come to your booth.
Fortunately just two days before the conference, the Asahi Linux project announced support for Steam via the Fex/muvm emulation, so I could utilize a large library of games I own have a license for on Steam. During the talk someone asked if it could run the Factorio game and it could, indeed.
We also had a Fedora conference box which includes a Fedora Slimbook laptop. It was a nice contrast to the Macbook because Slimbook focuses on Linux whereas Apple doesn’t care about Linux at all.
The booth was so busy that I was making a post about our presence for 2 hours because I couldn’t find even a few minutes to finish it.
I also did a bit of user support. An older gentleman approached our booth stating that he had traveled 100km to get help. He had a dual boot of Fedora and Ubuntu and an Ubuntu update had broken the bootloader. Regenerating the GRUB resolved the issue.
Pavel Píša, a doctor from Czech University of Technology, invited me to their booth to check out Fedora Linux running on a Milk-V box with a RISC-V CPU. I left a flyer regarding an open Fedora QA position for RISC-V because Red Hat is currently looking for someone to test Fedora Linux on RISC-V.
If you don’t want to install debuginfo packages for your entire system but still want really useful function names, this is for you. The system-configured debuginfod servers will provide you access to those debuginfo-enabled ELF binaries so that we can discover the appropriate symbol names.
Much like in gdb, you can cancel them if you don’t care and those symbols will fallback to the “In File lib.so+offset” you’re used to.
I expect a bit more UI work around this before GNOME 48 like preferences to disable it or configure alternate debuginfod servers.
We’re trying to increase the fwupd coverage score, so we can mercilessly refactor and improve code upstream without risks of regressions. To do this we run thousands of unit tests for each part of the libfwupdpublic API and libfwupdpluginprivate API. This gets us a long way, but what we really want to do is emulate the end-to-end firmware update of every real device we support.
It’s not trivial (or quick) connecting hundreds of devices to a specific CI machine, and so for some time we’ve supported recording USB device enumeration, re-plug, firmware write, re–re-plug and re-enumeration. For fwupd 2.0.0 we added support for all sysfs-based devices too, which allows us emulate a real world NVMe disk doing actual ioctls() and reads() in every submitted CI job. We’re now going to ask vendors to record emulations for existing plugins of the firmware update so we can run those in CI too.
The device emulation docs are complicated and there’s lots of things that the user can do wrong. What I really wanted was a “click, click, save-as, click” user experience that doesn’t need to use the command line. The tl;dr: is that we’ve now added the needed async API in fwupd 2.0.1 (probably going to be released on Monday) and added the click, click UI to gnome-firmware:
There’s a slight niggle when the user starts recording the first “internal” device (e.g. a NVMe disk) that we need to ask the user to restart the daemon or the computer. This is because we can’t just hotplug the internal non-removable device, and need to “start recording” then “enumerate device(s)” rather than the other way around. Recording all the device enumeration isn’t free in CPU or RAM (and is possibly a security problem too), and so we don’t turn it on by default. All the emulation is also all controlled using polkit now, so you need the root password to do anything remotely interesting.
Some of the strings are a bit unhelpful, and some a bit clunky, so if you see anything that doesn’t look awesome or is hard to translate please tell us and we can fix it up. Of course, even better would be a merge request with a better string.
If you want to try it out there’s a COPR with all the right bits for Fedora 41. It’ll might also work on Fedora 40 if you remove gnome-software. I’ll probably switch the Flathub build to 48.alpha when fwupd 2.0.1 is released too. Feedback welcome.
libadwaita got another new widget - AdwWrapBox - similar to GtkBox, but wrapping children when they can’t fit onto the same line. This can be useful for e.g. displaying tag pills
A new Hieroglyphic update has just been released. The classification backend is now powered by a machine learning model trained on the previously used classification data, improving classification speed and accuracy. However, due to a lack of good training data, some symbols are now less accurately classified.
To improve the training data/accuracy, users can now optionally submit their recognized symbols.
the GNOME Foundation is looking for volunteers to help organize and be at the GNOME stand during FOSDEM: https://discourse.gnome.org/t/call-for-help-for-a-fosdem-stand/24432 If you want to help with Outreach, this is your opportunity! If we don’t get enough people to sign up, we will not be able to apply to the stand
The GNOME Foundation Board published its minutes for the past four months this week. These can be found in two Discourse posts, one for June, and the other for July–September. This means that the board is now up to date with publishing its minutes.
The board also published its board responsibilities, ethical conduct, and confidentiality policies for the first time. These were included in a new policies wiki page, which was created as part of the board’s migration off wiki.gnome.org (which is planned for retirement).
as promised in our communication last week, the GNOME Foundation Board of Directors has written an update on the financial status and budget approved between October 2024 and September 2025: https://discourse.gnome.org/t/foundation-2024-2025-budget-and-economic-review/24436. We hope to continue providing regular updates on financial work
That’s all for this week!
See you next week, and be sure to stop by #thisweek:gnome.org with updates on your own projects!
As promised in the previous communication the Board would like to share some more details on our current financial situation and the budget for our 2024-2025 financial year, which runs from 1st October 2024 to 30th September 2025.
Background
The Foundation needs an approved budget in place because our spending policies use the budget to authorise what staff and committees are allowed to spend money on. This year we passed the budget on time for the start of the financial year, which was thanks to a lot of detailed and particularly challenging work by Richard, which the board is grateful for.
We consider the budget in 2 distinct parts:
Budget for our fiscally-sponsored projects. We consider their income, but not their expenses. The reason for that is that the Foundation takes a small part of the income as the fiscal sponsorship fee, supporting our administrative and operating costs. Funds received on behalf of other projects are tracked separately, called “reserved funds”, and the Foundation cannot spend money that belongs to the other projects.
General operating budget for the GNOME Foundation, which is what this post is all about! At any later point, when talking about the budget, we’re talking about the general/unrestricted operating funds and it is safe to assume that income for fiscally-sponsored projects is not included.
The budget for the previous 2023-2024 fiscal year was presented to the board as a roughly balanced break-even budget, anticipating $1.201M of revenue and $1.195M of expenses. The board considered two fundraising scenarios proposed by our previous ED, with the most ambitious scenario planning to raise an additional $2M for the Foundation, and one more conservative which anticipated an additional $475k of revenue from various sources (donations, grants, event sponsorship). This more conservative scenario was included in the budget, but in practice things did not work out as planned. This additional funding was not raised, meaning that in practice the Foundation once again ran at a deficit over the past year and used funds from our reserves.
The new 2024-2025 budget considers a total income of $586k, and total expense of $550k. Two things are clearly different from last year: the expenses have been greatly reduced, and we have aimed for a surplus instead of the deficit we ended up with last year. Both things were a consequence of the budget from previous year not being executed as expected. Since our reserve policy requires us to retain enough money to sustain core operations without income for another year (specifically, 1.1 times core spending), we’ve had to reduce expenses to save money and restore our reserves.
So, let’s dig into the details:
Income
$205,100 in donations. This number is based on previous years income, of individual contributions ($75,000), Advisory Board fees ($105,800), and other small contributions ($7,800) like matching donations (where companies double what employees donate). It also includes $16,500 currently pending from Wau Holland Stiftung, an organization we had a historic agreement with to collect funds from European donors that is tax deductible. We believe that there is a great potential for the GNOME Foundation to increase the amount of individual contributions received, and this has been included in the Strategic Plan and many board discussions. Unfortunately, without a permanent Executive Director, we cannot guarantee that we will be able to establish a program to do so in the short-term, so we have decided to budget conservatively to ensure economic sustainability.
$64,500 from event sponsorship. Most of that money comes from GUADEC ($61,000), with some from LAS and GNOME Asia, which is one of the main reasons why we are able to maintain our events: because they are sponsored separately, they are mostly self-sustaining.
$65,500 in fiscal sponsorship fees. This is based on a % fee the GNOME Foundation takes for our operational costs from hosting GIMP and Black Python Devs. This number is uncommonly high due as we have been workng with the GIMP on financial and legal arrangements to receive approx $1M of historical Bitcoin donations. (And sell them immediately – holding Bitcoin assets creates a regulatory/reporting problem for US nonprofits and our accountants have advised us against it.)
$1,000 in interest from money in the bank account. This is budgeted higher than previous years, as work is already in progress to change bank accounts to increase this income, as recommended by our auditors.
$500 profit from selling T-shirts and other goods ($2,500 income, $2,000 in expenses).
$250,000 from the 2nd year of an Endless grant that was approved last year. This grant provides $50,000 for general funds that the Foundation can use at its discretion, and $200,000 that need to be spent on specific tasks. Currently, those are assigned to Flathub, Parental Controls, GNOME Software maintenance, and internships. Some of those will be detailed in the expense section.
Expenditures
$10,000 interim ED salary. This is to be able to pay Richard to continue managing the Foundation and staff team until 10th December.
$100,000 for development contractors for work associated with the Endless grant. This work includes improvements in Parental Controls and GNOME Software, and is being executed by Philip Withnall (development), Sam Hewitt (design) and potentially one more developer over the coming year. Philip gave an update on the work in his presentation at GUADEC.
$110,600 in contractor costs for program staff, including events and infrastructure. This covers Kristi’s work which is the backbone of events such as GUADEC, LAS and GNOME.Asia, and Bart’s work running GNOME and Flathub infrastructure. The Flathub portion of this work is funded by the Endless grant.
$32,000 in Outreachy interships. This is a long-term partnership with Conservancy and commitment by the GNOME Foundation as the original birthplace of the Outreachy initiative. They are supported this year by reallocating some of the Endless grant, with their permission. This will pay for a total 4 interns between the winter and summer cohort.
$20,000 in contractor support. This is allocated for part-time contracting of Thibault Martin and Dawid Jankowiak to support the STF team and work on a crowdfunding platform for our development fundraising. Some of this is funded by the Endless grant and will be spent on coordinating the next steps of the Flathub payments/donations launch.
$158,000 in employment/contractor costs for operations and admin staff, supporting the GNOME Foundation across finances, events and community initiatives.
$47,500 in professional services, ie legal and accounting. These include a reserve for legal fees ($10,000), an external accounts audit for the previous financial year ($17,500), which is required due to our income (mostly due to STF) being over the $2M threshold, and accounting fees ($20,000). Some of the financial and legal costs are driven by work setting up Flathub LLC and are covered by the Endless grant.
$3,200 in office expenses, mostly related to postal expenses required for sending material between contractors, staff, and event organisers.
$54,000 in conferences and travel. These include the budget for the conferences themselves ($30,000), which includes GUADEC, GNOME Asia, and hackathons around the globe, but also travel for staff ($12,000) and community ($12,000). Travel particularly has been significantly reduced from previous year, but should still allow for staff/organisers to attend our events, and for the travel committee to support some community travel to GUADEC and GNOME Asia.
$15,000 in other fees. These include banking costs for sending money from the US to Europe, PayPal fees, and insurance. They might seem high, but are in total less than 1.5% of the cash flow of the Foundation, which is within the expected value for any organization.
Balance
As of the preparation of this budget, we have approx $140,000 in GNOME Foundation reserves. There’s a lot more money in the bank, but they are reserved funds held for GIMP and BPD.
We need to ensure that we meet our reserve policy of retaining 1.1 times core spending. Unfortunately, core spending is fairly loosely defined. This year, we have considered: Events and minimal staff travel, part-time infrastructure support, minimal staff, and some fees and professional services. In total, we accounted that we would need at least $158,000 at the end of the year to be able meet the policy.
The approved budget should put our reserves around $176,000 at the year end, which is slightly above our reserve policy. Considering we used a very limited interpretation of the reserves policy, it’s better to include a small safety margin for any unanticipated costs.
Conclusion
With limited time from our interim Executive Director (ED), Richard Littauer, who is working part-time, the board is prioritising: recruiting our new ED, delivering our current project/grant commitments (to STF and to Endless), and fundraising for development work. This includes working with the community to launch our development fund crowdfunder/platform and plan a follow-up project for STF grant, so that the GNOME Foundation can support and grow its direct investment in project development.
Keen readers will note that there is nothing in the current budget for the ED’s salary. We are in discussions with a potential donor to see whether we can find support for the salary for the ED for the first year. In any case, transparently sharing our financial situation and fundraising needs is an essential part of any ED recruitment process, so we could still recruit somebody with “raise money for your own salary” being their first priority.
Hopefully this additional detail helps to show the challenges of our current situation, and why we had to make really tough decisions, like parting ways with some greatly appreciated members of our staff team. We hope this sheds some more light on why those decisions were taken, provides confidence on the work done by the board and the ED, and where we currently stand. We are also very relieved to be able to provide a surplus budget for the first time in many years, and doing so while still being able to support the community: events, infrastructure, internships, travel funding, and meeting our commitment to donors for work done in some parts of the stack, e.g.: Flathub, parental controls and GNOME Software.
We welcome any feedback and questions from the GNOME community. Thanks to all of our GNOME members, contributors, donors, sponsors and advisory board members!
There is this extremely neat feature in GCC 10+ and Clang 9+ that allows you to determine which function should be patched in at runtime. Procress startup code will call your user-defined function to figure out which function should be used for the environment.
This can be handy when you’re dealing with complex branching based on the system. For example, imagine you need to check if you’re running under Valgrind as one does occasionally to satisfy an incorrect assumption about accessing past heap boundaries.
The typical way to handle this is to #include "valgrind.h" and branch based on if (RUNNING_ON_VALGRIND) ....
However, that inlines a bunch of assembly you probably don’t want in your ultra-fast new utf8-validation implementation. It might be better to just do that once at process startup and save yourself a bunch of CPU cycles.
This year PyCon Spain happened just down the road from me in Vigo, and I was able to attend with a few folk from Codethink. Python conferences are usually great as there’s so much variety in the talks and the bar for quality is also quite high.
I’m not sure when the videos will be out and they’ll anyway be in Spanish, but here are some notes for the talks I found most interesting. (Codethink sponsors our attendence to conferences on the basis that we write an internal report afterwards, which is where most of these notes came from).
Presented by Juan Luis Cano Rodríguez, who is a co-founder of Python España and PyCon ES. This talk was activism with some data engineering, in the context of the housing crisis that’s affecting most Spanish cities and tourist destinations.
This talk is the story of how he accidentally got involved as an activist after he noticed someone in his own appt building in Madrid was doing short-term rents to tourists, and wondered whether they had a license.
He started by asking the audience to guess how many long-term rental properties are available on Idealista for Madrid. Is it 7,000, 21,000 or 43,000? Then he asked to guess how many short-term rentals are available on AirBnB.
Tourist apartments in Madrid are licensed only after a professional inspection. The council publish data on how many licenses they’ve issued, and there are 1008 properties listed. Meanwhile the property register, which doesn’t require any inspection, lists 11,000 short-term rental properties. AirBnB meanwhile has 26,000 offers in Madrid. He realized that illegal apartment rentals are far worse than he had imagined.
He used Python (of course) to compare the register of properties against the license data to identify which properties were unlicensed, and participated in a group effort to send 10,000 individual reports to the council in one day. This made the national news back in June. The mayor of Madrid even praised this “citizens effort”.
AirBnB has a “license” field for short-term rentals in Spain, but they don’t do any checking of whether it’s valid. He experimented with their “Report” button with an apartment he knew was unlicensed, and AirBnB’s internal “investigation team” reported back a couple of days later that they had found nothing wrong.
Towards the end of the talk he offered the audience a choice, he could explain the data wrangling with Python in more detail, or give his thoughts on the housing crisis. The vote was around 90% for the latter, and his message was that (unsurprisingly) a lot more pressure is needed before anything is going to change. He finished by announcing a protest happening on the 13th October in Madrid.
The talk was very well received and could probably have filled 3 hours instead of 35 minutes.
Presented by Pablo Galindo Salgado, a Python core developer and Bloomberg engineer, this was a summary of the incredible work going on to remove CPython’s “Global Interpeter Lock” or GIL. The full technical details of this are in PEP 703, this is my short summary.
Again this talk could have been a full-day workshop but Pablo packed a lot of details into the time, using the simple of trick of talking incredibly fast.
CPython’s global lock means that writing multithreaded Python code is a waste of time, as the threads just get stuck waiting for the global lock. Today people use horrible workarounds such as the ‘multiprocessing’ library.
However, the GIL has advantages too, mainly its simplicitly. Deadlocks are a common problem in multithreaded code but currently CPython is immune to all that — a deadlock is caused when two mutexes block each other, but of course CPython only has the one lock.
So the big challenge is replacing the GIL with many separate locks, without making CPython slower, and avoiding deadlocks. CPython uses reference counting to manage memory, and this is the hardest piece of the puzzle as sharing objects between threads means every refcount change must be atomic and this can require locking, which limits throughput.
People have been trying to solve this since the mid 1990s and noone has yet succeeded, but the work here is actually being released this week in Python 3.13. It’s behind a compile-time configure flag, and the expectation is to spent around 5 years testing this before it’s enabled by default for everyone.
A minimal overview of how they’ve done it:
switch to biased ref counting for Python objects: this means there are now two refcounts per object, a thread-local refcount and a global refcount. The global one can be slow to access, but most of the time a local refcount is good enough and this is a fast path.
make common objects immortal: remember Python dates from the golden age of object-oriented programming, and everything is an object including True, False, None, the number 4 and so on. By setting the highest 2 bits of the refcount, these are now marked as immortal.
fast locking: using spinlocks from WebKit (the excellently named WTF::Lock) where possible
new memory allocator: use mimalloc, which allows linked list segments to be allocated contiguously instead of all over the place.
It seems a lot of this work has been funded by corporations, mainly Meta as noted in the Python 3.13 release notes, and we can assume Bloomberg are somewhat involved as well.
I hope Pablo does the same talk in English at some point because it was excellent. In the meantime you can hear him on the Changelog podcast:
The tsunami of disinformation: how Generative AI can help us in the fight against Fake News.
Presented by Rubén Míguez and Agustín Cañas from Newtral, a fact-checking organization in Spain.
There are many bullshit talks around about how generative AI will magically solve some or other intractable problem, as if a text synthesis machine built from Reddit comments is somehow going to succeed where our best scientists and leaders have so far failed. I recommend reading https://pivot-to-ai.com/ on that topic, but let me get back to the talk.
The talk began with an overview of fake news, which I’m sure I don’t need to repeat here, except the excellent statistic taken from Washington Post fact-checkers that during 2016-2020, the then US president made around 30,000 provably false statements, which works out at 21 lies a day, every day for four years including Sundays and bank holidays. So it’s been an interesting time in the world of fact-checkers.
Newtral exists since 2013 and began in a very analogue way: human fact checkers watching news programs with big notebooks and writing down things to be verified. The goal of the small technology team was not to replace the human fact checkers but to build tools that could make them work 30x as fast.
(So if you were wondering how generative AI tools could possibly detect fake news, when Google’s “state of the art” AI is telling people to throw car batteries into the ocean and put glue on pizza, the answer is of course: it’s not going to, there will still be a human fact checker in the loop).
The fact checking process is roughly as follows:
Monitor media
Spot facts
Verify them
Publish (“exploit”) the result
The talk detailed some tools to automate the first two parts: using speech-to-text models to transcribe political speeches and so on as they happen, and then text analysis tools to identify potentially “verifiable” statements. As well as deciding whether a statement can be verified, they have to analyse whether it matters, whether the misinformation could be dangerous, whether it’s a joke, and so on. There wasn’t much detail how they do this in the talk, unfortunately.
Then they mentioned a multimodal model named ClaimCheck which has run as a WhatsApp bot since 2020 where people can send forwarded messages to see if they are true. This works using RAG (Retrieval Augmented Generation) to search an existing database of human-verified facts. Again I’d have loved more details on how this works.
Finally they mentioned that Newtral will be launching a technology-focused spinoff named Trueflag, aiming to build the existing tech into a software-as-a-service tool for fact checking that can be used by news organisations, social media platforms and so on.
I found this article online which gives some more info:
I’m still very curious about how far they can get with this tech and will be following closely.
Other note
There was some talks I didn’t enjoy so much. One about using Pyomo to model the game Age of Empires 2, delivered by some mathmeticians who unfortunately didn’t actually hook up their model to the real game, and the second half the talk was just graphs when what we obviously wanted was footage from actual Age of Empires 2 games. Pyomo looks interesting though.
I also saw a talk on the Robot Framework for testing. The talk was well delivered but I fundamentally disagree with the premise that by hiding Python code behind a bunch of free form text, “non-programmers” will be able to write and adapt test suites. The tool itself looks fine, but I wonder when we’ll learn that there’s no tool that can magically make testing easy, people just have to learn to write excellent test suites using code, and that’s the only way anyone will be able to produce a good test suite. (I’m considering doing a talk myself called “How to write a good test suite using regular programming tools”.)
Vigo was of course fantastic despite the intermittent rain, minor travel issues and increasingly strange procedures for checking in to tourist apartments. (This is the first time I’ve stayed in a place with a “Virtual key”, a web page linked to a smart lock on the apartment door which means you need working internet to enter the place). At least the rental apartment did have a license.
I’d recommend Python community events to everyone. People use Python for all kinds of different things – science, journalism, art, activism, and so on – you always get an interesting mix of talks and not just dry technical presentations of screenshots of code. Even the very technical talk on the GIL was not dry at all. Wherever you live you probably have a local Python community and they may well organize an annual conference, worth investigating.Those of us outside the UK no doubt also have a national Python community who probably hold a conference from time to time. Get involved!
Thanks as always to Codethink for sponsoring accommodation, food and sponsoring the event itself.
A long overdue dev log. The last one was for September 2023. That's a
year. Stuff in life has happened.
Compiano
In November I switched Compiano to use pipewire directly for
sound. This mean removing bits of UI too.
I should look at a release, but I have a few blockers I need to
tackle. One key element is that there is a mechanism to download the
soundbanks and for now it doesn't ask consent to do so. I don't want a
release without this.
Lot has happened on that front. I want it to be a foundation of Niepce
and others.
First adding it to glycin triggered an alpha release on
crates.io. There started a long cycle of alpha release, we are at
alpha 8 as of now. Various changes:
Added the mp4parse crate directly as a module. A key reason is that
I already used a fork so it complicate things. Maybe I should make
these few bits upstreamable and use upstream.
Added a mime types API
Save up a lot of RAM when doing colour interpolation: it's done in
place instead of allocating a new buffer. This is significant as
this is done using a 64-bit float per component.
Fixed the rendering in many ways. The only thing it needs it to
apply the colour balance.
Fixed unpack or decompression of Olympus and Fuji files.
Got an external contribution: Panasonic
decompression. This
made me fix the loading of uncompressed Panasonic raw too. The more
recent Panasonic cameras are still failing though, there is a subtle
variant that needs to be handled.
Still missing from rendering: recent Nikon and all their exotic
variants and compression scheme, Canon CR3, GoPro, Sony.
Niepce
Not so much work directly done on Niepce in the last few month, but
still.
Ongoing features
Started a while ago some work towards the import and a rework of the
catalog (the main data storage).
The former is the implementation of a workflow that allow immporting
images into the catalog.
The latter involve reworking the catalog to become a self contained
storage as a sqlite3 database. One step I already did was to use it to
store the catalog preferences instead of a separate file. This should
also include fixing the UI open, creating, switching.
This big two things are user visible and are a stop forward what I
want to happen as an internal milestone. Then I can start pluging the
library import and maybe import my picture vault. A good starting
point towards managing the collection, but not really for photo
editing yet. Gotta make choices.
Images
Implemented support for HEIF which is being adopted by camera
manufacturers.
I updated the RT engine to 5.11 which came with RawTherapee 5.11. This
is still a soft work of the code base to use strip out Gtk3 and a more
recent version of glibmm. The latter patch might no longer be needed
as I have since removed gtkmm from Niepce.
I also implemented the GEGL pipeline using
gegl-rs, which I took
over to make it useful. At one I shall try to figure out how to write
a loader in Rust to use libopenraw with GEGL.
Cleanups
The UI is slowly moving to use blueprint, and I removed all the
first-party C++ code outside of bindings, no more Gtkmm and Glibmm is
only here because RT engine needs it.
Other
Stuff I contributed to.
STF
I took part to the STF effort and worked on fixing issues with
the desktop portal. The big chunk of the work related to the USB
portal, taking over code by Georges that is itself based on code by
Ryan. It spreads through multiple component of the stack: flatpak,
xdg-desktop-portal, xdg-desktop-portal-gnome, libportal and ashpd.
I also did a bunch of bug fixes, crashes, memory leaks, etc in
flatpak, flatpak-builder, and the rest of the stack.
I also implemented issue
#1 for
flatpak-builder: easy renaming of MIME files and icons, and also
properly fixing the id in the appstream file, a common problem in
flatpak.
Glycin
Glycin is a sandboxed image loader. I did implement the raw camera
loader
using libopenraw. It's written in Rust, so is libopenraw now. Thank
you Sophie for merging it.
Poppler
Jeff was complaining about a file being super
slow,
with sysprof flamegraph. That picked my curiosity and looked at
it. The peculiarity of the document is that it has 16000 pages and a
lot of cross references.
This lead to two patches:
The first
one
is in 24.06. There was a loop, calling for the length of the
container at each iteration. Turns out this is protected by a mutex
and that's a lot of time spend for nothing since the value is
immutable. Call it once before the loop and voila.
The other
one,
merged for 24.09 change that loop to be a hash table lookup. The
problem is that it want to locate the page by object reference, but
iterate through the page list. Lots of ref and lots of a page mean
even more iterator. The more complex approach is when building the
page cache (it's done on demand), we build a reference to page index
map. And the slow code is no longer slow and almost disappear from
the flamegraphs.
This make Evince and Okular faster to open that document and any
presenting similar attribute: a lot of bookmarks.
We want to provide you with an important update on recent developments at the GNOME Foundation.
What Has Happened
The GNOME Foundation Board of Directors has approved a budget for the coming financial year (October 1, 2024, to September 30, 2025). In the process, we’ve had to make some tough decisions to ensure the Foundation’s long-term financial sustainability.
What Is Impacted
Staff Changes
Regrettably, we have had to reduce our staff. Caroline Henriksen (Creative Director) and Melissa Wu (Director of Community Development) are no longer members of the GNOME Foundation staff team. We sincerely thank Caroline and Melissa for their significant contributions over the past years and wish them the best in their future endeavors.
Operational Adjustments
Critical tasks will be redistributed among remaining staff and the Board. We will be reaching out to the community for more support in areas such as:
Event organization and representation
Marketing initiatives
Fundraising efforts
Graphic design
Reduced Travel
Unless additional funds are secured, there will be significant reductions on community, board and staff travel to events. We’ll be reassessing which events are most critical for staff attendance.
What Is Unaffected
Key Events: LAS, GNOME.Asia, GUADEC
We are hosting the Linux App Summit in partnership with KDE this weekend, on 4th-5th October in Monterrey, Mexico. Other events such as GNOME.Asia and GUADEC are continuing as planned and remain a core priority for our team. We’re very grateful to our event sponsors for separately supporting these important events to bring our community together.
Internships
We have secured funding to continue our participation in Outreachy at our previous level of four interns a year, two later this year and two early next year. Our participation in Google Summer of Code is supported by our greatly-appreciated Internship Committee and is also not impacted by these changes.
Externally Sponsored Projects
Certain other projects, including all of the infrastructure, staff, legal and operating costs of Flathub, and ongoing development work on Digital Wellbeing / Parental Controls, are fully sponsored by an existing grant from Endless. We are also working with our team of contractors to complete the delivery of the remaining elements of the Sovereign Tech Fund contract. We are not able to divert these funds to different purposes, but equally, they are fully funded to continue even though we’ve had to reduce spending in other areas.
Infrastructure & Operations
All of the gnome.org infrastructure remains fully funded and staffed as before, together with our core finance, operations and administrative functions.
Why This Has Happened
Our plan for the previous financial year was to operate a break-even budget. We raised less than expected last year, due to a very challenging fundraising environment for nonprofits, on top of internal changes such as the departure of our previous Executive Director, Holly Million.
The Foundation has a reserves policy which requires us to keep a certain amount of money in the bank account, to preserve core operations in the event of interruptions to our income.
In order to meet our reserves policy, this year’s budget had to reduce our expenditure to below expected income, and generate a small surplus to reinstate the Foundation’s financial reserves to the necessary level.
What’s Next
Executive Director Recruitment: We are actively searching for a new Executive Director. This process is unaffected by the current budget constraints as we are exploring different ways to fund the role.
Community Support: We’re asking for your support in several ways:
Look out for opportunities to volunteer your time and skills in areas where we’ve had to reduce staff involvement.
Share ideas on how to organize and improve our activities in this new context.
Consider making donations to support the GNOME Foundation’s core priorities, if you’re able.
Transparency: In the coming weeks, we’ll share a more detailed report of the current finances and the approved budget, including specific categories and the reasoning behind various decisions.
Ongoing Communication: We’re committed to providing regular updates on our progress and welcome your feedback on how often you’d like to hear from us.
Through these difficult decisions, the GNOME Foundation is able to meet its reserves policy, ensuring sufficient funds for the coming year. Our budget for the new financial year is realistic and supports four full time staff, who are able to support key operations like finance, infrastructure and events. We are additionally contracting a number of other individuals on a short term or part time basis, to help with fund raising, websites and delivering on our project commitments.
We are going to be looking to the GNOME community to help with the areas that are most affected by our reduced staffing. If you would like to help GNOME with its events, marketing, or fundraising, we would love to hear from you. We would also welcome the community’s input on the best way to organize these activities, so please feel free to reach out to our Interim Executive Director, Richard Littauer (richard@gnome.org), or the Board if you have ideas.
Thank you all for your continued commitment to the GNOME community, and thanks to all of the existing donors, sponsors and advisory board members whose support year on year is essential to maintain the Foundation’s core operations.
I have just made the 0.12 release of CapyPDF. It does not really have new features, but the API has been overhauled. It is almost guaranteed that no code developed against 0.11 will work without code changes. Such is the joy of not having any users.
Experimental C++ wrapper
CapyPDF has a plain C API. This makes it stable and easy to use from any programming language. That also makes it cumbersome to use. Here is what you need to write to create a PDF file that has a single rectangle:
Given that this is C, you can't really do better. However I was asked if I could create something more ergonomic for C++ users. This seemed like an interesting challenge, so I did some experimentation, which ships with the 0.12 release.
The requirements
The C++ wrapper should fulfill the following requirements:
Fully type safe
No manual memory management
Ideally a single header
Zero overhead
Fast to compile
All objects are move-only
IDE code completion friendly
Does not need to maintain API or ABI stability (those who need that have to use the C API anyway)
The base
After trying a bunch of different things I eventually came up with this design:
Basically it just stores an underlying CapyPDF object type and a helper class used to deallocate it. The operators merely mean that the object can be cast to a pointer of the underlying type. Typically you want conversion operators to be explicit but these are not, because begin implicit removes a ton of boilerplate code.
For example let's look what the Color object wrapper looks like. First you need the deleter:
The class definition itself is simple:
The class has no other members than the one from the base class. The last thing we need before we can start calling into CapyPDF functions is an error handler. Because all functions in the API have the exact same form, this can be done with a single macro:
This makes the constructor look like this:
Method calls bring all of these things together.
Because the wrapper types are implicitly convertible to the underlying pointer types you can pass them directly to the C API functions. Otherwise the wrapper code would be filled with .get() method calls or the like.
The end result
With all that done, the C code from the beginning of this post can be written like this:
Despite using templates, inheritance and stdlib types the end result can be proven to have zero overhead:
After this what remains is mostly the boring work of typing out all the wrapper calls.
Last weekend we had another edition of last year’s post-All Systems Go hackfest in Berlin. This year it was even more of a collaborative event with friends from other communities, particularly postmarketOS. Topics included GNOME OS, postmarketOS, systemd, Android app support, hardware enablement, app design, local-first sync, and many other exciting things.
This left us with an awkward branding question, since we didn’t want to name the event after one specific community or project. Initially we had a very long and unpronounceable acronym (LMGOSRP), but I couldn’t bring myself to use that on the announcement post so I went with something a bit more digestible :)
“Boiling The Ocean” refers to the fact that this is what all the hackfest topics share in common: They’re all very difficult long-term efforts that we expect to still be working on for years before they fully bear fruit. A second, mostly incidental, connotation is that the the ocean (and wider biosphere) are currently being boiled thanks to the climate crisis, and that much of our work has a degrowth or resilience angle (e.g. running on older devices or local-first).
I’m not going to try to summarize all the work done at the event since there were many different parallel tracks, many of which I didn’t participate in. Here’s a quick summary of a few of the things I was tangentially involved in, hopefully others will do their own write-ups about what they were up to.
Mobile
Mainline Linux on ex-Android phones was a big topic, since there were many relevant actors from this space present. This includes the postmarketOS crew, Robert with his camera work, and Jonas and Caleb who are still working on Android app support via Alien Dalvik.
To me, one of the most exciting things here is that we’re seeing more well-supported Qualcomm devices (in addition to everyone’s favorite, the Oneplus 6) these days thanks to all the work being done by Caleb and others on that stack. Between this, the progress on cameras, and the Android app support maybe we can finally do the week-long daily driving challenge we’ve wanted to do for a while at GUADEC 2025 :)
Design
On Thursday night we already did a bit of pre-event hacking at a cafe, and I had an impromptu design session with Luca about eSIM support. He has an app for this at the moment, though of course ideally this should just be in Settings longer-term. For now we discussed how to clean up the UI a bit and bring it more in line with the HIG, and I’ll push some updates to the cellular settings mockups based on this soon.
On Friday I looked into a few Papers things with Pablo, in particular highlights/annotations. I pushed the new mockups, including a new way to edit annotations. It’s very exciting to see how energetic the Papers team is, huge kudos to Pablo, Qiu, Markus, et al for revitalizing this app <3
On Saturday I sat down with fellow GNOME design contributor Philipp, and looked at a few design questions in Decibels and Calendar. One of my main takeaways is that we should take a fresh look at the adaptive Calendar layout now that we have Adwaita breakpoints and multi-layout.
47 Release Party
On Saturday night we had the GNOME 47 release party, featuring a GNOME trivia quiz. Thanks to Ondrej for preparing it, and congrats to the winners: Adrian, Marvin, and Stefan :)
Local-First
Adrian and Andreas from p2panda had some productive discussions about a longer-term plan for a local-first sync system, and immediate next steps in that direction.
We have a first collaboration planned in the form of a Hedgedoc-style local-first syncing pad, codenamed “Aardvark” (initial mockups). This will be based on a new, more modular version of p2panda (still WIP, but to be released later this year). Longer-term the idea is to have some kind of shared system level daemon so multiple apps can use the same syncing infrastructure, but for now we want to test this architecture in a self-contained app since it’s much easier to iterate on. There’s no clear timeline for this yet, but we’re aiming to start this work around the end of the year.
GNOME OS
On Sunday we had a GNOME OS planning meeting with Adrian, Abderrahim, and the rest of the GNOME OS team (remote). The notes are here if you’re interested in the details, but the upshot is that the transition to the next-generation stack using systemd sysupdate and homed is progressing nicely (thanks to the work Adrian and Codethink have been doing for our Sovereign Tech Fund project).
If all goes to plan we’ll complete both of these this cycle, making GNOME OS 48 next spring a real game changer in terms of security and reliability.
Community
Despite the very last minute announcement and some logistical back and forth the event worked out beautifully, and we had over 20 people joining across the various days. In addition to the usual suspects I was happy to meet some newcomers, including from outside Berlin and outside the typical desktop crowd. Thanks for joining everyone!
Thanks also to Caleb and Zeeshan for helping with organization, and the venues we had hosting us across the various days:
Today I tagged fwupd 2.0.0, which includes lots of new hardware support, a ton of bugfixes and more importantly a redesigned device prober and firmware loader that allows it to do some cool tricks. As this is a bigger-than-usual release I’ve written some more verbose releases notes below.
The first notable thing is that we’ve removed the requirement of GUsb in the daemon, and now use libusb directly. This allowed us to move the device emulation support from libgusb up into libfwupdplugin, which now means we can emulate devices created from sysfs too. This means that we can emulate end-to-end firmware updates on fake hidraw and nvme devices in CI just like we’ve been able to emulate using fake USB devices for some time. This increases the coverage of testing for every pull request, and makes sure that none of our “improvements” actually end up breaking firmware updates on some existing device.
The emulation code is actually pretty cool; every USB control request, ioctl(), read() (and everything inbetween) is recorded from a target device and saved to a JSON file with a unique per-request key for each stage of the update process. This is saved to a zip archive and is usually uploaded to the LVFS mirror and used in the device-tests in fwupd. It’s much easier than having a desk full of hardware and because each emulation is just that, emulated, we don’t need to do the tens of thousands of 5ms sleeps in between device writes — which means most emulations take a few ms to load, decompress, write and verify. This means you can test [nearly] “every device we support” in just a few seconds of CI time.
Another nice change is the removal of GUdev as a dependency. GUdev is a nice GObject abstraction over libudev and then sd_device from systemd, but when you’re dealing with thousands of devices (that you’re poking in weird ways), and tens of thousands of device children and parents the “immutable device state” objects drift from reality and the abstraction layers really start to hurt. So instead of using GUdev we now listen to the netlink socket and parse those events into fwupd FuDevice objects, rather than having an abstract device with another abstract device being used as a data source. It has also allowed us to remove at least one layer of caching (that we had to work around in weird ways), and also reduce the memory requirement both at startup and at runtime at the expense of re-implementing the netlink parsing code. It also means we can easily start using ueventd, which makes it possible to run fwupd on Android. More on that another day!
The oldThe new
The biggest change, and the feature that’s been requested the most by enterprise customers is the ability to “stream” firmware from archives into devices. What fwupdmgr used to do (and what 1_9_X still does) is:
Send the cabinet archive to the daemon as a file descriptor
The daemon then loads the input stream into memory (copy 1)
The memory blob is parsed as a cabinet archive, and the blocks-with-header are re-assembled into whole files (copy 2)
The payload is then typically chunked into pieces, with each chunk being allocated as a new blob (copy 3)
Each chunk is sent to the device being updated
This worked fine for a 32MB firmware payload — we allocate ~100MB of memory and then free it, no bother at all.
Where this fails is for one of two cases: huge firmware or underpowered machine — or in the pathological case, huge video conferencing camera firmware with inexpensive Google ChromeBook. In that example we might have a 1.5GB firmware file (it’s probably a custom Android image…) on a 4GB-of-RAM budget ChromeBook. The running machine has a measly 1GB free system memory, and then fwupd immediately OOMs when just trying to parse the archive, let alone deploy the firmware.
So what can we do to reduce the number of in memory copies, or maybe even remove them all completely? There are two tricks that fwupd 2.0.x uses to load firmware now, and those two primitives we now use all over the source tree:
Partial Input Stream:
This models an input stream (which you can think of like a file descriptor) that is made up of a part of a different input stream at a specific offset. So if you have a base input stream of [123456789] you can build two partial input streams of, say, [234] and [789]. If you try and read() 5 bytes from the first partial stream you just get 3 bytes back. If you seek to offset 0x1 on the second partial input stream you get the two bytes of [89].
Composite Input Stream
This models a different kind of input stream, which is made up of one or more partial input streams. In some cases there can be hundreds of partial streams making up one composite stream. So if you take the first two partial input streams defined a few lines before, and then add them to a composite input stream you get [234789] — and reading 8 bytes at offset 0x0 from that would give you what you expect.
This means the new way of processing firmware archives can be:
Send the cabinet archive to the daemon as a file descriptor
The daemon parses it as a cab archive header, and adds the data section of each block to a partial stream that references the base stream at a specific offset
The daemon “collects” all the partial streams into a composite stream for each file in the archive that spans multiple blocks
The payload is split into chunks, with each chunk actually being a partial stream of the composite file stream
Each chunk is read from the stream, and sent to the device being updated
Sooo…. We never actually read the firmware payload from the cabinet file descriptor until we actually send the chunk of payload to the hardware. This means we have to seek() all over the place, possibly many times for each chunk, but in the kernel a seek() is really just doing some pointer maths to a memory buffer and so it’s super quick — even faster in real time than the “simple” process we used in 1_9_X. The only caveat is that you have to use uncompressed cabinet archives (the default for the LVFS) — as using MSZIP decompression currently does need a single copy fallback.
This means we can deploy a 1.5GB firmware payload using an amazingly low 8MB of RSS, and using less CPU that copying 1.5GB of data around a few times. Which means, you can now deploy that huge firmware to that $3,000 meeting room camera from a $200 ChromeBook — but also means we can do the same in RHEL for 5G mobile broadband radios on low-power, low-cost IoT hardware.
Making such huge changes to fwupd meant we could justify branching a new release, and because we bumped the major version it also made sense to remove all the deprecated API in libfwupd. All the changes are documented in the README file, but I’ve already sent patches for gnome-firmware, gnome-software and kde-discover to make the tiny changes needed for the library bump.
My plan for 2.0.x is to ship it in Flathub, and in Fedora 42 — but NOT Fedora 41, RHEL 9 or RHEL 10 just yet. There is a lot of new code that’s only had a little testing, and I fully expect to do a brown paperbag 2.0.1 release in a few days because we’ve managed to break some hardware for some vendor that I don’t own, or we don’t have emulations for. If you do see anything that’s weird, or have hardware that used to be detected, and now isn’t — please let us know.
TLDR: if you know what EVIOCREVOKE does, the same now works for hidraw devices via HIDIOCREVOKE.
The HID standard is the most common hardware protocol for input devices. In the Linux kernel HID is typically translated to the evdev protocol which is what libinput and all Xorg input drivers use. evdev is the kernel's input API and used for all devices, not just HID ones.
evdev is mostly compatible with HID but there are quite a few niche cases where they differ a fair bit. And some cases where evdev doesn't work well because of different assumptions, e.g. it's near-impossible to correctly express a device with 40 generic buttons (as opposed to named buttons like "left", "right", ...[0]). In particular for gaming devices it's quite common to access the HID device directly via the /dev/hidraw nodes. And of course for configuration of devices accessing the hidraw node is a must too (see Solaar, openrazer, libratbag, etc.). Alas, /dev/hidraw nodes are only accessible as root - right now applications work around this by either "run as root" or shipping udev rules tagging the device with uaccess.
evdev too can only be accessed as root (or the input group) but many many moons ago when dinosaurs still roamed the earth (version 3.12 to be precise), David Rheinsberg merged the EVIOCREVOKE ioctl. When called the file descriptor immediately becomes invalid, any further reads/writes will fail with ENODEV. This is a cornerstone for systemd-logind: it hands out a file descriptor via DBus to Xorg or the Wayland compositor but keeps a copy. On VT switch it calls the ioctl, thus preventing any events from reaching said X server/compositor. In turn this means that a) X no longer needs to run as root[1] since it can get input devices from logind and b) X loses access to those input devices at logind's leisure so we don't have to worry about leaking passwords.
Real-time forward to 2024 and kernel 6.12 now gained the HIDIOCREVOKE for /dev/hidraw nodes. The corresponding logind support has also been merged. The principle is the same: logind can hand out an fd to a hidraw node and can revoke it at will so we don't have to worry about data leakage to processes that should not longer receive events. This is the first of many steps towards more general HID support in userspace. It's not immediately usable since logind will only hand out those fds to the session leader (read: compositor or Xorg) so if you as application want that fd you need to convince your display server to give it to you. For that we may have something like the inputfd Wayland protocol (or maybe a portal but right now it seems a Wayland protocol is more likely).
But that aside, let's hooray nonetheless. One step down, many more to go.
One of the other side-effects of this is that logind now has an fd to any device opened by a user-space process. With HID-BPF this means we can eventually "firewall" these devices from malicious applications: we could e.g. allow libratbag to configure your mouse' buttons but block any attempts to upload a new firmware. This is very much an idea for now, there's a lot of code that needs to be written to get there. But getting there we can now, so full of optimism we go[2].
[0] to illustrate: the button that goes back in your browser is actually evdev's BTN_SIDE and BTN_BACK is ... just another button assigned to nothing particular by default.
[1] and c) I have to care less about X server CVEs.
[2] mind you, optimism is just another word for naïveté
Automeris Naranja and Hari Rana have continued work this week to modernise GNOME Software’s UI with libadwaita 1.6 widgets, while Sid is working on making submission of app reviews not block the UI.
For the new Identity v0.7 release, I implemented image loading via the glycin library. It lets you open newer image formats like AVIF and JPEG XL, and improves rendering for other images, like using the correct color spaces for regular JPEGs.
It’s a simple utility for the calculation and analysis of IP subnet values, designed to simplify network configuration tasks. There’s no longer a need to rely on various ad-laden websites with awkward designs, as Netsleuth is native and available offline.
The lock screen now adjusts to smaller resolutions and we fixed a whole lot of bugs around (but not limited to) OSK text input when using text prediction - up to a point where we now enable it by default when the app requests it (and the OSK supports it). squeekboard saw another round of layout improvements and additions.
This update features a brand new rewrite of Parabolic in C++. Users should now have a faster and more stable downloading experience, with the continued reliability and customizability loved by our users.
We have also redesigned the user interfaces on both the GNOME and Windows platforms to make it easier to find the Parabolic features you love and want to use. We have also refined and improved the options available when configuring individual downloads, playlists, and Parabolic as a whole. Besides new things, we have fixed tons of bugs throughout the downloading backend that users have been waiting for.
We hope you all love this release as much as we loved making it! A huge thank you to all users who worked with us in fixing issues, testing the betas, contributing translations and more ❤️
Here’s the full changelog:
Parabolic has been rewritten in C++ for faster performance
The Keyring module was rewritten. As a result, all keyrings have been reset and will need to be reconfigured
Audio languages with audio description are now correctly recognized and handled separately from audio languages without audio description
Audio download qualities will now list audio bitrates for the user to choose from
Playlist downloads will now be saved in a subdirectory with the playlist’s title within the chosen save folder
When viewing the log of a download, the command used to run the download can now also be copied to the clipboard
The length of the kept download history can now be changed in Preferences
On non-sandbox platforms, a browser can be selected for Parabolic to fetch cookies from instead of selecting a txt file in Preferences
Added an option in Preferences to allow for immediate download after a URL is validated
Added an option in Preferences to pick a preferred video codec for when downloading video media
Fixed validation issues with various sites
Fixed an issue where a specified video password was not being used
Redesigned user interface
Updated yt-dlp
Flatseal
A graphical utility to review and modify permissions of Flatpak applications.
Flatseal 2.3.0 is out! This new release has caught up with new Flatpak permissions, added Hindi translation thanks to Scrambled777, and updated its runtime to GNOME 47. Make sure to keep your GNOME runtime up-to-date to prevent some common issues!
(Almost) all of my Gnome extensions do support Gnome 47 now. What’s even better is that all the updates came from contributors, so all I had to do was to merge a couple of PRs. Thanks!
GNOME Asia 2024 is happening in Bengaluru, India between 6th to 8th of December. We’re happy to announce that the call for proposals is now open and will close on October 8th, 2024. This is your chance to submit a proposal and speak at the conference!
This weekend, from Oct 4 to Oct 5, the Linux App Summit is taking place in Monterrey, Mexico. LAS is dedicated to collaboration on all aspects aimed at accelerating the growth of the Linux application ecosystem.
The last months, the GNOME Foundation Board of directors has been busy with budgeting and other strategic decisions, and we’ve written a small update to keep everybody informed: https://discourse.gnome.org/t/update-from-the-board-2024-10/24346 We will follow-up with a more detailed discussion of the budget in the following weeks
That’s all for this week!
See you next week, and be sure to stop by #thisweek:gnome.org with updates on your own projects!
When you have a generational collector, you aim to trace only the part
of the object graph that has been allocated recently. To do so, you
need to keep a remembered set: a set of old-to-new edges, used as
roots when performing a minor collection. A language run-time maintains this set by
adding write barriers: little bits of collector code that run when a
mutator writes to a field.
Whippet’s nofl
space
is a block-structured space that is appropriate for use as an old
generation or as part of a sticky-mark-bit generational collector. It
used to have a card-marking write barrier; see my article diving into V8’s new
write
barrier,
for more background.
Unfortunately, when running whiffle
benchmarks, I was seeing no improvement for generational configurations relative to whole-heap collection. Generational collection
was doing fine in my tiny microbenchmarks that are part of Whippet
itself, but when translated to larger programs (that aren’t yet proper
macrobenchmarks), it was a lose.
I had planned on doing some serious tracing and instrumentation to
figure out what was happening, and thereby correct the problem. I still
plan on doing this, but instead for this issue I used the old noggin
technique instead: just, you know, thinking about the thing, eventually concluding that unconditional card-marking barriers are inappropriate
for sticky-mark-bit collectors. As I mentioned in the earlier article:
An unconditional card-marking barrier applies to stores to slots in
all objects, not just those in oldspace; a store to a new object will
mark a card, but that card may contain old objects which would then
be re-scanned. Or consider a store to an old object in a more dense
part of oldspace; scanning the card may incur more work than
needed. It could also be that Whippet is being too aggressive at
re-using blocks for new allocations, where it should be limiting
itself to blocks that are very sparsely populated with old objects.
That’s three problems. The second is well-known. But the first and
last are specific to sticky-mark-bit collectors, where pages mix old and
new objects.
a precise field-logging write barrier
Back in 2019, Steve Blackburn’s paper Design and Analysis of
Field-Logging Write
Barriers
took a look at the state of the art in precise barriers that record not
regions of memory that have been updated, but the precise edges (fields)
that were written to. He ends up re-using this work later in the 2022
LXR paper
(see §3.4), where the write barrier is used for deferred reference counting and a
snapshot-at-the-beginning (SATB) barrier for concurrent marking. All in
all field-logging seems like an interesting strategy. Relative to card-marking,
work during the pause is much less: you have a precise buffer of all
fields that were written to, and you just iterate that, instead of
iterating objects. Field-logging does impose some mutator cost, but perhaps the
payoff is worth it.
To log each old-to-new edge precisely once, you need a bit per field
indicating whether the field is logged already. Blackburn’s 2019 write
barrier paper used bits in the object header, if the object was small
enough, and otherwise bits before the object start. This requires some
cooperation between the collector, the compiler, and the run-time that I
wasn’t ready to pay for. The 2022 LXR paper was a bit vague on this
topic, saying just that it used “a side table”.
Iterability / interior pointers: is there an object at a given
address? If so, it will have a recognizable bit pattern.
End of object, to be able to sweep without inspecting the object
itself
Pinning, allowing a mutator to prevent an object from being
evacuated, for example because a hash code was computed from its
address
A hack to allow fully-conservative tracing to identify ephemerons at
trace-time; this re-uses the pinning bit, since in practice such
configurations never evacuate
Bump-pointer allocation into holes: the mark byte table serves the
purpose of Immix’s line mark byte table, but at finer granularity.
Because of this though, it is swept lazily rather than eagerly.
Generations. Young objects have a bit set that is cleared when they
are promoted.
Well. Why not add another thing? The nofl space’s granule size is two
words, so we can use two bits of the byte for field logging bits. If
there is a write to a field, a barrier would first check that the object
being written to is old, and then check the log bit for the field being
written. The old check will be to a byte that is nearby or possibly the
same as the one to check the field logging bit. If the bit is unsert,
we call out to a slow path to actually record the field.
preliminary results
I disassembled the fast path as compiled by GCC and got something like
this on x86-64, in AT&T syntax, for the young-generation test:
mov %rax,%rdx
and $0xffffffffffc00000,%rdx
shr $0x4,%rax
and $0x3ffff,%eax
or %rdx,%rax
testb $0xe,(%rax)
The first five instructions compute the location of the mark byte, from
the address of the object (which is known to be in the nofl space). If
it has any of the bits in 0xe set, then it’s in the old generation.
Then to test a field logging bit it’s a similar set of instructions. In
one of my tests the data type looks like this:
struct Node {
uintptr_t tag;
struct Node *left;
struct Node *right;
int i, j;
};
Writing the left field will be in the same granule as the object
itself, so we can just test the byte we fetched for the logging bit
directly with testb against $0x80. For right, we should be able
to know it’s in the same slab (aligned 4 MB region) and just add to the
previously computed byte address, but the C compiler doesn’t know that
right now and so recomputes. This would work better in a JIT. Anyway I
think these bit-swizzling operations are just lost in the flow of memory
accesses.
For the general case where you don’t statically know the offset of the
field in the object, you have to compute which bit in the byte to test:
mov %r13,%rcx
mov $0x40,%eax
shr $0x3,%rcx
and $0x1,%ecx
shl %cl,%eax
test %al,%dil
Is it good? Well, it improves things for my whiffle benchmarks,
relative to the card-marking barrier, seeing a 1.05×-1.5× speedup across
a range of benchmarks. I suspect the main advantage is in avoiding the
“unconditional” part of card marking, where a write to a new
object could cause old objects to be added to the remembered set. There are still quite a
few whiffle configurations in which the whole-heap collector outperforms
the sticky-mark-bit generational collector, though; I hope to understand
this a bit more by building a more classic semi-space nursery, and
comparing performance to that.
Implementation links: the barrier
fast-path,
the slow
path,
and the sequential store
buffers.
(At some point I need to make it so that allocating edge buffers in the
field set causes the nofl space to page out a corresponding amount of
memory, so as to be honest when comparing GC performance at a fixed heap
size.)
I'm happy to announce that the last tweaks have landed and that the fully FOSS libcamera software ISP based IPU6 camera support in Fedora 41 now has no known bugs left. See the Changes page for testing instructions.
Supported hardware
Unlike USB UVC cameras where all cameras work with a single kernel driver, MIPI cameras like the Intel IPU6 cameras require multiple drivers. The IPU6 input-system CSI receiver driver is common to all laptops with an IPU6 camera, but different laptops use different camera sensors and each sensor needs its own driver and then there are glue ICs like the LJCA USB IO-expander and the iVSC (Intel Visual Sensing Controller) and there also is the ipu-bridge code which translates Windows oriented ACPI tables with sensor info into the fwnodes which the Linux drivers expect.
This means that even though IPU6 support has landed in Fedora 41 not all laptops with an IPU6 camera will work. Currently the IPU6 integrated in the following CPU models works if the sensor + glue hw/sw is also supported:
Tiger Lake
Alder Lake
Raptor Lake
Jasper Lake and Meteor Lake also have an IPU6 but there is some more integration work necessary to get things to work there. Getting Meteor Lake IPU6 cameras to work is high on my TODO list.
The mainline kernel IPU6 CSI receiver + libcamera software ISP has been successfully tested on the following models:
Various Lenovo ThinkPad models with ov2740 (INT3474) sensor (1)
Various Dell models with ov01a10 (OVTI01A0) sensor
Dell XPS 13 PLus with ov13b10 (OVTIDB10/OVTI13B1)
Some HP laptops with hi556 sensor (INT3537)
To see which sensor your laptop has run: "ls /sys/bus/i2c/devices" this will show e.g. "i2c-INT3474:00" if you have an ov2740, with INT3474 being the ACPI Hardware ID (HID) for the sensor. See here for a list of currently known HID to sensor mappings. Note not all of these have upstream drivers yet. In that cases chances are that there might be a sensor driver for your sensor here.
We could really use help with people submitting drivers from there upstream. So if you have a laptop with a sensor which is not in the mainline but is available there, you know a bit of C-programming and you are willing to help, then please drop me an email so that we can work together to get the driver upstream.
1) on some ThinkPads the ov2740 sensor fails to start streaming most of the time. I plan to look into this next week and hopefully I can come up with a fix.
MIPI camera Integration work done for Fedora 41
After landing the kernel IPU6 CSI receiver and libcamera software ISP support upstream early in the Fedora 41 cycle, there still was a lot of work to do with regards to integrating this into the rest of the stack so that the cameras can actually be used outside of the qcam test app.
The whole stack looks like this "kernel → libcamera → pipewire | pipewire-camera-consuming-app". Where the 2 currently supported pipewire-camera consuming apps are Firefox and GNOME Snapshot.
Once this was all up and running testing found quite a few bugs which have all been fixed now:
Firefox showing 13 different cameras in its camera selection pulldown for a single IPU6 camera (fix).
Installing pipewire-plugin-libcamera leads to UVC cameras being powered on all the time causing significant battery drain (bug, bug, discussion, fix).
Pipewire does not always recognizes cameras on login (bug, bug, bug, fix).
Pipewire fails to show cameras with relative controls (fix).
spa_libcamera_buffer_recycle sometimes fails, causing stream to freeze on first frame (bug, fix)
Firefox chooses bad default resolution of 640x480. I worked with Jan Grulich to get this fixed and this is fixed as of firefox-130.0.1-3.fc41. Thank you Jan!
Snapshot prefers 4:3 mode, e.g. 1280x1080 on 16:9 camera sensors capable of 1920x1080 (pending fix)
Added intel-vsc-firmware, pipewire-plugin-libcamera, libcamera-ipa to the Fedora 41 Workstation default package-set (pull, pull, pull)
It’s been over two months but I still haven’t gotten around to writing a blog post about this year’s Berlin Mini GUADEC. I still don’t have time to write a longer post, but instead of putting this off forever I thought I’d at least share a few photos.
Overall I think our idea of running this as a self-organized event worked out great. The community (both Berlin locals and other attendees) really came together to make it a success, despite the difficult circumstances. Thanks in particular to Jonas Dreßler for taking care of recording and streaming the talks, Ondřej Kolín and Andrei Zisu for keeping things on track during the event, and Sonny Piers for helping with various logistical things before the event.
Thanks to everyone who helped to make it happen, and see you next year!
WebKitGTK and WPEWebKit recently released a new stable version 2.46. This version includes important changes in the graphics implementation.
Skia
The most important change in 2.46 is the introduction of Skia to replace Cairo as the 2D graphics renderer. Skia supports rendering using the GPU, which is now the default, but we also use it for CPU rendering using the same threaded rendering model we had with Cairo. The architecture hasn’t changed much for GPU rendering: we use the same tiled rendering approach, but buffers for dirty regions are rendered in the main thread as textures. The compositor waits for textures to be ready using fences and copies them directly to the compositor texture. This was the simplest approach that already resulted in much better performance, specially in the desktop with more powerful GPUs. In embedded systems, where GPUs are not so powerful, it’s still better to use the CPU with several rendering threads in most of the cases. It’s still too early to announce anything, but we are already experimenting with different models to improve the performance even more and make a better usage of the GPU in embedded devices.
Skia has received several GCC specific optimizations lately, but it’s always more optimized when built with clang. The optimizations are more noticeable in performance when using the CPU for rendering. For this reason, since version 2.46 we recommend to build WebKit with clang for the best performance. GCC is still supported, of course, and performance when built with GCC is quite good too.
HiDPI
Even though there aren’t specific changes about HiDPI in 2.46, users of high resolution screens using a device scale factor bigger than 1 will notice much better performance thanks to scaling being a lot faster on the GPU.
Accelerated canvas
The 2D canvas can be accelerated independently on whether the CPU or the GPU is used for painting layers. In 2.46 there’s a new setting WebKitSettings:enable-2d-canvas-acceleration to control the 2D canvas acceleration. In some embedded devices the combination of CPU rendering for layer tiles and GPU for the canvas gives the best performance. The 2D canvas is normally rendered into an image buffer that is then painted in the layer as an image. We changed that for the accelerated case, so that the canvas is now rendered into a texture that is copied to a compositor texture to be directly composited instead of painted into the layer as an image. In 2.46 the offscreen canvas is enabled by default.
There are more cases where accelerating the canvas is not desired, for example when the canvas size is not big enough it’s faster to use the GPU. Also when there’s going to be many operations to “download” pixels from GPU. Since this is not always easy to predict, in 2.46 we added support for the willReadFrequently canvas setting, so that when set by the application when creating the canvas it causes the canvas to be always unaccelerated.
Filters
All the CSS filters are now implemented using Skia APIs, and accelerated when possible. The most noticeable change here is that sites using blur filters are no longer slow.
Color spaces
Skia brings native support for color spaces, which allows us to greatly simplify the color space handling code in WebKit. WebKit uses color spaces in many scenarios – but especially in case of SVG and filters. In case of some filters, color spaces are necessary as some operations are simpler to perform in linear sRGB. The good example of that is feDiffuseLighting filter – it yielded wrong visual results for a very long time in case of Cairo-based implementation as Cairo doesn’t have a support for color spaces. At some point, however, Cairo-based WebKit implementation has been fixed by converting pixels to linear in-place before applying the filter and converting pixels in-place back to sRGB afterwards. Such a workarounds are not necessary anymore as with Skia, all the pixel-level operations are handled in a color-space-transparent way as long as proper color space information is provided. This not only impacts the results of some filters that are now correct, but improves performance and opens new possibilities for acceleration.
Font rendering
Font rendering is probably the most noticeable visual change after the Skia switch with mixed feedback. Some people reported that several sites look much better, while others reported problems with kerning in other sites. In other cases it’s not really better or worse, it’s just that we were used to the way fonts were rendered before.
Damage tracking
WebKit already tracks the area of the layers that has changed to paint only the dirty regions. This means that we only repaint the areas that changed but the compositor incorporates them and the whole frame is always composited and passed to the system compositor. In 2.46 there’s experimental code to track the damage regions and pass them to the system compositor in addition to the frame. Since this is experimental it’s disabled by default, but can be enabled with the runtime feature PropagateDamagingInformation. There’s also UnifyDamagedRegions feature that can be used in combination with PropagateDamagingInformation to unify the damage regions into one before passing it to the system compositor. We still need to analyze the impact of damage tracking in performance before enabling it by default. We have also started an experiment to use the damage information in WebKit compositor and avoid compositing the entire frame every time.
GPU info
Working on graphics can be really hard in Linux, there are too many variables that can result in different outputs for different users: the driver version, the kernel version, the system compositor, the EGL extensions available, etc. When something doesn’t work for some people and work for others, it’s key for us to gather as much information as possible about the graphics stack. In 2.46 we have added more useful information to webkit://gpu, like the DMA-BUF buffer format and modifier used (for GTK port and WPE when using the new API). Very often the symptom is the same, nothing is rendered in the web view, even when the causes could be very different. For those cases, it’s even more difficult to gather the info because webkit://gpu doesn’t render anything either. In 2.46 it’s possible to load webkit://gpu/stdout to get the information as a JSON directly in stdout.
Sysprof
Another common symptom for people having problems is that a particular website is slow to render, while for others it works fine. In these cases, in addition to the graphics stack information, we need to figure out where we are slower and why. This is very difficult to fix when you can’t reproduce the problem. We added initial support for profiling in 2.46 using sysprof. The code already has some marks so that when run under sysprof we get useful information about timings of several parts of the graphics pipeline.
Next
This is just the beginning, we are already working on changes that will allow us to make a better use of both the GPU and CPU for the best performance. We have also plans to do other changes in the graphics architecture to improve synchronization, latency and security. Now that we have adopted sysprof for profiling, we are also working on improvements and new tools.
I am pleased to announce a new Cambalache stable release, version 0.92.0!
This comes with two major dependencies changes, the first one is a very basic port to Adwaita and webkit/broadway replacement with a custom Wayland compositor widget based on wlroots.
What’s new:
Basic port to Adwaita
Use Casilda compositor widget for workspace
Update widget catalogs to SDK 47
Improved Drag&Drop support
Improve workspace performance
Enable workspace animations
Fix window ordering
Support new desktop dark style
Support 3rd party libraries
Streamline headerbar
Lots of bug fixes and minor improvements
Adwaita
The port to Adwaita gives Cambalache the new modern look and enables dark mode support. The headerbar is simplified only keeping most common used actions, everything else was moved to the main menu.
Cambalache editing Cambalache UICambalache editing Cambalache UI in dark mode
Casilda Compositor
Up until this release, Cambalache showed windows from a different process in its workspace running broadwayd or gtk4-broadwayd backend depending on the gtk version of your project and using a WebView to connect to it and show the windows in an HTML canvas.All of this was replaced with a simple Wayland compositor widget which reduces hard dependencies a lot.
On top of that we get all the optimizations from using Wayland instead of a protocol meant to go over the internet.
With Broadway, the client would render the window in memory, the broadway backend would compress the image and sent it over TCP to the webview which has to uncompress it and render it on an HTML5 canvas.
Now, the client just renders in shared memory which is directly available to the compositor widget to use. This also leave the option to further improve performance by adding support for dmabuf which would allow to offload the composition to the host compositor reducing the number of memory copies to show the windows on the screen.
This allowed me to re enable Gtk animations since they no longer impact the workspace performance.
Special thanks to emersion, kennylevinsen, vyivel and the wlroots community for their support and awesome project, I would not have been able to do this without wlroots and their help.
Cambalache now loads 3rd party catalogs from GLib.get_system_data_dirs()/cambalache/catalogs and ~/.cambalache/catalogs
These catalog files are generated from Gir data with a new tool bundled in Cambalache calledcmb-catalog-gen. This used to be an internal and still lacks proper documentation but you can see an example of how its used internally here
So what is a catalog anyway?
A catalog is a XML file with all the necessary data for Cambalache to produce UI files with widgets from a particular library, this includes the different GTypes, with their properties, signals and everything else except the actual object implementations.
Runtime objects are created in the workspace by loading the GI namespace specified in the catalog.
Feel free to contact me on matrix if you are interested in adding support for a 3rd party library.
Improved Drag&Drop
After the extensive rework done porting the main widget hierarchy from GtkTreeView to GtkColumnView and implementing several GListModel interfaces to avoid maintaining multiple lists I was able to reimplement and extend Drag&Drop code so now its possible to drop widgets in different parents.
Data Model
History handling for Undo/Redo was simplified from multiple history tables (one per table tracked) into one history table by adding a few extra columns to store data change in JSON format.
Hey all, I had a fun bug this week and want to share it with you.
numbers and representations
First, though, some background. Guile’s numeric operations are defined over the complex numbers, not
over e.g. a finite field of integers. This is generally great when
writing an algorithm, because you don’t have to think about how the
computer will actually represent the numbers you are working on.
In practice, Guile will represent a small exact integer as a
fixnum,
which is a machine word with a low-bit tag. If an integer doesn’t fit
in a word (minus space for the tag), it is represented as a
heap-allocated bignum. But sometimes the compiler can realize that
e.g. the operands to a specific bitwise-and operation are within (say)
the 64-bit range of unsigned integers, and so therefore we can use
unboxed operations instead of the more generic functions that do
run-time dispatch on the operand types, and which might perform heap
allocation.
Unboxing is important for speed. It’s also tricky: under what
circumstances can we do it? In the example above, there is information
that flows from defs to uses: the operands of logand are known to be
exact integers in a certain range and the operation itself is closed over
its domain, so we can unbox.
But there is another case in which we can unbox, in which information
flows backwards, from uses to defs: if we see (logand n #xff), we know:
the result will be in [0, 255]
that n will be an exact integer (or an exception will be thrown)
we are only interested in a subset of n‘s bits.
Together, these observations let us transform the more general logand
to an unboxed operation, having first truncated n to a u64. And
actually, the information can flow from use to def: if we know that n
will be an exact integer but don’t know its range, we can transform the
potentially heap-allocating computation that produces n to instead
truncate its result to the u64 range where it is defined, instead of
just truncating at the use; and potentially this information could
travel farther up the dominator tree, to inputs of the operation that
defines n, their inputs, and so on.
needed-bits: the |0 of scheme
Let’s say we have a numerical operation that produces an exact integer,
but we don’t know the range. We could truncate the result to a u64
and use unboxed operations, if and only if only u64 bits are used. So
we need to compute, for each variable in a program, what bits are needed
from it.
I think this is generally known a needed-bits analysis, though both
Google and my textbooks are failing me at the moment; perhaps this is
because dynamic languages and flow analysis don’t get so much attention
these days. Anyway, the analysis can be local (within a basic block),
global (all blocks in a function), or interprocedural (larger than a
function). Guile’s is global. Each CPS/SSA variable in the function
starts as needing 0 bits. We then compute the fixpoint of visiting each
term in the function; if a term causes a variable to flow out of the
function, for example via return or call, the variable is recorded as
needing all bits, as is also the case if the variable is an operand to
some primcall that doesn’t have a specific needed-bits analyser.
Currently, only logand has a needed-bits analyser, and this is because
sometimes you want to do modular arithmetic, for example in a hash
function. Consider Bon Jenkins’ lookup3 string hash
function:
#define rot(x,k) (((x)<<(k)) | ((x)>>(32-(k))))
#define mix(a,b,c) \
{ \
a -= c; a ^= rot(c, 4); c += b; \
b -= a; b ^= rot(a, 6); a += c; \
c -= b; c ^= rot(b, 8); b += a; \
a -= c; a ^= rot(c,16); c += b; \
b -= a; b ^= rot(a,19); a += c; \
c -= b; c ^= rot(b, 4); b += a; \
}
...
(define (jenkins-lookup3-hashword2 str)
(define (u32 x) (logand x #xffffFFFF))
(define (shl x n) (u32 (ash x n)))
(define (shr x n) (ash x (- n)))
(define (rot x n) (logior (shl x n) (shr x (- 32 n))))
(define (add x y) (u32 (+ x y)))
(define (sub x y) (u32 (- x y)))
(define (xor x y) (logxor x y))
(define (mix a b c)
(let* ((a (sub a c)) (a (xor a (rot c 4))) (c (add c b))
(b (sub b a)) (b (xor b (rot a 6))) (a (add a c))
(c (sub c b)) (c (xor c (rot b 8))) (b (add b a))
...)
...))
...
These u32 calls are like the JavaScript |0
idiom,
to tell the compiler that we really just want the low 32 bits of the
number, as an integer. Guile’s compiler will propagate that information
down to uses of the defined values but also back up the dominator tree,
resulting in unboxed arithmetic for all of these operations.
All that was just prelude. So I said that needed-bits is a fixed-point
flow analysis problem. In this case, I want to compute, for each
variable, what bits are needed for its definition. Because of loops, we
need to keep iterating until we have found the fixed point. We use a
worklist to represent the conts we need to visit.
Visiting a cont may cause the program to require more bits from the
variables that cont uses.
Consider:
(define-significant-bits-handler
((logand/immediate label types out res) param a)
(let ((sigbits (sigbits-intersect
(inferred-sigbits types label a)
param
(sigbits-ref out res))))
(intmap-add out a sigbits sigbits-union)))
This is the sigbits (needed-bits) handler for logand when one of its
operands (param) is a constant and the other (a) is variable. It
adds an entry for a to the analysis out, which is an intmap from
variable to a bitmask of needed bits, or #f for all bits. If a
already has some computed sigbits, we add to that set via
sigbits-union. The interesting point comes in the sigbits-intersect
call: the bits that we will need from a are first the bits that we
infer a to have, by forward type-and-range analysis; intersected with
the bits from the immediate param; intersected with the needed bits
from the result value res.
If the intmap-add call is idempotent—i.e., out already contains
sigbits for a—then out is returned as-is. So we can check for a
fixed-point by comparing out with the resulting analysis, via eq?.
If they are not equal, we need to add the cont that defines a to the
worklist.
The bug? The bug was that we were not enqueuing the def of a, but
rather the predecessors of label. This works when there are no
cycles, provided we visit the worklist in post-order; and regardless, it
works for many other analyses in Guile where we compute, for each
labelled cont (basic block), some set of facts about all other labels
or about all other
variables.
In that case, enqueuing a predecessor on the worklist will cause all
nodes up and to including the variable’s definition to be visited,
because each step adds more information (relative to the analysis
computed on the previous visit). But it doesn’t work for this case,
because we aren’t computing a per-label analysis.
GUADEC was in Denver this year! I meant to write an update right after the conference, but Real Life got in the way and it took a while to finish this post. I finally found a little spare time to collect my thoughts and finish writing this.
It was a smaller crowd than normal this year. There were ~100 people registered, though unfortunately a number of people were unable to make it at the last minute due to Cloudstrike– and visa– related issues.
Denver City Hall
I gave two talks: Crosswords, Year Three (slides) and a spur-of-the-moment lightning talk on development docs. The first talk was nominally about authoring crosswords, but I also presented the architecture we used to create the game. Although rushed, I hope I got most of the points about our design across. It’s definitely worth a full blog post at a future date.
Other highlights of the conference included Martin’s very funny (and brave) live demo of gameeky, Scott’s talk about being bold with design, the AGM, and a fabulous Thunderbird keynote about the power of money. That last one spurred conversations about putting a fundraising request popup in GNOME itself to raise funds. The yearly popup in Thunderbird appears to continue being wildly successful. Since GUADEC, I see that KDE has attempted to do that as well. I’d love for GNOME to do something similar. Maybe this is something the new board can pick up.
Original Nikolai Tesla generator in the Tivoli Brewing Co.
It was a very chill GUADEC, and I enjoyed the change of pace. I had never spent time in Denver (other than at the airport), and found it to be a surprisingly intimate city with a very walkable downtown. The venue was absolutely fabulous. Every conference should have a pub on-site, and the Tivoli Brewing Co definitely surpassed expectations. It even has an original Nikolai Tesla generator in its basement.
Reflections
It was really nice having GUADEC relatively close to me for once. There was a different crowd than normal: there were long-time GNOME people I haven’t seen in a very long time (Hi Owen, Behdad, and Michael!) as well as numerous new folks (welcome Richard!) Holding it in North America opened us up to different contributors, and maybe let us reengage with long-time gnomies.
Let’s not pretend that a video conference or a hybrid BOF is the same as an in-person meetup. Once you’ve sung karaoke with someone, or explored Meow Wolf, or camped in the desert in Utah together, your relationship is richer than when you only interacted via Gitlab pull requests and BigBlueButton. You have more empathy and you can resolve conflicts better.
Fragmentation is always a danger with distributed endeavors and any group bigger than two will have politics, but it feels like our best tool to deal with those issues is fragmenting too.
Personally, as someone who has schlepped across the Atlantic for over two decades to meet with other folks, it doesn’t feel great to have comparatively few people come the other direction. There are a plenty of good individual decisions that lead to this, but collectively it felt like a misfire.
I also really appreciate the commitment of our South American / Asian / African developers who have tough travel routes to get to the Euro/American events.
The first GUADEC poster
In some sense, it feels like we’ve gone full-circle. When GNOME started, development was strongly centered in North America. The GIMP started in Berkeley, and GNOME itself was founded in Mexico, and there were quite a few other pockets of GNOME activity (Boston, North Carolina, etc). Proportionally, Europe was underrepresented — so GUADEC was proposed as a way to build a European community. It took sustained engagement to build it up. Twenty-four years on, it appears we need to do the reverse.
What’s next? Well for me, it’s time to look more local. We used to have a Bay Area GNOME community and it has fallen on hard times. Maybe it’s worth trying to push some local enthusiasm. If you’re a Bay Area GNOME person, drop me a note. We should hold a release party!
Nonograms
While in Denver, ptomato and I nerd-sniped each other into writing a nonogram game. Nonograms are a popular puzzle-type, and are quite common on existing mobile platforms. Conceptually, they’re pen-and-paper grid-based games and could easily be implemented as an .ipuz extension.
I’ve been slowly changing the libipuz API over the summer to work with gobject-introspection, and was excited at the chance to get someone to test it out. Meanwhile, Philip had been wanting to write an app with typescript. So, I sketched out an extension and put together an API for Philip to use. With a little back-and-forth, he got something to render. Exciting!
I don’t think it is playable yet but it’s lovely to see the potential emerging. There’s a lot of great pixel art floating around GNOME. Some of it might make the basis for a really fun nonogram game.
As a bonus, Philip has been experimenting with using the stateless design we use in Crosswords. I’m hoping he’ll be able to provide additional validation and feedback to our architectural approach.
With GNOME 47 out, it’s time for my bi-annual wallpaper deep dive. For many, these may seem like simple background images, but GNOME wallpapers are the visual anchors of the project, defining its aesthetic and identity. The signature blue wallpaper with its dark top bar remains a key part of that.
In this release, GNOME 47 doesn’t overhaul the default blue wallpaper. It’s more of a subtle tweak than a full redesign. The familiar rounded triangles remain, but here’s something neat: the dark variant mimics real-world camera behavior. When it’s darker, the camera’s aperture widens, creating a shallower depth of field. A small but nice touch for those who notice these things.
The real action this cycle, though, is in the supplemental wallpapers.
We haven’t had to remove much this time around, thanks to the JXL format keeping file sizes manageable. The focus has been on variety rather than cutting old designs. We aim to keep things fresh, though you might notice that photographic wallpapers are still missing (we’ll get to that eventually, promise.
In terms of fine tuning changes, the classic, Pixels has been updated to feature newer apps from GNOME Circle.
The dark variant of Pills also got some love with lighting and shading tweaks, including a subtle subsurface scattering effect.
As for the new wallpapers, there are a few cool additions this release. I collaborated with Dominik Baran to create a tube-map-inspired vector wallpaper, which I’m particularly into. There’s also Mollnar, a nod to Vera Molnar, using simple geometric shapes in SVG format.
Most of our wallpapers are still bitmaps, largely because our rendering tools don’t yet handle color banding well with vectors. For now, even designs that would work better as vectors—like mesh gradients—get converted to bitmaps.
We’ve introduced some new abstract designs as well – meet Sheet and Swoosh. And for fans of pixel art, we’ve added LCD and its colorful sibling, LCD-rainbow. Both give off that retro screen vibe, even if the color gradient realism isn’t real-world accurate.
Lastly, there’s Symbolic Soup, which is, well… a bit chaotic. It might not be everyone’s cup of tea, but it definitely adds variety.
Preview
If you’re wondering about the strange square aspect ratio, take a look at the wallpaper sizing guide in our GNOME Interface Guidelines.
Also worth noting is the fact that all of these wallpapers have been created by humans. While I’ve experimented with image generation for some parts of the workflow in some of of my personal projects, all this work is AIgen-free and explicitly credited.
As a JavaScript engine developer at Igalia I don’t find myself writing much plain C code anymore. I’m either writing JS or TypeScript, or hacking on large compiler codebases in C++1, or writing ECMAScript specification language. Frankly, that is fine with me. C’s time may not be over yet, but I wouldn’t be sad if I never had to write another line of it. (Hopefully this post conveys why.)
However, while working on modernizing an app written in C for the GNOME platform, that I hack on in my spare time, I wanted to copy a folder recursively using the GIO async APIs. Like cp -R at the shell, but without freezing up your GUI while it works.
C’s callback style for async programming, combined with lack of capturing variables in closures, is like going back to the dark ages if you’ve gotten used to languages with async/await style or even C++’s lambdas. I would’ve avoided writing this if I could, but apparently no one else had done it publicly on the internet that I could find.2 So here it is for your enjoyment.
typedef struct {
GFile *dest_folder;
GQueue *files_to_copy;
GQueue *folders_to_copy;
GFileCopyFlags flags;
} CopyRecursiveClosure;
/* Pre-declare so we can read them in the order they are executed: */
static void on_recursive_make_dir_finish(GFile *file, GAsyncResult *res, GTask *data);
static void on_recursive_file_enumerate_finish(GFile* file, GAsyncResult *res, GTask *data);
static void on_recursive_file_next_files_finish(GFileEnumerator *children, GAsyncResult *res, GTask *data);
static void copy_file_queue_async(GTask *task);
static void on_recursive_file_copy_finish(GFile *file, GAsyncResult *result, GTask *data);
static void on_recursive_folder_copy_finish(GFile *file, GAsyncResult *result, GTask *data);
static void copy_folder_queue_async(GTask *task);
static void copy_recursive_closure_free(CopyRecursiveClosure *ptr);
/**
* copy_recursive_async:
* @src: The source folder
* @dest: Destination folder in which to place the copy of @src
* @flags: #GFileCopyFlags to apply to copy operations
* @prio: I/O priority, e.g. #G_PRIORITY_DEFAULT
* @cancel: #GCancellable that will interrupt the operation when triggered
* @done_cb: Function to call when the operation is finished
* @data: Pointer to pass to @done_cb
*
* Copy the folder @src and all of the files and subfolders in it into the
* folder @dest, asynchronously.
*
* The only @flags supported are #G_FILE_COPY_NONE and #G_FILE_COPY_OVERWRITE.
*/
void
copy_recursive_async(GFile *src, GFile *dest, GFileCopyFlags flags, int prio, GCancellable *cancel,
GAsyncReadyCallback done_cb, void *data)
{
g_return_if_fail(G_IS_FILE(src));
g_return_if_fail(G_IS_FILE(dest));
g_return_if_fail(flags == G_FILE_COPY_NONE || flags == G_FILE_COPY_OVERWRITE);
g_return_if_fail(!cancel || G_IS_CANCELLABLE(cancel));
g_autoptr(GTask) task = g_task_new(src, cancel, done_cb, data);
g_task_set_priority(task, prio);
CopyRecursiveClosure *task_data = g_new0(CopyRecursiveClosure, 1);
g_autofree char *basename = g_file_get_basename(src);
task_data->dest_folder = g_file_get_child(dest, basename);
task_data->files_to_copy = g_queue_new();
task_data->folders_to_copy = g_queue_new();
task_data->flags = flags;
g_task_set_task_data(task, task_data, (GDestroyNotify)copy_recursive_closure_free);
g_file_make_directory_async(task_data->dest_folder, prio, cancel,
(GAsyncReadyCallback)on_recursive_make_dir_finish, g_steal_pointer(&task));
}
/**
* copy_recursive_finish:
* @src: The source folder
* @result: The #GAsyncResult passed to the callback
* @error_out: (nullable): Return location for a #GError
*
* Complete the asynchronous copy operation started by copy_recursive_async().
*
* Returns: %TRUE if the operation completed successfully, %FALSE on error.
*/
bool
copy_recursive_finish(GFile *src, GAsyncResult *result, GError **error_out)
{
g_return_val_if_fail(G_IS_FILE(src), false);
g_return_val_if_fail(G_IS_TASK(result), false);
g_return_val_if_fail(g_task_is_valid(result, src), false);
return g_task_propagate_boolean(G_TASK(result), error_out);
}
static void
on_recursive_make_dir_finish(GFile *file, GAsyncResult *result, GTask *task_ptr)
{
g_autoptr(GTask) task = g_steal_pointer(&task_ptr);
g_autoptr(GError) error = NULL;
GCancellable *cancel = g_task_get_cancellable(task);
int prio = g_task_get_priority(task);
if (!g_file_make_directory_finish(G_FILE(file), result, &error)) {
/* With the OVERWRITE flag, don't error out when the folder already
* exists. (Hopefully plopping all the files in the existing folder is
* sufficient. If not, another way to do this would be to delete the
* existing folder recursively, so that extra existing files not in the
* source don't remain in the destination.) */
CopyRecursiveClosure *data = g_task_get_task_data(task);
bool overwrite = !!(data->flags & G_FILE_COPY_OVERWRITE);
if (!overwrite || !g_error_matches(error, G_IO_ERROR, G_IO_ERROR_EXISTS)) {
g_autofree char *path = g_file_get_path(file);
g_task_return_prefixed_error(task, g_steal_pointer(&error),
"Error creating destination folder %s: ", path);
return;
}
}
GFile *src = g_task_get_source_object(task);
g_file_enumerate_children_async(src, "standard::*", G_FILE_QUERY_INFO_NONE, prio, cancel,
(GAsyncReadyCallback)on_recursive_file_enumerate_finish, g_steal_pointer(&task));
}
static void
on_recursive_file_enumerate_finish(GFile *file, GAsyncResult *result, GTask *task_ptr)
{
g_autoptr(GTask) task = g_steal_pointer(&task_ptr);
g_autoptr(GError) error = NULL;
GCancellable *cancel = g_task_get_cancellable(task);
int prio = g_task_get_priority(task);
g_autoptr(GFileEnumerator) children = g_file_enumerate_children_finish(G_FILE(file), result, &error);
if (!children) {
g_autofree char *path = g_file_get_path(file);
g_task_return_prefixed_error(task, g_steal_pointer(&error),
"Error reading folder %s: ", path);
return;
}
g_file_enumerator_next_files_async(children, 10, prio, cancel,
(GAsyncReadyCallback)on_recursive_file_next_files_finish, g_steal_pointer(&task));
}
static void
on_recursive_file_next_files_finish(GFileEnumerator *children, GAsyncResult *result, GTask *task_ptr)
{
g_autoptr(GTask) task = g_steal_pointer(&task_ptr);
g_autoptr(GError) error = NULL;
GCancellable *cancel = g_task_get_cancellable(task);
int prio = g_task_get_priority(task);
g_autolist(GFileInfo) next_files = g_file_enumerator_next_files_finish(children, result, &error);
if (error) {
g_autofree char *path = g_file_get_path(g_file_enumerator_get_container(children));
g_task_return_prefixed_error(task, g_steal_pointer(&error),
"Error reading files from folder %s: ", path);
return;
}
CopyRecursiveClosure *data = g_task_get_task_data(task);
if (next_files) {
for (GList *iter = next_files; iter != NULL; iter = g_list_next(iter)) {
GFileInfo *info = G_FILE_INFO(iter->data);
GFileType type = g_file_info_get_file_type(info);
g_autoptr(GFile) file = g_file_enumerator_get_child(children, info);
switch (type) {
case G_FILE_TYPE_DIRECTORY:
g_queue_push_tail(data->folders_to_copy, g_steal_pointer(&file));
break;
case G_FILE_TYPE_REGULAR:
g_queue_push_tail(data->files_to_copy, g_steal_pointer(&file));
break;
default:
g_warning("Unhandled file type %d in recursive copy: %s", type, g_file_info_get_name(info));
continue;
}
}
g_file_enumerator_next_files_async(children, 10, prio, cancel,
(GAsyncReadyCallback)on_recursive_file_next_files_finish, g_steal_pointer(&task));
return;
}
copy_file_queue_async(g_steal_pointer(&task));
}
static void
copy_file_queue_async(GTask *task_ptr)
{
g_autoptr(GTask) task = task_ptr;
CopyRecursiveClosure *data = g_task_get_task_data(task);
g_autoptr(GFile) file = g_queue_pop_head(data->files_to_copy);
if (file) {
GCancellable *cancel = g_task_get_cancellable(task);
int prio = g_task_get_priority(task);
g_autofree char *basename = g_file_get_basename(file);
g_autoptr(GFile) dest = g_file_get_child(data->dest_folder, basename);
g_file_copy_async(file, dest, data->flags, prio, cancel,
/* progress_callback = */ NULL, NULL,
(GAsyncReadyCallback)on_recursive_file_copy_finish, g_steal_pointer(&task));
return;
}
copy_folder_queue_async(g_steal_pointer(&task));
}
static void
on_recursive_file_copy_finish(GFile *file, GAsyncResult *result, GTask *task_ptr)
{
g_autoptr(GTask) task = task_ptr;
g_autoptr(GError) error = NULL;
if (!g_file_copy_finish(file, result, &error)) {
g_autofree char *path = g_file_get_path(file);
g_task_return_prefixed_error(task, g_steal_pointer(&error),
"Error copying file %s: ", path);
return;
}
copy_file_queue_async(g_steal_pointer(&task));
}
static void
copy_folder_queue_async(GTask *task_ptr)
{
g_autoptr(GTask) task = task_ptr;
CopyRecursiveClosure *data = g_task_get_task_data(task);
g_autoptr(GFile) folder = g_queue_pop_head(data->folders_to_copy);
if (folder) {
GCancellable *cancel = g_task_get_cancellable(task);
int prio = g_task_get_priority(task);
copy_recursive_async(folder, data->dest_folder, data->flags, prio, cancel,
(GAsyncReadyCallback)on_recursive_folder_copy_finish, g_steal_pointer(&task));
return;
}
g_task_return_boolean(task, true);
}
static void
on_recursive_folder_copy_finish(GFile *folder, GAsyncResult *result, GTask *task_ptr)
{
g_autoptr(GTask) task = task_ptr;
g_autoptr(GError) error = NULL;
if (!copy_recursive_finish(folder, result, &error)) {
g_autofree char *path = g_file_get_path(folder);
g_task_return_prefixed_error(task, g_steal_pointer(&error),
"Error copying folder %s: ", path);
return;
}
copy_folder_queue_async(g_steal_pointer(&task));
}
static void
copy_recursive_closure_free(CopyRecursiveClosure *ptr) {
g_object_unref(ptr->dest_folder);
g_queue_free_full(ptr->files_to_copy, g_object_unref);
g_queue_free_full(ptr->folders_to_copy, g_object_unref);
g_free(ptr);
}
You are welcome to take this code and customize it to your needs. I’m putting it into the public domain so hopefully nobody else has to go through this.
Although if you really want to, it could be improved by implementing progress callbacks like g_file_copy_async() has.
Just so you can understand what’s going on at a glance, here’s what it would look like in about 30 lines of JavaScript, with async/await style:
(This excludes the imports and calls to Gio._promisify that you would have to do; hopefully we’ll get native async operations in GNOME 48!)
[1] C++ before C++11 used to be a worse experience than C. However, I don’t have to deal with that because the three major JS engines use C++17. It’s … its own category of special, but better.
[2] No, ChatGPT couldn’t do it either; it made up GIO APIs that don’t exist. If that programming technique is on the table, then sure, it’d have been a lot easier.
I have resurrected my camera old raw thumbnailer so that I can browse
directories full of of camera raw images in Nautilus. This is version
47.0.1, because GNOME 47 is out.
Like the old one, it uses libopenraw to extract the previews from the
raw files.
But, it now supports more raw formats, and if needed will render the
raw image to generate a preview, like it has to for my old Ricoh GR
Digital II images (the one from 2007). This leverage libopenraw
0.4.0 (still in
alpha stage) that has been rewritten in Rust.
Sadly to get is in the hands of users the only good solution is a
distribution package. At the time of writing there is none, but I put
together smething that allowed me to build a package for Fedora 40 to
install on my big rig.
If you feel like it you can download the source
code from
GNOME, and the
repository is
on GNOME gitlab.
This is how it looks with Nautilus 46 with a bunch of images from my
old Olympus E-P1:
Loupe is GNOME’s default image viewer since GNOME 45. It is powered by the newly written safe image loading and editing library glycin.
What’s new in 47
With GNOME 47, Loupe version 47 is available as well. This release mostly consists of a lot of subtle changes. For JPEGs, the image rotation feature now writes the new orientation to the image file. While Loupe 46 was still defaulting to an older GTK renderer, the new version is using the same defaults as all other apps. Thanks to work by Benjamin Otte in GTK, Loupe now also handles very large images (larger than 256 megapixels) reliably on systems with limited VRAM while also increasing the loading speed.
Loupe and the underlying image loading and editing library glycin now support much better error reporting if it is not possible to load an image. The new glycin version uses a different decoder for JPEG images, improving loading speed and fixing all known compatibility issues. As part of my work on the GNOME STF grant, glycin now also provides bindings for other programming languages than Rust including C, GJS, Python, and Vala. If necessary, glycin now automatically disables its sandbox features in Flatpak development environments, simplifying development.
What we are working on
But there is more! We have already merged the first GNOME 48 features, which will be released in March 2025. Allan Day worked on a new design for overlay controls, especially zoom. This is already implemented and merged. It allows the selection of zoom levels like 100% without using keyboard shortcuts like Ctrl+1 and additionally gives the option to select arbitrary zoom levels. There is also a new experimental design for dragging images into the Loupe window.
On the more technical side, Hubert Figuière has written an initial loader implementation for raw image formats which is now merged into glycin. Last but not least, I’m planning to finally have some initial image editing features beyond image rotation in Loupe 48. I’m currently working on all the basics and an image cropping feature.
A huge thanks goes out to everyone who contributed to this work including all the people that are kind enough to support my work financially! If you want to get weekly behind-the-scenes development updates or just support my work financially, you can do so via Patreon, Ko-Fi, GitHub, or OpenCollective.
Focus stealing prevention exists for two main reasons: One is security, since we need to prevent rogue apps from deceiving users into e.g. typing their password into another window. If apps can silently claim keyboard focus and open their own window over the currently focused one, this enables phishing and other similar attacks. The other is user experience: Even if an app isn’t maliciously taking over your focus, it can be annoying to have a new window popping up while you’re typing something and have half your sentence end up in the wrong app.
At the same time there are cases where you want apps to be able to request focus, for example when clicking a link in a chat app and wanting it to open in the browser. In this case you want the focus to move to the browser window.
This is why our compositor library mutter implements focus stealing prevention mechanisms, which allow the currently focused app to request that a specific other app be allowed to claim focus now.
<App> is ready??
Most users have probably seen an “<App> is ready” notification in GNOME Shell at some point. Unfortunately this notification doesn’t really explain why it’s being shown and what’s happening, which may cause confusion.
Because of this there have been proposals to disable focus stealing prevention until it works better (mutter issue 673), and a number of GNOME Shell extensions).
These are the main cases where the notification is shown:
An app requests focus for one of its windows, but was not activated in a valid way (e.g. because it wasn’t started by a user action)
An app requests focus for a new window, but it’s slow to start and in the meantime there are additional user interactions. In this case we don’t want to interrupt, and show the notification so people can switch at their convenience.
An app is launched from an environment that isn’t able to use the XDG Activation protocol (e.g. a terminal)
The protocol responsible for this, XDG Activation, the Wayland equivalent to the X11-specific startup notification spec was introduced somewhat recently (2020), and needs to be adopted by UI toolkits. GNOME 46 and 47 saw a few fixes and the feature was polished both in the client toolkit side (GTK and xdg-desktop-portal, as well as in the compositor implementation mutter, but there are still cases where XDG activation isn’t hooked up properly.
How XDG activation works
XDG activation flow for moving focus between two existing windows
The way the protocol works is that the currently focused app asks the compositor to create a token linked to the focused window (Wayland surface) and the most recent user interaction (an input event serial associated with a seat).
This token is then used by the app that should receive focus when it requests to be activated. In GNOME Shell, activation means that the the window receives focus and is placed on top of other windows. An activation token may still be rejected, for example if the window linked to the token doesn’t have focus or when the linked user interaction isn’t recent enough.
In addition to handling focus, GNOME Shell also tracks app launching. Until the new app window is actually shown, GNOME Shell uses a “loading spinner” mouse cursor to indicate to the user that the app is loading. If the app doesn’t implement the XDG Activation protocol, the loading indicator only disappears after a timeout because GNOME Shell doesn’t know that the application finished loading and has presented the target window.
The protocol doesn’t define how tokens are given to the target app. One reason for this is because it depends on how the app is started. The main options are:
Setting the XDG_ACTIVATION_TOKEN environment variable
D-Bus Activation using the platform-data field, which contains the activation token
XDG portals that will launch an app (e.g. the OpenURI or OpenFile portals)
The target app then needs to collect the token and use it to have its window activated to receive focus and to signal to the compositor that it started successfully.
Not smart enough
When I started looking into how our focus prevention mechanism works to investigate the issues mentioned above, I was initially pretty confused. There were a lot of cases where the focus window switch worked fine, but other times it wouldn’t. I realized quickly that with existing windows, the “<App> is ready” notification is shown, but new window would get focus immediately.
This struck me as odd: Why are new windows allowed to do whatever, but existing windows are restricted in the way they can take over focus?
I first thought this was some sort of bug, but then I discovered that the behavior was by design: Mutter has a gsettings property called focus-new-windows that controls the focus stealing prevention mechanism. This property can be strict or smart (the latter being the default).
smart means that in most cases new windows get focus (even without asking for it) and are raised to the top of the window stack
strict means they get focus (are “activated”, in technical terms) only when they are actually supposed to
The smart mode exists in part because there are some cases where our current focus prevention system does not work well. These issues include:
Launching apps via terminal (vte issue #2788). The main issue is that the terminal executing a command does not know whether that process will present a window or not. For example, if you launch vim there’s no new window, but if you launch firefox there is.
Launching apps via Run a Command in GNOME Shell (gnome-shell issue #7704) shares similar issues as running apps from the terminal
Apps launched via custom keyboard shortcut (e.g. set up in Settings > Keyboard > Keyboard Shortcuts)
The lack of implementation of the appropriate protocols in apps or toolkits
Because the cases where a new window is opened are a significant percentage of the overall cases where focus prevention is triggered, this smart mode is making it appear as though apps actually implement the XDG Activation protocol, even if they don’t. While it does somewhat reduce annoyance for users, it gives developers the false impression that they don’t have to do anything.
It also makes it harder to debug issues where something doesn’t work as expected or is missing the correct implementation. For example, even in GTK4 the focus transferring is broken in some cases and took a long time to be discovered (gtk issue #6711).
Security implications
Unfortunately the current situation with smart as the default means that we’re not getting most of the benefits of focus stealing prevention. Apps are able to spawn a new window over your current one and grab keyboard focus, because the smart mode just gives the new window focus, circumventing the safety measures. This is trivial to exploit by malicious apps: All they need to do is open a new window, and focus stealing prevention doesn’t apply.
Next steps
While some people have asked for focus stealing prevention to be disabled completely until it’s implemented by most apps and toolkits, I’m not sure this is the best way forward. If we did that, nobody would notice which apps don’t implement it, so there’d be no reason for toolkits to do so.
On the other hand, there are some remaining issues around terminal applications and similar use cases that we don’t have a plan for yet, so just switching to strict to flush out app bugs isn’t ideal either at the moment.
There is currently no consensus in the team as to how to proceed. The two main directions we could take are:
Switch to strict mode by default (mutter issue #3486) once a few remaining issues are resolved, perhaps with a “flag day” deadline so apps have time to implement it.
Slowly make the smart mode stricter over time.
Either way we need to raise more awareness of the issue to get app and toolkit developers interested in improving things in this area, which this blogpost is a part of
It’d also be helpful if more people (especially developers) turn on strict mode on their system, so we get more testing for which apps work and which don’t. This is the relevant gsetting:
gsettings set org.gnome.desktop.wm.preferences focus-new-windows 'strict'
Thanks
Thanks to the Sovereign Tech Fund for allowing me to take the time to properly work through this as part of my broader effort around improving notifications. Thanks also to Sonny Piers and Tobias Bernard for organizing the STF project, Florian Müllner, Sebastian Wick, Carlos Garnacho, and the rest of the GNOME Shell team for reviewing my MRs, and Jonas Dreßler and Jonas Ådahl for reviewing the blogpost.
The problem without plymouth and AMD GPUs is that the amdgpu driver is a really really big driver, which easily takes up to 10 seconds to load on older PCs. The delay caused by this may cause plymouth to timeout while waiting for the GPU to be initialized, causing it to fallback to the 3 dot text-mode boot splash.
There are 2 workaround for this depending on the PCs configuration:
1. With older AMD GPUs the radeon driver is actually used to drive the GPU but even though it is unused the amdgpu driver still loads slowing things down.
To check if this is the case for your PC start a terminal in a graphical login session and run: "lsmod | grep -E '^radeon|^amdgpu'" this will output something like this:
amdgpu 17829888 0 radeon 2371584 37
The second number after each is the usage count. As you can see in this example the amdgpu driver is not used. In this case you can disable the loading of the amdgpu driver by adding "modprobe.blacklist=amdgpu" to your kernel commandline:
2. If the amdgpu driver is actually used on your PC then plymouth not showing can be worked around by telling plymouth to use the simpledrm drm/kms device created from the EFI framebuffer early on boot, rather then waiting for the real GPU driver to load. Note this depends on your PC booting in EFI mode. To do this run:
This is a regular meson package and can be installed the usual way.
# Configure project in _build directory
meson setup --wipe --prefix=~/.local _build .
# Build and install in ~/.local
ninja -C _build install
How to use it
To add a Wayland compositor to your application all you have to do is create a CasildaCompositor widget. You can specify which UNIX socket the compositor will listen for clients connections or let it will choose one automatically.
compositor = casilda_compositor_new ("/tmp/casilda-example.sock");
gtk_window_set_child (GTK_WINDOW (window), GTK_WIDGET (compositor));
Once the compositor is running you can connect to it by specifying the socket in WAYLAND_DISPLAY environment variable.
Last cycle wasn’t particularly exciting, only featuring the new dialogs and a few smaller changes, but this one should be more interesting. So let’s look at what’s new.
Bottom sheet
Last cycle libadwaita got new dialogs, which can be presented as bottom sheets on mobile, and I mentioned that they will also be available as a standalone widget in future – so AdwBottomSheet exists and is public now.
As a standalone widget, bottom sheets work a bit differently from dialogs – they are persistent instead of being destroyed upon closing, more like the sidebar of AdwOverlaySplitView.
They also have a few new features, such as a drag handle, or a bottom bar presentation. This is useful for apps like music players.
AdwHeaderBar also integrates with bottom sheets – it hides the title when used in a bottom sheet with a drag handle.
Spinner
Libadwaita also has a new spinner widget – AdwSpinner. It both refreshes visuals and addresses various problems with GtkSpinner.
GtkSpinner is a really simple widget. Both the spinner itself and the animation are set in CSS. The spinner is just a symbolic icon, and the animation is a CSS animation. This approach has a few problems, however.
First, the old spinner has a gradient. Symbolic icons don’t actually support gradients, so it has to resort to dithering, as Jakub Steinerexplained in his blog a few years ago. This works well if the spinner is small enough (16×16 – 32×32), but becomes very noticeable at larger sizes. This means that the spinner didn’t work well for loading screens, or status pages.
Meanwhile, CSS animations are entirely disabled when system animations are off. Usually that makes sense, except here it means the spinner freezes, defeating the entire point of having it (indicating that the app isn’t frozen during long operations).
And, while CSS animations are pretty sophisticated, you can only do so much with a single element – so it’s literally a spinning icon. elementary OS does a more interesting thing – it spins it in steps, while the icon consists of 12 dashes, so it looks like they change color instead. Even then, more complex animations are impossible.
AdwSpinner avoids all of these issues. Since it’s in libadwaita and not in GTK, it can be more opinionated with regard to styling, so instead of using an icon and CSS, it’s just custom drawing. And since it’s not using CSS animations, it can keep spinning with animations off, and can animate in a more involved way than a simple spinning icon.
It still has a size limit – 64×64 pixels. While it can scale further, we don’t really need larger sizes and capping the size makes it easier to use – to make a loading screen using GtkSpinner, you have to set the :halign and :valign properties to CENTER, as well as :width-request and :height-request properties to 32. If you fail to do these steps, the spinner will either be too large, or too small respectively:
Meanwhile if you just put an AdwSpinner into a large bin, it will look right by default.
Oh, and GtkSpinner is invisible by default and you have to set the :spinning property to true as well. This made sense back in the age of foot and dinosaur spinners, where the spinner would stay in place when not animating, but that’s not really a thing anymore.
(though Nautilus wasn’t actually using GtkSpinner)
It also didn’t help that until this cycle, GtkSpinner would continue to consume CPU cycles even when not visible if the :spinning property is left enabled, so you had to start the spinner in the ::map signal and stop it in ::unmap. That is fixed now, but it was a major source of lag in, say, Epiphany in the past (which had a spinner in every tab, another spinner in every mobile tab switcher row and another one in the floating bar that shows URLs on hover, copied from Nautilus).
Spinner paintable
In addition to AdwSpinner, there’s also AdwSpinnerPaintable. It can be used with GtkImage, any other place that accepts paintables (such as status pages) or just manually drawn. It is a bit more awkward to use than the widget, as it needs to reference another widget in order to animate (since paintables cannot access the frame clock on their own), but it allows to use spinners in contexts that wouldn’t be possible otherwise.
AdwStatusPage even has a special style for spinner paintable – similar to the .compact style, but applied automatically.
Button row
Another widget we have now is AdwButtonRow – a list row that looks more or less like a button. It has a label, optionally icons on either side, and can use destructive and suggested style classes.
This pattern isn’t new – it has been used in mockups for a while (at least as early as 2021) – but it varied quite a bit between different mockups and implementations and so having a standard widget for it wasn’t viable. This cycle Jamie Gravendeel and kramo took time to standardize the existing designs into a tangible proposal – so it exists as a standard widget now.
Most of the time these rows aren’t meant to be linked together, so AdwPreferencesGroup has a new property :separate-rows. When enabled, the rows within will appear separately. This is mostly useful for button rows, but also e.g. entry rows. When not using AdwPreferencesGroup, the same effect can be achieved by using the .boxed-list-separate style class instead of .boxed-list.
Multi-layout view
Libadwaita 1.4 introduced AdwBreakpoint, which allowed to easily set properties on window size changes. However, a lot of apps need layout changes that can’t be expressed via simple properties – say, switching between a sidebar and a bottom sheet. While it is possible to do it programmatically anyway, it’s fairly involved and not a lot of apps went to those lengths.
Back then I also prototyped a widget for automatically reparenting children between different layouts via using a property mentioned a future widget for automatically reparenting children between different layouts, and now it’s finished and available for use as AdwMultiLayoutView.
It has changed somewhat since the prototype, e.g. it doesn’t dynamically create or destroy layouts anymore, just parents/unparents them, but the gist is still the same:
Put one or more AdwLayoutSlot into each layout, give them IDs
Define children matching those IDs
Then those children will be placed into the slots for the current layout. When you switch the layout, they will be reparented into slots from that layout instead.
So now it’s possible to define completely different layouts for desktop and mobile entirely via UI files.
CSS variables and colors
I’ve already talked about this in a lot of detail in my last blog post, but GTK has a lot of new CSS goodies, and libadwaita 1.6 makes full use of them.
Libadwaita now provides CSS variables for all of its old named colors, with a docs page to go with it, as well as new variables: --dim-opacity, --disabled-opacity, --border-opacity and --window-radius.
This also allowed to have matching focus ring color on .destructive-action buttons, as well as matching accent color for the .error, .warning and .success style classes. And because overriding accent color for a specific widget is now possible, .opaque button style class has been deprecated in favor of overriding accent colors on .suggested-action. Meanwhile, the white accent color of .osd is now more reliable and automatically works for custom widgets, instead of trying (and often failing) to manually override it for every standard widget.
I mentioned that it might be possible to generate standalone accent/error/etc colors from their respective background colors. However, the question was how to make that automatic, so at the time we didn’t actually integrate that. Now it is integrated, though it’s not completely automatic – only for :root.
Specifically, there’s a new variable: --standalone-color-oklab, corresponding to the correct color transformation for the current style.
So, when overriding accent color for a specific widget, there is a bit of boilerplate to copy:
It’s still an improvement over calculating the color manually, both for light and dark styles (which a lot of apps didn’t do at all, resulting in poor contrast), so still worth it. Maybe one day we’ll be able to make it completely automatic – e.g. by ensuring that using variables with wildcards doesn’t regress performance.
Another big feature is system accent color support. While it’s not a strictly libadwaita change, this is the developer-facing part, so it makes sense to talk about it here.
Behind the scenes it’s using the settings portal which provides a standardized key for the system accent color. Many other environments support it as well, so libadwaita apps will follow their accent color preferences too, while non-GNOME apps that follow the preference will follow it on GNOME too. Note that while the portal exposes arbitrary sRGB colors, libadwaita will pick the closest color from a list of nine colors, as visible on the screenshot above. This is done in the Oklch color space, mostly based on hue, so should work even for really dull colors.
Accent colors are also supported when running on Windows and macOS, and like with the color scheme and high contrast, the libadwaita page in GTK inspector allows to toggle the system accent color now.
Apps are still free to set their own accent color. CSS always takes priority over the system accent.
A lot of people helped push this over the finish line, with particular thanks to Jamie Murphy, kramo and Jamie Gravendeel.
API
AdwStyleManager provides new properties for fetching the system color – :accent-color and :accent-color-rgb, as well as :system-supports-accent-colors for querying whether the system has accent color preferences – same as for color scheme.
The :accent-color property returns a color from the AdwAccentColor enum, so that individual colors can be special cased (say, when using bitmap assets). This color can be converted both to background color RGBA (using adw_accent_color_to_rgba()) and to standalone color (adw_accent_color_to_standalone_rgba()).
All of these colors use white foreground color, so there’s no API for fetching it, at least for now.
Note that :accent-color-rgba will still return the system color even if the app overrides its accent color using CSS. It only exists for convenience and is equivalent to calling adw_accent_color_to_rgba() on the :accent-color value.
While we still don’t have a general replacement for deprecated gtk_style_context_lookup_color(), the new accent color API can replace at least some of its uses.
On CSS side, there are new variables corresponding to each accent color: --accent-blue for blue and so on. Additionally, every system color, along with their standalone colors for both light and dark, is documented and can be used as a reference.
Destructive buttons
Having accent color that’s not always blue means having to rethink other style choices. In particular, .destructive-action buttons were just a red version of .suggested-action, same as in GTK3. This was already questionable from accessibility perspective, but breaks entirely with accent colors, since suggested buttons would look exactly same as a destructive ones with red accent. And so .destructive-action has a distinct style now, less prominent than suggested.
Alert dialogs
Old and new alert dialogs side by side
Another area that needed updates was AdwAlertDialog – it was also using color for differentiating suggested and destructive buttons.
Coincidentally, the alert dialog style went almost unchanged from GTK3 days, and looked rather out of place with the rest of the platform. So kramo came up with an updated design.
AdwMessageDialog and GtkAlertDialog received the same style, or at least an approximation – it’s not possible to replicate it entirely in GTK dialogs. Even though neither is recommended for use (when using libadwaita, anyway – nothing wrong with using GtkAlertDialog in plain GTK), regressing apps that aren’t fully up to date with the platform wouldn’t be very good.
Adapting apps
Accent colors are supported automatically, and in most cases apps don’t need any changes to make use of them. However, here’s a checklist to ensure it works well:
Make use of the accent color variables in custom CSS, like --accent-bg-color. Using the old named colors like @accent_bg_color works as well. Don’t assume accent color will be blue.
Conversely, don’t use accent color when you mean blue. We have variables like --blue-3 for that – or even --accent-blue.
When using accent color in custom drawing (say, drawing a graph), make sure to redraw it when AdwStyleManager:accent-color value changes – same as for color scheme and high contrast.
Deprecations
Last cycle we introduced new dialog widgets that are based on AdwDialog rather than GtkWindow. However, that happened right at the end of the cycle, without giving apps a lot of time to port their existing dialogs. Because of that, the old widgets (AdwMessageDialog, AdwPreferencesWindow, AdwAboutWindow) weren’t deprecated and I mentioned that they will be deprecated in future instead. So, they are now.
If you haven’t migrated to the new dialogs yet, see the migration guide for how to do so.
Other changes
As always, there are smaller changes that don’t warrant their own sections, so let’s look at those:
AdwWindow and AdwApplicationWindow now have a default minimum size (360×200 px), meaning you don’t have to set it manually to use breakpoints or dialogs anymore. Apps can still override it if they need a different size, but it works out of the box now.
GtkTextView now supports the .inline style class, removing its background and resetting its foreground color. This allows to use it in contexts like cards.
Future
As usual, there are changes that didn’t make it this cycle and will land the next cycle instead. Most notably, the old toggle groups branch by Maximiliano is finally finished and will land early next cycle.
Big thanks to STF for funding a lot of this work (GTK CSS improvements, bottom sheets, finishing multi-layout view and toggle groups, general maintenance), as well as people organizing the initiative and all contributors who made this release happen.
A couple of weeks ago, I went to see Alien: Romulus. While many of my friends were disappointed, I actually enjoyed it. In fact, it exceeded my expectations — mainly because I didn’t expect much! :)
Fede Alvarez delivered exactly what producer Ridley Scott asked of him, leaning heavily on the nostalgia of the original masterpiece while skirting the edge of a reboot. The world of Prometheus wasn’t ignored, but purposedly avoided referencing too deeply.
The dystopian world of corporate feudalism set a tone even darker than the original, to the point where the xenomorph didn’t seem like the worst thing that could happen. I’m still holding out hope for 90-minute movies as the gold standard, but the two-hour runtime was manageable—though my aging buttocks may disagree. The slow-burn first act was actually the most enjoyable part, as that’s where the fresh world-building took center stage. Even as the familiar plot unfolded, Alvarez delivered memorable suspense and action scenes.
Of course, it’s never going to feel the same as seeing Alien or Aliens as a teenager. I can’t fully dive into my minor criticisms without spoilers, but let’s just say the movie understood that “less is more” — except in one area. Other than that, Alien: Romulus proved that going to the movies can still be a pretty great experience.
I wrote a post about so called "M type" and "S type" processes in software development. Unfortunately it discusses the concept of human sexuality. Now, just to be sure, it does not have any of the "good stuff" as the kids might say. Nonetheless this blog is syndicated in places where such topics might be considered controversial or even unacceptable.
Thus I can't really post the text here. Those of you who are of legal age (whatever that means in your jurisdiction) and out of their own free will want to read such material, can access the PDF version of the article via this link.
As it's now aproaching mid-September and the autumn release of GNOME, so is there also a new release of Maps approaching for GNOME 47.
Switch to Vector-based Map
The biggest change since last release is that we now use the vector-based map by default and the old raster map has also been retired since we wanted to move forward with things like enabling, and relying on clickable POIs directly in the map view so we could the remove the old tedious “What's here?” context menu doing a reverse geocoding to get details about a place (which is also a bit hit-and-miss with regards to how close to where you point the actual result is).
Apart from this other benefits we get (and this has already been mentioned in earlier posts) localized names (when tagged in OpenStreetMap) and finally a proper dark mode with our new GNOME map style.
Light (default) theme variant of the map in 47
Dark theme variant of the map in 47
Redesigned Search Bar
The “explore” button to open the search for nearby POIs by category menu has now been integrated into the entry to avoid a theme issue when using the linked button style where the rounded corners disappears on the “other side” when the results popover is showing.
Search bar with explore button
This also looks a bit sleeker I think…
Improved Public Transit Routing
Public transit routing is now using the Transitous project (https://transitous.org) to provide transit routing for new regions. And as this is a crowd sourcing initiativ you can also help out by adding additional missing GTFS feeds to the project and it should automatically get supported by Maps.
For the time being, the regions we already supported by third-party APIs (such as Resrobot for Sweden, and OpenData.ch for Switzerland) will still use those providers to avoid possible regressions. It also gives us some leeway to improve MOTIS (the backend used by Transitous). The implementation in Maps also lacks e.g. support for specifying via locations.
Showing some travel itinerary options in Prague
Showing a sample of an itinerary from Lund, Sweden to Hamburg, Germany
Showing a sample of an itinerary in Denver, Colorado
Some changes had to be made to our internal shield rendering library (we couldn't use the OSM Americana implementation directly, as we had to implement ours using Cairo rendering and so on) to support the new convenience shortcut for a “pill” shield shape, and also being able to define hard-coded route references “ref” directly in the shield definition rather than getting from the tile data.
Rochester Inner Loop in Rochester, New York, using a fixed “LOOP” reference label
And on that note, a funny bug I discovered during some testing is that we currently always assume the “pointsUp” attribute in the shield definitions would always default to true when not explicitly set, while the OSM Americana code has different defaults depending on the following shape. So specifically for the “fishhead” shape it should actually be false when not set. I guess this is a bit odd to assume different values depending on following JSON elements, but…
Highway shields in New Zeeland
It was a bit funny I discovered this bug when “browsing around“ in New Zeeland considering Northern Hemisphere-centric jokes about people “Down-under” walking upside-down 😀. But this actually also affects some highways in the US…
I guess this should be fixed before the 47.0 release.
The WebRTC nerds among us will remember the first thing we learn about WebRTC, which is that it is a specification for peer-to-peer communication of media and data, but it does not specify how signalling is done.
Or put more simply, if you want call someone on the web, WebRTC tells you how you can transfer audio, video and data, but it leaves out the bit about how you make the call itself: how do you locate the person you’re calling, let them know you’d like to call them, and a few following steps before you can see and talk to each other.
WebRTC signalling
While this allows services to provide their own mechanisms to manage how WebRTC calls work, the lack of a standard mechanism means that general-purpose applications need to individually integrate each service that they want to support. For example, GStreamer’s webrtcsrc and webrtcsink elements support various signalling protocols, including Janus Video Rooms, LiveKit, and Amazon Kinesis Video Streams.
However, having a standard way for clients to do signalling would help developers focus on their application and worry less about interoperability with different services.
(author’s note: the puns really do write themselves :))
As the names suggest, the specifications provide a way to perform signalling using HTTP. WHIP gives us a way to send media to a server, to ingest into a WebRTC call or live stream, for example.
Conversely, WHEP gives us a way for a client to use HTTP signalling to consume a WebRTC stream – for example to create a simple web-based consumer of a WebRTC call, or tap into a live streaming pipeline.
WHIP and WHEP
With this view of the world, WHIP and WHEP can be used both for calling applications, but also as an alternative way to ingest or play back live streams, with lower latency and a near-ubiquitous real-time communication API.
We know GStreamer already provides developers two ways to work with WebRTC streams:
webrtcbin: provides a low-level API, akin to the PeerConnection API that browser-based users of WebRTC will be familiar with
webrtcsrc and webrtcsink: provide high-level elements that can respectively produce/consume media from/to a WebRTC endpoint
At Asymptotic, my colleagues Tarun and Sanchayan have been using these building blocks to implement GStreamer elements for both the WHIP and WHEP specifications. You can find these in the GStreamer Rust plugins repository.
Our initial implementations were based on webrtcbin, but have since been moved over to the higher-level APIs to reuse common functionality (such as automatic encoding/decoding and congestion control). Tarun covered our work in a talk at last year’s GStreamer Conference.
Today, we have 4 elements implementing WHIP and WHEP.
Clients
whipclientsink: This is a webrtcsink-based implementation of a WHIP client, using which you can send media to a WHIP server. For example, streaming your camera to a WHIP server is as simple as:
whepclientsrc: This is work in progress and allows us to build player applications to connect to a WHEP server and consume media from it. The goal is to make playing a WHEP stream as simple as:
The client elements fit quite neatly into how we might imagine GStreamer-based clients could work. You could stream arbitrary stored or live media to a WHIP server, and play back any media a WHEP server provides. Both pipelines implicitly benefit from GStreamer’s ability to use hardware-acceleration capabilities of the platform they are running on.
GStreamer WHIP/WHEP clients
Servers
whipserversrc: Allows us to create a WHIP server to which clients can connect and provide media, each of which will be exposed as GStreamer pads that can be arbitrarily routed and combined as required. We have an example server that can
play all the streams being sent to it.
whepserversink: Finally we have ongoing work to publish arbitrary streams over WHEP for web-based clients to consume this media.
The two server elements open up a number of interesting possibilities. We can ingest arbitrary media with WHIP, and then decode and process, or forward it, depending on what the application requires. We expect that the server API will grow over time, based on the different kinds of use-cases we wish to support.
GStreamer WHIP/WHEP server
This is all pretty exciting, as we have all the pieces to create flexible pipelines for routing media between WebRTC-based endpoints without having to worry about service-specific signalling.
If you’re looking for help realising WHIP/WHEP based endpoints, or other media streaming pipelines, don’t hesitate to reach out to us!
GNOME is interested in participating in the Outreachy December-March cohort, and while we already have a few great projects, we are looking for experienced mentors with a couple more project ideas. Hurry up, we have until September 11 to conclude our list of ideas.
As of today, Mutter will style legacy titlebars (i.e. of X11 / Xwayland apps that don’t use client-side decorations) using Adwaita on GNOME.
Shadows match the Adwaita style as well, including shadows of unfocused windows. These titlebars continue to follow the system dark and light mode, even when apps don’t.
Should make using legacy apps a little less unpleasant
We have finally reached the final week of GSoC. It has been an amazing journey! Let’s summarize what was done, the current state of the project and what’s next.
Introduction
This summer, I had the opportunity to work as a student developer under the Google Summer of Code 2024 program with the GNOME Community. I focused on creating a web-based Integrated Development Environment (IDE) specifically designed for writing and executing SPARQL queries within TinySPARQL (formerly Tracker).
This user-friendly interface empowers developers by allowing them to compose and edit multiline SPARQL queries directly in a code editor, eliminating the need for the traditional terminal approach. Once a query is written, it can be easily executed via the HTTP SPARQL endpoint, and the results will be displayed in a visually appealing format, enhancing readability and user experience.
By lowering the barrier to entry for newcomers, boosting developer productivity with visual editing, and fostering collaboration through easier query sharing, this web IDE aims to significantly improve the experience for those using libtracker-sparql to interact with RDF databases.
I would like to express my sincere gratitude to my mentors, Carlos and Sam, for their guidance and support throughout the internship period. Their expertise was invaluable in helping me navigate the project and gain a deeper understanding of the subject matter. I would also like to thank my co-mentee Rachel, for her excellent collaboration and contributions to making this project a reality and fostering a fast-paced development environment.
I’m excited to announce that as the internship concludes, we have a functional web IDE that enables users to run SPARQL queries and view the results directly in their web browser. Here is the working demo of the web IDE that was developed from scratch in this GSoC Project.
Working of TinySPARQL Web IDE
What was done
This project was divided into two primary components: the backend C code, which enabled the web IDE to be served and run from the command line, and the frontend JavaScript code, which enhanced the web IDE’s visual appeal and added all user-facing functionalities. I primarily focused on the backend C side of the project, while Rachel worked on the frontend. Therefore, this blog post will delve into the backend aspects of the project. To learn more about the frontend development, please check out Rachel’s blog.
The work done by me, could be divided into three major phases:
Pre-Development Phase
During the pre-development phase, I focused on familiarizing myself with the existing codebase and preparing it for easier development. This involved removing support for older versions of libraries, such as Libsoup.
TinySPARQL previously supported both Libsoup 2 and Libsoup 3 libraries, but these versions had different function names and macros.
This compatibility requirement could significantly impact development time. To streamline the process, we decided to drop support for Libsoup 2.
The following merge requests document the work done in this phase:
In this phase, I extended the HTTP endpoint exposed by the tinysparql endpoint command to also serve the web IDE. The goal was to enable the endpoint to serve HTML, CSS, and JavaScript files, in addition to RDF data. This was a crucial step, as frontend development could only begin once the basic web IDE was ready.
During this phase, the HTTP module became more complex. To aid in debugging and diagnosing errors, we added a debugging functionality. By running TRACKER_DEBUG=http, one can now view logs of all GET and POST methods, providing valuable insights into the HTTP module’s behavior.
The following merge requests document the work done in this phase:
The web IDE added significant size (around 800KB-1MB) to the libtracker-sparql library. Since not all users might need the web IDE functionality, we decided to separate it from libtracker-sparql. This separation improves efficiency for users who won’t be using the web IDE.
To achieve this isolation, we implemented a dedicated subcommand tinysparql webide for the web IDE, allowing it to run independently from the SPARQL endpoint.
Here’s a breakdown of the process:
Isolating HTTP Code: I started by extracting the HTTP code from libtracker-sparql into a new static library named libtracker-http. This library contains the abstraction TrackerHttpServer over the Libsoup server, which can be reused in the tinysparql webide subcommand.
Creating a Subcommand: Following the isolation of the web IDE into its own library and the removal of relevant gresources from libtracker-sparql, we were finally able to create a dedicated subcommand for the web IDE. As a result, the size of libtinysparql.so.0.800.0 has been reduced by approximately 700KB.”
The following merge requests document the work done in this phase:
This is the web IDE we developed during the internship. Check out this demo video to see some of the latest changes in action.
TinySPARQL Web IDESPARQL Query successfully executedError handling
Future Work
Despite having a functional web IDE and completing many of the tasks outlined in the proposal (even exceeding the original scope due to the collaborative efforts of two developers), there are still areas for improvement.
I plan to continue working on the web IDE in the future, focusing on the following enhancements:
Multi-Endpoint Support: Implement a mechanism for querying different SPARQL endpoints. This could involve adding a text box input to the frontend for dynamically entering endpoint URLs or providing a connection string option when creating the web IDE instance from the command line.
Unified HTTP Handling: Implement a consistent HTTP handler for all cases, allowing TrackerEndpointHttp to handle requests both inside and outside the /sparql path.
SPARQL Extraction: Extract the SPARQL query from POST requests in TrackerEndpointHttp or pass the raw POST data in the ::request signal, enabling TrackerEndpointHttp to determine if it contains a SPARQL query.
Avahi Configuration: Move the Avahi code for announcing server availability or assign TrackerEndpointHttp responsibility for managing the content and type of broadcasted data.
CORS Configuration: Make CORS settings configurable at the API level, allowing for more granular control and avoiding the default enforcement of the * wildcard.
GUADEC Experience
One of the highlights of my GSoC journey was the opportunity to present my project at GUADEC, the annual GNOME conference. It was an incredible experience to share my work with a diverse audience of developers and enthusiasts. Be sure to check out our presentation on the TinySPARQL Web IDE, delivered by Rachel and me at GUADEC.
Final Remarks
Thank you for taking the time to read this. Your support means a great deal to me. This internship was a valuable learning experience, as it was my first exposure to professional-level C code and working with numerous libraries solely based on official documentation. I am now more confident in my skills than ever. I gained a deeper understanding of the benefits of collaboration and how it can significantly accelerate development while maintaining high code quality.
got in the way and it took a while to finish this post. I finally found a little spare time to collect my thoughts and finish writing this.
