December 22, 2009

Lernid Gets Presentation Support

I just wanted to share a cool new feature I just hacked into Lernid today. The current cut of the feature is still very new, but it works great. I have added slide support. :-)

This is how it works: imagine you are giving a session at the upcoming UBuntu Developer Week or Ubuntu Open Week. The sessions are scheduled in the iCal feed, and in the Description you can add the following to your session:

[SLIDES: http://www.myaddress.com/slides.pdf]

This PDF file contains slides for your presentation.

When it comes time to run your session, you type in [SESSIONSTART] in the IRC channel and all Lernid clients will go and download the PDF file and display the first slide in a small and convenient, and resizable pane within the interface.

The session leader can then change to the next slide by typing the following in the session:

[SLIDE 2]

This will then change the slide and display it for all Lernid clients. This now means that in tutorial sessions we can display presentations to viewers while the session is going on. This is a huge feature for our learning weeks. Also, don’t forget that we also have functionality built into Lernid to display URLs shown in the channel too in the built-in web browser. This is all helping Lernid to become a solid platform for online training.

This is how it looks:

The feature landed in bazaar today, but it will take a little while for me to eek out some of the bugs. As such, don’t expect it to work flawlessly in the daily builds of Lernid. My aim is to have these bugs squashed by the new year. :-)

Rock and roll!

New Moonlight Covenant has been posted

As I mentioned a few days ago, there is a new covenant form Microsoft for Moonlight, it has been posted.

Cena Linuxera en el DF, hoy

Cena Linuxera/Monera hoy (Diciembre 22) en el bar/restaurante del Covadonga a las 7pm. Para todo p�blico (incluso talibanes).

Direcci�n: Puebla 121 cerca de el Metro Insurgentes.

Tue 2009/Dec/22

  • If you've had the chance to play already with a N900, you've probably noticed that you can typeahead your contacts' names in the Phone application, in order to filter them and find the person you're looking for quickly. Similarly, the media player will allow you to filter the list of artists/albums/songs/etc. by simply typing the first letters of the media you're looking for. This is probably one of the most praised features of these two applications, specially in times when contact lists grow up to several hundreds of people and dumping your media collection inside the phone takes no effort.

    This feature, known as live search, used to live inside these two applications, each with a different implementation. Third party maemo developers wanting to have a similar feature would have needed to cook it themselves inside their applications. Clearly, a waste of effort.

    But luckily, that's not the case anymore. I spent a big part of the last two months (when not in holidays nor flying from and to A Coruña) working in a consolidation of the live search widget originally written by the good old Jörgen and later rewritten by Xavier, part of the osso-abook library. With the blessing of Nokia, I took it and removed all the osso-abook dependant bits and made it as generic as possible, in order to allow its usage in any application using a GtkTreeView and a GtkTreeModelFilter. This widget, now LGPL and named HildonLiveSearch, landed in hildon's master around a month ago. Better yet, I also took HildonLiveSearch and made it a feature of HildonTouchSelector, the hildon widget for pannable lists that you can see in most of the N900 applications. This already landed in the stable branch of hildon and I'm confident that Nokia will release it at some point with an update of the phone's software (don't ask me when, though). Rumor has it, this is already a feature in one of the latest releases of Hildon Application Manager.

    HildonLiveSearch in a HildonTouchSelector

    For developers interested: if you use a HildonTouchSelector, don't worry, you'll have live search for free in your application. If you just use GtkTreeView, have a look at the HildonLiveSearch widget implementation and its simple example, as well as the HildonTouchSelector implementation for a more complete example. Of course, you can imagine that the changes to make it work inside HildonTouchSelector are quite invasive, so I'd please ask you to grab hildon's code from the repository, build packages, smoke test it in your application, and file bugs if you find any. This way, we minimize the possibility for regressions by the time this hits the public. Shall you have any concern or question, don't hesitate to contact us in the hildon-devel mailing list.

    Let this be my christmas present to the world for this year. Special thanks to Xavier, who was of great help during the push of the live search into hildon.

WebKitGTK+ HackFest!

The WebKit hackfest is now over, and I think it was a very productive week. Thank you very much to all who attended, to Igalia for organizing the hackfest, and hosting us so well, to Collabora for having sponsored the event, and allowed me to spend the week working on it, and to the GNOME foundation for having payed all of my costs!


Xan blogged about day 0, and also a summary of all that was done, so I’ll focus on the stuff he forgot to mention ;D. The hackfest, for me, started on day -1 with me not allowing Xan to go sleep before he had reviewed a couple patches of mine to fix DOM context menu handling. It always bothered me that Epiphany failed to open right-click menus in some pages, or let pages handle the right click. Well, this is fixed now, and Zimbra users can now have their right click menus, and WoW players can remove talent points from their calculators =P.

It turns out that many of the attendees don’t like pages messing with their context menus, though, and they had some good points to back up their positions (like pages making it hard to save images, for instance), so I implemented a way to force openning the custom menu: Ctrl-rightclick.

We wanted to use a GtkInfoBar to display questions regarding the form saving - our initial implementation always saved all credentials, but that didn’t sound good enough. Xan and I thought it would be very complicated to make this work, because there were assumptions in the code regarding which widget contains which, but it turned out to be quite trivial - making EphyEmbed a descendant of GtkVBox instead of GtkScrolledWindow, fixing a small number of assumptions, and that was it.

The passwords are saved in the GNOME Keyring. It’s interesting to point out that GNOME Keyring seems to be unhappy with the number of passwords a browser stores - Xan’s daemon was hanging, crashing, and spawning a large number of threads. My daemon decided to take up some 300MB of RAM at one point. It’s somewhat funny to see how much a browser pushes the limits of our platform. We are hoping this will improve with the new keyring APIs, and the rewrite that is ongoing. It’s nice to see my browser form passwords in seahorse, though, and be able to manage them like any other.

One more thing worth of notice, although this post is already a bit too big: one of the main concerns people had during the Hackfest was on making build time smaller. Touching a single file in WebCore causes a debug build of 10 minutes on my laptop. Evan Martin and Benjamin Otte made a push at removing unnecessary includes from WebCore, and WebKitGTK+ files, which brough the build time down a bit. They end up inspiring Aroben, from Apple, to go even further into this, and remove many includes from files all over WebKit.

Evan was also able to bring linking time down by making it possible to link libwebkit without having to build all the intermediate libraries, which brought build time down to 1 minute, when touching a single file in WebCore. Behdad and I also started looking into breaking WebCore up into lots of shared libraries for Debug builds, since we don’t care too much about speed penalties in those. None of these experiments got committed yet, but I am hoping we will be having a better time hacking on WebKitGTK+ in the near future.

It was awesome meeting everyone, by the way! See you around =).

Solstice

<quote>

Hello Astronomers

it is Solstice.

(this from Wiki)

Of the many ways in which solstice can be defined, one of the most common (and perhaps most easily understood) is by the astronomical phenomenon for which it is named, which is readily observable by anyone on Earth: a “sun-standing.” This modern scientific word descends from a Latin scientific word in use in the late Roman republic of the 1st century BC: solstitium. Pliny uses it a number of times in his Natural History with the same meaning that it has today. It contains two Latin-language segments, sol, “sun”, and -stitium, “stoppage.”[2] The Romans used “standing” to refer to a component of the relative velocity of the Sun as it is observed in the sky. Relative velocity is the motion of an object from the point of view of an observer in a frame of reference. From a fixed position on the ground, the sun appears to orbit around the Earth.[3]

To an observer in inertial space, the Earth is seen to rotate about an axis and revolve around the Sun in an elliptical path with the Sun at one focus. The Earth’s axis is tilted with respect to the plane of the Earth’s orbit and this axis maintains a position that changes little with respect to the background of stars. An observer on Earth therefore sees a solar path that is the result of both rotation and revolution.

The component of the Sun’s motion seen by an earthbound observer caused by the revolution of the tilted axis, which, keeping the same angle in space, is oriented toward or away from the Sun, is an observed diurnal increment (and lateral offset) of the elevation of the Sun at noon for approximately six months and observed daily decrement for the remaining six months. At maximum or minimum elevation the relative motion at 90° to the horizon stops and changes direction by 180°. The maximum is the summer solstice and the minimum is the winter solstice. The path of the Sun, or ecliptic, sweeps north and south between the northern and southern hemispheres. The days are longer around the summer solstice and shorter around the winter solstice. When the Sun’s path crosses the equator, the days and nights are of equal length; this is known as an equinox. There are two solstices and two equinoxes.[4]

We used to observe and celebrate this event widely, the Romans moved the birth of Christ to Solstice to usurp it .

Now people celebrate fiction instead of fact ..unless your an Astronomer that is.

Praise the Laws and Happy Solstice.

Ordono Mundi

</quote>

ps. I think Solstice was yesterday, but oh well..

Rygel 0.4.8 (Till the Blood Runs Clear) is out!

Here goes the release log:

A bug-fix release in stable 0.4.x series to fix seeking in general and playback
for clients that always seek (Sony PS3).

Dependency-related changes:

- Require valac >= 0.7.10.

All contributors to this release:

Zeeshan Ali (Khattak) <zeeshanak>

Download source tarball from here.

gtkmm and Maemo 5

There’s a decent number of examples of Maemo 5 applications written in C or even python, but there’s not too many real-world applications in (non-Qt) C++. So I’ve written up a quick, cliché little demo app in gtkmm/hildonmm. It probably isn’t useful for much other than perhaps as an example (who really needs dedicated application for splitting a bill anyway?). But I thought I’d put a git repository online in case it’s useful for somebody.

screenshot01.png

Presenting at CSUN

So my talk How to Make Friends and Remove Access Barriers In Open Source Software
has been accepted for CSUN 2010. The presentation is scheduled for 8 AM, I expect high attendance of dairy farmers, since they will probably be taking their lunch break around that time.

Here is a short abstract:

This presentation will consist of two parts. In the first part we will demonstrate the contrasts that exist when developing accessible applications between proprietary platforms and free platforms. In the second part we will become familiar, through real-life examples, with the culture in GNOME, a Free and Open Source desktop environment, and how it provides a conducive atmosphere for accessibility innovation and contribution, by developers, writers, educators, and users.

I am going to have a lot of fun putting together slides and media for this.

Still need to do some last-minute Christmas Shopping?

<script type="text/javascript">//

gnome-amazon

In case you still need to buy some gifts for Christmas you can help GNOME by buying your gifts at Amazon via the GNOME Amazon Store or via one of the searchplugins listed below. The GNOME foundation will receive between 4-6% of whatever you buy as an affiliate fee.

Below and on http://www.gnome.org/friends/amazon you find the links to install the GNOME Amazon Search Plugin of your favorite Amazon store. NOTE the links below might not work if you read this post in an RSS reader, because it needs a javascript command to install the search plugin. Just install the search plugin by going to this post directly.

SearchPlugin

Art Of Community Rated #2 Must Have Social Media Book By Mashable

Two lovely nuggets of Art Of Community information to share:

  • Firstly, The Art Of Community was rated #2 in Top 10 must-have books on social media by Mashable. Wow, I am tickled pink about this. Thanks, Mashable folks!
  • Secondly, I have have heard from a bunch of people that the legendary .net Magazine reviewed The Art Of Community saying it was well written, well structured and intriguing that O’Reilly support it as a free e-book too. Rock and roll. :-)

If you lovely people spot any more reviews, do let me know. :-)

December 21, 2009

Some thoughts on Copyright Assignment

Quick links

Overview & Rational

The collaborative model of Free Software development has taken the world by storm; increasingly corporations are contributing to the FLOSS world, and opening up large existing code-bases, while exploring many well-trodden ways of making a return on their investment. This is, of course, to be welcomed. Copyright assignment is one of the perennial tools chosen in the equation around monetization, control and risk.

This is particularly relevant today; as we are seeing something of a resurgence of interest - with several stake-holders developing a trend towards promoting copyright assignment - even where it was not required previously. I try to bring the perspective of a long-time, pragmatic Free software advocate to the topic, which (of course) intersects with both licensing and governance. This at least, is how I try to understand this area. Hopefully some little experience in the signing, and soliciting of both personal and corporate assignment of copyright to several parties helps. Also, having tried (and sometimes succeeded) to contribute code to the FSF's gcc, Sun's OpenOffice, xpdf by Derek Noonberg, Raph Levien's libart_lgpl, Novell's Evolution & Mono, Moblin, and a large number of eclectically owned projects - I hope I have a useful perspective.

This relevance is increased by the controversy around the acquisition by Oracle of MySQL: as an aside, I believe there are strong technical and business reasons why MySQL should present no obstacle to an Oracle acquisition of Sun. This post is (as always) not representative of Novell's position, but is my own. Great thinkers in our community have taken either side on MySQL: RMS, and Eben, for example; and there was recent controversy over Canonical's new copyright assignment policy, with an interesting LWN article on the topic.

In the past, I have been an enthusiastic advocate of the justice, and reasonableness of copyright assignment (in some cases). I have many friends, who have seen me trying to collect copyright assignment forms like autographs: sign the JCA today. Even in those days, I would meet developers I greatly respected who would politely refuse; indeed, I recall arguing vehemently with Bradley Kuhn that such things were risk free, and necessary. Thus I hope I have seen the argument from several sides.

An early example of copyright assignment problems

The issue of copyright assignment has been present from the early days of Free software. The FSF has, for a long time, required assignment, ostensibly to defend against license violations. Interestingly, one of the early 'religious' schisms in 1993 - XEmacs vs. Emacs was related to copyright assignment (though doubtless other things too: eg. should we require an X toolkit ?). A real problem here, making merging impossible was assignment:

Why are there two Emacsen?

Hysterical reasons. See the public flames if you must know. Currently the largest problems is that Sun owns the copyright for large part of the code. It is copyleft, but they refuse to sign over the changes to the FSF. Some people think it is good to have an Emacs variant where you can contribute code without signing any papers. Feel free to disagree.

One position we often hear these days - is that "copyright assignment does not deter contribution". At a minimum, XEmacs appears to demonstrate that for some companies, even those (supposedly) involved in open development from the beginning - copyright assignment creates a formidable barrier to entry. Furthermore - this is an undeniable reality from near the beginning of this practise. More can be read in Ben's opinion on the split. He also introduces a new (and I believe unrealistic, in the FSF's case) concern:

One of the dangers of assigning copyright to the FSF is that if the FSF happens to be taken over by some evil corporate identity or anyone with different ideas than RMS, they will own all copyright-assigned code, and can revoke the GPL and enforce any license they please. If the code has many different copyright holders, this is much less likely of a scenario.

While this problem appears to be precluded by the current FSF copyright assignment - it is in general, a problem worth considering in when assigning copyright to any entity.

Autobiographical, on-topic ramblings

Moving forward in time to 1995, shortly before I got involved with Linux, we saw a clutch of copyright assignment based businesses setup: Qt - May 1995, MySQL 1.0 - May 1995, xpdf - Dec 1995. Of these the first two are often held up to be resounding successes both exiting via acquisition: MySQL for a large multiple of it's true value to Sun; and Qt to Nokia. Today it is often claimed that we must not do, or advocate anything to harm such 'MySQL' business models.

Personally, my initial experience of copyright assignment was (I believe) with the FSF. In mid 1998 I wanted to get a patch into gcc; I wrote and posted the patch to RMS (who graciously replied), and put the assignment machinery into action; after some weeks, I got my company to sign it, posted it, and was done - it even seemed an exciting recognition of my (somewhat trivial) work. Unfortunately my patch lingered for another two years, un-reviewed before being committed. Sometimes I wonder if that sort of first-impression hiatus could be related to the latency of the assignment approval process. Needless to say, I moved onto other projects.

Then in 1996, the KDE project was founded, built on the (at the time) non-Free-Software Qt; and the year after - GNOME was started in response. Propelled by enthusiasm for the FSF's mission, and the fun of hacking with Miguel, I and many others joined the GNOME project. Strategically, this seemed a good move - and, technical debates aside, for years we dueled for different visions of the world: first Open Source, vs. Free Software, then - one where any company could write desktop software for Linux vs. one where all proprietary desktop software had to pay a single company (per developer). Even the FSF chose the LGPL for glibc; and so the arguments raged. Seemingly eventually the GNOME vision won (for now), and Qt moved - first to the GPL, and then the LGPLv2; well and good - except for the vast amount of acrimony, duplication and wasteage that continues to this day. Ultimately the choice of asymmetric license plus proprietary option for Qt, at the root of the schism, was enabled by their copyright-ownership, cemented by assignment for external contributions.

Later of course (in 1999) Ximian (ne. Helix Code), was founded, and from the beginning we insisted on copyright assignment for Evolution: we were building a platform to sell services - travel booking: hotels, cars, flights etc. integrated into your groupware suite (perhaps a vision in need of resurrection on the web today). Later on, those rights allowed us to switch tack to creating a proprietary Exchange connector. However - the GPL licensing of evolution had an untold impact on it's adoption; with several (known) others hindered in terms of their business model, depriving Evolution of contributors and keeping it substantially a single-company project. After Novell acquired Ximian, the exchange connector was made Free software, available to all - and this helped grow interest and contribution to the software. Recognizing these problems, as Evolution was re-licensed from GPLv2 to 'LGPLv2 or 3' the requirement for copyright assignment was dropped. The project now has an expanding contributor base from Novell, RedHat, Intel, Collabora, to the collection of great individuals working on it.

Another experience of copyright assignment was trying to contribute to xpdf whose regimen is GPL + assignment. I was re-writing the renderer to get high quality anti-aliased rendering using the (then) new & exciting Gnome Canvas infrastructure. My recollection was that, in stark contrast to the instant gratification of getting code included into GNOME, xpdf was acutely frustrating. It was really difficult, and that despite my willingness to sign the relevant paperwork, and my changes being (I hope) reasonably good. Similarly, it was not so easy to get changes into libart whose regimen was LGPL + assignment. Raph Levien (an hirsute man of immense talent & charisma) wanted to retain copyright to that too. Of course, eventually my changes worked, I had a beautiful, anti-aliased, PDF rendering Bonobo component - but, they were eventually mostly lost. As a footnote, it is interesting to see that in more recent times, braver souls have re-re-written the xpdf rendering to target a new and better rendering library (cairo). The result has been forked as poppler. As a freedesktop project, that is now used by both Evince and Okular. Cairo has long ago obsoleted Libart_lgpl which is being written out of the GNOME stack, along with the canvas that depended on it, and Gnome-Print that depended on the canvas is long dead. I suspect that no small factor in my experience, and the subsequent trajectory of these projects was copyright ownership.

Rolling forwards to 2000, OpenOffice.org was created, and the StarOffice code released under the LGPLv2 - with copyright assignment. As an advocate - I spent much of my time talking about the project at conferences, and trying to whip up support for working on it, while encouraging people of the justice and reasonableness of Sun's copyright assignment regimen. Subsequently, as Sun open sourced other products such as Java, that copyright assignment regimen was expanded, and strengthened with the SCA.

Some time later, Sun - on the ropes financially - decided (despite having no ISV community) that it might be a good idea to try to monetize the OO.o asset by changing the license to GPL to driving proprietary revenue from ISVs. This came as rather a shock to me. Although we had no proprietary pieces at all, it was obvious that competing with Microsoft Office, in building an ecosystem would be horrible, if every ISV had to pay a single vendor to write plug-ins. Worse, large companies have high getting-out-of-bed payment threshold. Thankfully, after several wobbles, and considerable uncertainty it was decided not to re-license to a restrictive license.

Subsequently, of course I've had to continue working on and off with various projects with strange and wonderful licensing and assignment terms - always with a negative effect. Even the pain of digging out, cutting and pasting some waiver language into an E-mail forms a real barrier to communication. So - that's the part of my experience available for public comment; but, now synthesizing that into something more concrete, and examining the reality, and potential problems.

Copyright Assignments

What are you signing ?

Of course, there are a diversity of different copyright assignment regimens out there; in many different flavors. The best of these include fall-back to a license for jurisdictions where copyright cannot be assigned, cover accounting obligations, copyright registration and so on. One of the most thorough is Sun's SCA.

you hereby assign to us joint ownership, and to the extent that such assignment is or becomes invalid, ineffective or unenforceable, you hereby grant to us a perpetual, irrevocable, non-exclusive, worldwide, no-charge, royalty-free, unrestricted license to exercise all rights under those copyrights. This includes, at our option, the right to sub-license these same rights to third parties through multiple levels of sub-licenses or other licensing arrangements;

However this is phrased this is essentially you transferring (or more usually 'sharing') the ownership right. There is often considerable marketing effort put into the Joint or Shared nature of such assignments, however these provisions are meaningless in most real-world cases. Having a shared right, to a small part of another code-base, is usually not at all useful, indeed it can be argued that this is a mere fig-leaf of reciprocity.

In addition, there is often either a license, or a covenant covering any patents you hold - that read on your contribution, or even some wider, vaguer scope. In contrast to many open-source licenses which have termination clauses to allow the defensive use of patents, these tend not to be present in such assignments. In addition the patent language can be unhelpfully leaky as to what combinations of whose products are covered, and to what extent.

Some - of the better copyright assignments have criteria restricting the outbound license types under which the code could be re-licensed (FSF, Sun, Apache), others focus more on their proprietary licensing intention. Apache's example would be:

In return, the Foundation shall not use Your Contributions in a way that is contrary to the public benefit or inconsistent with its nonprofit status and bylaws in effect at the time of the Contribution.
This hints, interestingly, at some fairly broad, generic protections that come from assignment to a non-profit.

To whom are you assigning it ?

Clearly while studying such a starkly one-sided handing over of your work and rights, it is reasonable to question the integrity, stability and actions of the organization to which you are handing it. Organizations such as the FSF, have traditionally been thought to be good stewards of this trust; although the transition to GPLv3 has diversified views here. Other companies - such as MySQL, appear to have a stable and understood business model built around re-licensing the code; while yet others appear to have no discernible need, or use for such rights.

Other ways to do similar things

Various other methods are used to achieve the same effect. Some common ones - are asking for a very liberal license: BSD-new, MIT/X11, or even Public Domain on the contribution, and then including it into the existing, more restrictively licensed work. These of course have the disadvantage of a lack of clarity over associated patent rights and are often reserved for small code changes. Some use a copyright waiver for small contributions, and yet others don't require much for very small changes.

Weighing the risks and benefits

Of course, what a benefit and a risk is, depends very much on your stance; I've tried to categorize these from the perspective of an active, non-affiliated Free-software contributor. It should of course be noted, that I am not a lawyer.

Benefits

Single owner

Having a single point of contact, a steward for a project, who is simultaneously the legal owner potentially can give flexibility to a project. It is always clear whose code-base this is, governance is ultimately entirely their responsibility and you have a single neck to strangle from a community perspective. This ownership also yields extraordinary power to change things, if change is required that can be a benefit.

re-licensing

On occasion, it is thought necessary to re-license code, this could be for a variety of reasons: to facilitate re-use in some larger context, to meet the changed needs of a stake-holder etc. In these cases, it is easy for a code owner to re-license the code without obtaining consent from all contributors. This contrasts with the real difficulty of obtaining consent from all contributors in an eclectically owned code-base. This is particularly a problem when companies enter administration, transferring their assets to new, expensive, community-disconnected owners. In a similar vein individuals can also (more tragically) expire, leaving new (and potentially unclear) owners of an asset who are unlikely to understand. On the positive side, my experience of encouraging individuals to re-license their small contributions has been that, for the most part, people are only too happy to help the project they loved.

open-source re-licensing

Re-licensing to a new open-source license might be in response to a new threat to freedom identified by developing circumstances such as Tivoisation in the move to GPLv3, or conversely by the FSF re-licensing some fstype code for gnome-vfs2 from GPL to LGPL. Having a single code owner makes it easy to negotiate and effect such transitions.

proprietary licensing

A far more commonly advocated advantage - is to allow proprietary versions of a product to be sold, based on an open source code-base. This has the advantage of providing a simple, traditional, per-unit license based business model, while also having an open source version. In many cases (it is argued) funding technical infrastructure development is hard, and allowing this can provide a large amount of good quality, well maintained, open source software backed by a stable revenue stream.

Proprietary versions can also have features unavailable as open-source, and (often) competitors can be excluded by choosing a license that prohibits others from making proprietary pieces - such as the GPL. This is often called the 'Open Core' model. Variations on this can exploit a patent licensing advantage available to a proprietary version, and - unavailable in general, eg. perhaps for some Media codec.

'defending' the code

Advocates of copyright assignment frequently make much of the need to 'defend' the code from a potential license infringer. It is claimed that this is easier for a sole copyright holder. As a non-lawyer this seems (to me) one of the least comprehensible explanations - is the GPL really toothless without a single owner ? Some of the highest profile license litigations that have been seen are around Linux (SCO), and BusyBox (20 suits at a time) - where there is no aggregated ownership. It seems to me that this is a hard argument to sustain, and begins to look increasingly self-serving.

Closing the code

While the possibility to close the code, and release all subsequent versions as proprietary software may seem a purely negative benefit - yet perversely the possibility of doing this may well allow timid, or inexperienced management to test the 'Open Source' water. This in turn may lead them (eventually) into a full embrace of Free Software. It is also possible that assignment expedites various accounting problems, such as book values of assets, thus avoiding (nominally) expensive write-downs impacting the corporate bottom-line.

Sun's SCA FAQ claims, in answer to How does the Contributor Agreement help me?

It allows Sun to sponsor the projects to which you want to contribute, while retaining the ability to offer commercial licenses. Without this ability, Sun could not responsibly open source code bases such as OpenJDK or OpenSolaris that represent hundreds of millions of dollars of shareholder investment.

Risks

Single owner

The owner with whom you are in relationship, and have happily assigned rights to - with no doubt many unwritten expectations attached, may suddenly, and without warning change completely. The trigger in a business could be acquisition, change in management, inter-corporate politics, shareholder pressure, or any number of other factors. For a non-profit, there may be similar board-room coups, drifts in attitude and focus, or even bankruptcy necessitating asset disposal to the highest bidder.

With a business, when management changes it is easy to loose the legacy of trust embedded in those relationships. Worse, a business may (simultaneously) hit such hard times - that the relational value of pleasing you (as a community member) pales in comparison to the struggle to stay afloat. Companies on the ropes often suddenly turn toxic with huge switches in strategy - cf. SCO's transition from a contributor to Linux, to a litigation outfit. Even worse, a company will always have some threshold where they have a fiduciary duty to sacrifice the community's valuable (but abstract) long-term community status, in favor of financial gain. Of course, non-profits may have similar problems with changes of control, or perhaps gradual change in attitude of those in control.

re-licensing

The need to avoid obtaining consent for re-licensing primarily affects individual community contributors. Of course, there will always be a need to appear to consult the community, and clearly large corporate interests will be either threatened, or placated behind the scenes. A single owner can bring about a death of transparency to this process, leading to a corporatist bent. It is entirely possible again, that it is a company's duty to twist the screws by choosing more restrictive licensing, to try to 'monetize' an asset they control. It is also quite possible to end up in a situation where you have invested in, and assigned rights for contributions to a code-base, only to find yourself having to pay for the privilege of continuing to use it, or even unable to use it. So - by assigning away your rights, you effectively deliberately dis-empower yourself.

open-source re-licensing

While changes here may appear benign, there are a huge diversity of (even OSI approved) licenses with many varying restrictions. It is as easy to switch an assigned code-base that is in popular from LGPL to GPL to QPL, as it is to go in the converse direction. An obvious change here might be the removal of the non-copy-left SISSL option for OpenOffice (something I supported at the time). Particularly retrograde choices here would be licenses incompatible with the GPL.

Often the very logic of community licensing - to encourage maximum use of Free Software - radically conflicts with the logic of extracting maximum financial advantage from proprietary licensing, driven by the disconnect between open vs. proprietary licensing. Thus, the benefit of the cash investment enabled by proprietary licensing is created by having something far less useful (until you pay, or deal).

closing or abandoning the code

This seems unlikely to be a huge issue for any liberally licensed piece of software that is widely used, potentially this may even be positive: encouraging more contributors to help drive the public version. However, for more restrictively licensed code, the situation is different - this could be the impetus to a support based model, a pure community based approach, or simply the death of the project.

proprietary licensing - FUD

The possibility of proprietary licensing puts a potential sales-man in a rather unpleasant place. He could go around explaining exactly how much freedom (say) the GPL gives you, or he could go for the quick sale option, by bad-mouthing it, and playing up the risks, while down-playing the rights given you. That is something I hear of too frequently.

proprietary licensing - confidentiality

One of the problems of proprietary licensing is its opacity. Whereas open source licenses, necessarily are available for all to inspect and comment on, proprietary licenses are frequently not just unavailable, but protected by severe confidentiality clauses and penalties. This lack of transparency has the ability to hide the terms on which people enter and interact in (what can look like) a community. This can significantly hinder interaction, discussion, code inclusion, and be highly corrosive to trust. If you are unable to explain your actions, or even explain why you can't explain - being in community sucks.

proprietary licensing - surrender of rights

Free software enthusiasts tend to see proprietary licensing as somewhat complementary to Free software licensing; with RMS publicly lauding this model. However, in a world of dominant re-licensing monopolies who alone can combine (eg. MySQL) with key plug-ins - it may not be possible to get support from anyone but a single vendor. At this point, it is entirely likely that in your contract you will find terms substantially limiting, not (perhaps) your freedom to modify the code - but certainly your freedom of association: where you can send it (not to a fork), requirements that it must be re-assigned to the vendor, and potentially yet more troublesome terms.

Coming from a community based on openness, these practices are unexpected, and extremely unwelcome, but are entirely commensurate with the normal back-room corporate bludgeoning that occurs daily, safely behind closed doors. Worse - they can yield a spurious uniformity of behavior, which combined with an inability to use forking to encourage positive change can condemn a project to ossification. The appearance of community, without the associated dynamism and substance can yield slow death.

proprietary licensing - portfolio confusion / tying

Another risk of copyright assignment (and the single company dominance that this tends to bring), is of product tying. As a corporation with a diverse set of open source products, it is tempting to form alliances based around supporting one with another. Thus - as a well publicized example contributors to Java - are simultaneously driving MySQL sales, helping promote Microsoft search, and lifting Sun's bottom-line. This is in part made possible by trademark ownership - there are great dangers in building up trademarks you don't control. Nonetheless - a community owned project is unlikely to have made this sort of cross-portfolio choice; and this sort of thing is unlikely to endear Java to a hypothetical PostgreSQL hacker working at Google. There are less well known examples of even greater perversity.

proprietary patent indemnification

If you assign rights, you give the owner of those rights - the ability to offer special indemnity, for patents, unavailable to others covering their proprietary licensed, or sub-licensed versions; creating perverse incentives.

Barriers to entry

The Paralysis of Uncertainty

Since corporate entities may not act rationally, or predictably, it can be difficult to know whether it is a good idea to get involved with that code-base for the long term. That risk of instability creates a barrier of indecision to overcome. Should we re-use this code ? or just re-write it. Sadly far too many programmers, in my experience, prefer to re-write rather than re-use, without any even slightly-convincing excuse like this. Thus, it is more likely, that as an assigning contributor your code will be re-written, and your contribution lost.

The death of trust

In the (fascinating) research paper, in the field of privacy The Best of Strangers it was found that, (paradoxically) presenting assurances as to privacy before asking for data created a significant negative bias against yielding it. While it is extremely hard to measure such a thing, my fear is that the death of trust implied by demanding a signed and faxed legal document before accepting code has a sizeable negative impact - starving your project of contributors.

As an aside, whether it is correct or not, seeing your own, lovingly created software bolted to a Frankenstein (shouting) header reading:

 * DO NOT ALTER OR REMOVE COPYRIGHT NOTICES OR THIS FILE HEADER.
 *
 * Copyright 2008 by Someone Else, Inc.
with no trace of who put the work in, tends to breed feelings of a lack of trust with its absence of fair accreditation.

Corporate unwillingness

If Free software development is turned into a corporate land-grabbing activity, then those areas where new work is being done will necessarily attract acrimony over ownership of the new pieces. New pieces will be positioned as extensions of other old ones, and the potential for conflict, and the spread of ill-will can sterilize the space.

Brian Aker summarises this well, from his experience:

"The conflict inherent in trying to reserve rights, and take the rights from others, leads to conflict."

Furthermore, getting a copyright assignment read, reviewed, re-negotiated, approved, signed takes significant time for those with the least available time: corporate executives and legal. Often there are risks, of patent leakage eg. that complicate matters. Furthermore it is necessary to explain to them why they should invest substantially in adding to code that appears as an asset on their competitors bottom-line. Communicating an advantage, high up a corporate chain can be hard indeed - thus starving your project of timely corporate contributions.

Corporate Overhead for Consumption

These days, much open-source development is driven by full-time, paid, corporate contributors, who can bring a lot to a project. However in many cases where public licenses are commercially unfriendly it is necessary to negotiate some form of proprietary sub-license.

Assuming there is no reasonable, boilerplate license, or that there is conflict and competition between companies - the process of negotiating these agreements can be extraordinarily fraught, and take tens of months. This can consume vast executive, business development, legal and technical resources, and still yield nothing. These negotiations can be torturous, involve brinkmanship and tricks. To require this in order to attract corporate contributors is anathema in a world of rapidly moving development. By the time a corporation has noticed the opportunity, and successfuly concluded negotiation, it is long gone.

Scarcity mentality

Often this mentality grips a copyright owner; the thought that someone else might make a dollar from a combined work, is immediately thought of as a loss for the owner. Thus, perversely, instead of trying to grow the corporate developer base (and creating competition for themselves), the instinct is often to try to suppress and coerce such potential partners and allies. This in turn can make dominance by a single-company self-perpetuating, and further weaken the value and lifetime of your contribution.

Lack of scalability

An assignment based model, tends to lead to the desire to build, and include into the assigned software only pieces that comply with the commercial proprietary licensing needs. This in turn requires the use of liberally licensed components all the way up the stack, until the assigned piece - available under a more restrictive license. eg. X11 below - GPL above.

It is clear, that this is essentially a non-scalable business; it will work well for a single entity in one vertical, perhaps two; but if every component in the stack is owned, and licensed by a different vendor - then all proprietary licenses require a cascade of others. This situation is also of incredible risk - a single malign vendor in the chain could hold the rest effectively hostage. Compared to this, the problem of internet micro-payments is child's play.

Some companies make their intention here quite clear: talking of a [toll] "... bridge between different communities using the same code under different licenses." How many toll-booths can be erected before no-one uses the road - is left unanswered. If this underlying model does not scale, then it is unlikely that other corporations will become involved and help grow your community and software. Where is the opportunity for multiple bridges in this sort of stack:

Delayed Gratification

When you have (often without asking anyone) disappeared and written something wonderful to contribute - it can often be a shock to discover, that before even reading it, discussing it, empathizing, and encouraging you - it is necessary to delay until you have signed and faxed/sent a form, it has been reviewed, and you have been approved by some paralegal. This is particularly frustrating when the change is acutely cool - and you want to get it in immediately; delay is toxic. Worse - you know, that every minute you delay increases the chance of someone else (on the inside) doing the same thing, denying you the pleasure of getting included.

Duplication

Hopefully, with an Open Development model, decisions are made on the basis of clear, visible-to-all, technical merit - indeed, what is technically best yields quite enough discussion without further criteria. Adding another criteria - which goes way beyond licensing (re-writing for licensing reasons is fairly common) - into ownership is far more pernicious. Re-writing a piece of code, contributed by an assignee unwilling to give over copyright - yielding a re-write with the same out-bound license is particularly controversial. The example of Sun rejecting Kohei's solver for OpenOffice.org was a particularly instructive attempt at coercion here - with predictably community dividing results.

Trying to reason and interact with, and understand the decisions of developers who have non-public, corporate commitments to different technologies can be acutely frustrating. As a (banal) example, recently an Intel / Nokia agreement precluded prior discussion of the rational around oFono / Connman.

Technical separation can itself be highly problematic. If a plug-in does not have to be assigned, but (when bundled with the product) it has to be - then suddenly there are non-technical barriers to creating a well integrated product that have to be engineered around. That can affect code re-use, choice of libraries, and so on. If two products need to be integrated, which place the code lives can be dictated not by good house-keeping, but by ownership requirements.

The kind of communal conflict created by such duplication is entirely caustic to the growth of a healthy developer community.

Community strangulation

The combination of these risks, pragmatically appears to lead to a self-stultifying community. Companies that demand assignment, typically attract very few external contributions; so for example we hear that "it is a fact that MySQL was almost fully developed by employees of MySQL Ab and later Sun's MySQL division". Similar patterns appear to hold for other projects. This in turn can become a self-reinforcing argument: no-one else is contributing, so why should we open up ? When no-one else is contributing because the project is insufficiently open, that is a sad attitude.

The counter-examples

I am not aware of a single project that mandates copyright assignment to a corporation that has a diverse, and thriving developer community. If there was even one, the business model of "communitising" a code-base, then firing all your developers, sitting back, and collecting an effort-free rent might become attractive. In contrast I am aware of many diverse and thriving communities that have eclectic ownership, and also some less thriving ones that are dominated by single entities.

Of course, there are a number of entities run by foundations that aggregate copyright that appear to do remarkably well despite that. Common examples are the Eclipse and Apache foundations. These appear to neutralize some of the worst problems of copyright assignment by several approaches: first by eschewing an asymmetric licensing model ie. all users get the same license - a liberal (non-copy-left) one. Secondly, they provide strong, stable, governance structures, that include the major players, and that avoid the possibility of disruptive change. Finally in the case of Apache they actively work to exclude politics from the technical sphere of the project; decisions are based as purely as possible on technical merit. Of course Apache also gets governance right: being an organization based on (roughly) the very same individuals licensing their copyright to the foundation, with no direct corporate engagement. Having said all that, the general "red tape" effect whenever some paperwork is involved is an issue. I asked Gianugo whether he though the apache project would have got going if the whole bunch of Apache policies to abide by (including assignment) had been required from day one, and his answer was "No, probably not." (when exploring these topics as a corporation you could do a lot worse than talking to Sourcesense incidentally).

Naturally, with both Eclipse and Apache, their licenses are not copy-left. This is highly unfortunate, and once again can make it harder to attract contributions and grow community, particularly corporate contributions. A prominent example of the effect of licensing can (perhaps) be seen in Apple's non-contribution to the code derived from the BSDs in it's devices, as contrasted to it's deep contribution to WebKit, as required by the copy-left.

Then of course there are the counter-counter-examples; some of the most successful open source projects require no copyright assignment: Linux, Busybox, Xorg, Mozilla/Firefox, GNOME, KDE, GStreamer, and a whole clutch of others too many to mention; indeed this is arguably the normal state of Free software projects.

Picturing a dystopia

Can we imagine a future world where corporate ownership and deal-making runs to it's logical conclusion ? Clearly, there are problems scaling the dual-licensing business-model, which should presumably lead to a large degree of conflict around choices of technology. Whose stack software is built on, becomes then a question of deep corporate concern, and presumably the stacks will do battle in public and private. Of course, who owns which vertical silo, and where exactly the toll-booths are located, and who signs what co-operation agreements to ensure their access to other silos is of great interest. Clearly, those with multiple assets can use them co-operatively to drive each other forward here.

Another thing that will (no doubt) cause grief is the proliferation of platforms, which will be exacerbated by ownership concerns. No sooner than Eclipse created an IDE, than suddenly it became an application development framework to re-write the world in; arguably, GIMP no sooner factored out it's toolkit, than there was a desktop platform springing up around it. With more claims on ownership - expect a plague of yet more 'platforms', and duplication of applications to use them. This has the danger of further fragmenting the Free software developer base, retarding progress, and destroying value.

Acquisition risk also increases here; without good, widely used mechanisms to ensure the stewardship of a codebase in the event of a hostile takeover, the more agreements that are in-place, and the more consolidated ownership - the greater the risk of a hostile entity purchasing and smothering some key piece of the stack.

But is the wide-spread use of assignment really likely ? A Gartner analyst predicts:

By 2012, at least 70% of the revenue from commercial OSS will come from vendor-centric projects with dual-license business models.
Personally, I profoundly hope that this will not be so, and that we will continue to see rich collaboration among competitors in the world of Free software. There is, of course hope - that individuals, smaller corporations and consultants will see the dangers, rebel against them and invest and adopt only the truly open technologies.

Finally, perhaps the most destressing aspect of assignment, is the evisceration of influence of even the most productive and valued individual contributors. Deprived of insight into the web of proprietary licensing, with insiginificant personal assets to apply swapping games with, and providing an essentially expendible contribution they are placed at the mercy of forces beyond their control. It would be sad beyond belief to work in an un-forkable project, that gets handed technical decisions based on back-room corporate machination instead of merit.

Alternatives to Corporate Assignment

There are a number of possible solutions, that address some of the risks of assignment, or capture some of the benefits without it. Firstly - without assignment, there are two traditional approaches to re-licensing.

Re-licensing

The first, and most obvious is 'plus' licensing, with subsequent version language:

 * This library is free software; you can redistribute it and/or
 * modify it under the terms of the GNU Library General Public
 * License as published by the Free Software Foundation; either
 * version 2 of the License, or (at your option) any later version.
This of course, relies on people trusting the FSF's governance, continuing grip on reality, and is another way of, giving a powerful ownership like right of the code to the FSF. As such, some corporations are very skeptical of licensing their code under a regimen that may change under their feet, as happened in the transition to GPLv3. Indeed, arguably that transition has provoked many to search for new (more dangerous) ways of enabling license migration, such as copyright assignment.

A similar clause can be found in Sun's CDDL:

4.1. New Versions
Sun Microsystems, Inc. is the initial license steward and may publish revised and/or new versions of this License from time to time. Each version will be given a distinguishing version number. ... Otherwise, You may also choose to use, distribute or otherwise make the Covered Software available under the terms of any subsequent version of the License published by the license steward.
Thus, a different license steward can be specified; but of course new versions of the license have to be published.

The GPLv3 enshrines a proxy who can decide whether a new version of the license should be applied (section 14.):

If the Program specifies that a proxy can decide which future versions of the GNU General Public License can be used, that proxy's public statement of acceptance of a version permanently authorizes you to choose that version for the Program.
Which is an interesting twist - giving the FSF the sole ability to create new versions, but allowing the delegation / aggregation of the choice as to whether to adopt them.

Another, common approach is to use the Lesser GPL's ability to convert to the GPL:

3. You may opt to apply the terms of the ordinary GNU General Public License instead of this License to a given copy of the Library.
Interestingly, this is what makes the LGPLv2 more compatible with GPLv3, than GPLv2.

It appears (to me) that choosing a license that can be upgraded and bug-fixed in-flight by a responsible steward or proxy substantially removes the requirement of assignment for re-licensing.

Finally, another - perhaps less optimal - approach to re-licensing is to choose a license so liberal that it can be converted or incorporated into another licensed work, if necessary later, eg. perhaps the BSD-new license.

Assignment with a non-profit

Of course, this can be indistinguishable from assignment to a company if the governance is not open and equitable. However, assuming reasonable governance, this method can allow re-licensing, defense of the code, and stability - without undesirable asymmetry and proprietary antics.

Joint assignment via a non-profit

As a less optimal, but pragmatic approach - where a company is open-sourcing a large existing code-base, with tangled proprietary agreements in-place surrounding it - onward assignment from a foundation might be the best approach. Thus two entities would both, individually own the complete aggregate work.

This has the benefit of ensuring continuity, and stability allowing a company to continue it's proprietary business, and potentially providing community governance too. By requiring suitable voting super-majorities for changing the constitution, and therein proscribing high thresholds for any sunset clause - a clear path via contribution to a level playing field can be provided.

A much more limited, constrained, and less open (yet interesting) foundation, similar to this was created for the FreeQt foundation - that protected Qt from strangulation by acquisition, or sudden death by bankruptcy.

Recommendations

Condensing the risks and benefits, there are perhaps a few clear recommendations - for those corporations and individuals who are interested in open, thriving development communities built around their software.

  • As a contributor - avoid copyright assignment to corporate entities wherever possible.
  • As a project leader: use a license, or dual license combination that provides the ability to move to another license.
  • Use a plus-license of some sort, and include or any later version in your [L]GPL headers. If you cannot stomach this; then when using a specified version of the [L]GPLv3 - specify a proxy whom you trust to approve such things: preferably a well governed, pragmatic non-profit that you feel represents you.
  • If you feel you must demand copyright assignment, consider doing so via an onward assignment from a well-governed proxy non-profit. First however consider using an extremely liberal, GPL compatible license as an alternative, preferably one that is copy-left.
  • If signing away your rights appears unavoidable - and all attempts to change the situation: selecting different software, re-writing the software, joining forces with others to fork the software etc. fail, then prioritize assignments that:
    • Have strong guarantees, and a commitment to future open-source licensing, eg. the FSF's, or Sun's SCA
    • Refuse to sign an assignment that does not ensure that licensees retain their freedom to fork, and contribute to any open source project
    • Mandate that all proprietary licenses will be under F/RAND terms, and (preferably) published - this is notably absent from any existing agreements. Failing that, demand that the ownership should be administered by a separate (for profit) blind entity.
    • Pressure non-profits demanding assignment to have clean, transparent, and representative governance - so your voice is heard.
  • If you contribute to, or build software based around assigned projects where there is, or may be substantial licensing asymmetry, and conflicts of interest - eg. a project under the GPL, be aware that you are entrenching a non-scalable approach that may ultimately harm your contribution. Please do not try to present your project as 'open', or demonise those wanting to change the situation. Also - don't accept sham community governance that has no legal existence or force.
  • If you want to open-source your code, choose a license for it that you, yourself could accept. If you are a business, consider what a competitor would want - this way, you can ensure the critical survival of the code-base; if that means using a liberal license, please use the LGPL as well, as an option in a dual license (eg. Mozilla's MPL / LGPL). You should also strongly consider choosing a copy-left license, to encourage contribution back to the project.
  • If you are faced with aggression from a copyright owner, turn their asymmetry against them: ask them to accept code under the same terms they provide to others. If they become upset, point out that you're happy for them to assign their rights to you, and license them back. Failing that, just soft-fork the project, a-la MariaDB - paradoxically you may want to collect copyright assignment yourself to be able to affect an eventual reconciliation.
  • If you are considering dropping a copyright assignment requirement, and concurrently re-licensing, be encouraged - this is a good way to go. Experience suggests this will diversify the contributor base, and increase adoption of your project. If you have a substantial book-value for the original asset - it is worth re-considering such valuations anyway: do they truly reflect the commercial gap between the public and private license ? are they accurate, given the high degree of volatility between commercial needs and the license of the day ? and their fragility: could they be sustained in the presence of a popular fork ? can the change be viewed as a gradual tax-efficient depreciation of an old re-licensing ability ?

Conclusion

Copyright assignment brings some benefits, but also creates very substantial risks of damage to the Free software developer ecosystem. In more simple terms: do you want to work with other companies and individuals, growing code in a social environment built for the common good ? or do you want to work disproportionately for the good of a single entity ? Viewed from a social perspective - such open development - against the odds, herds competitors together, and excludes business concerns in the single minded, focused quest for technical excellence. Poisoning that arena with corporate edicts, issued from a faceless business elite who are not present is highly unwelcome. The risk of creating a dysfunctional community is of course highest when the assignment is to a corporate entity, and the license is restrictive, yet there is some real risk of backsliding and malfeasance wherever copyright assignment is practiced.

As a thought experiment, if you had two projects to choose from which you felt inclined to help out, or re-use in your project: would you choose the one where you were assured of subnormal community growth, dominance by (and dependence on) a single corporation, regular inter-corporate conflict, unpredictable changes of license and being surrounded by a web of invisible confidential agreements ? Or - would you be more likely to choose a diverse, non-vendor-dominated community ? how about if that community had poorer quality code, and needed more people to work on it ? (incidentally, that is often a great time to join a project).

I believe, that copyright assignment, (in some way like patents) grants a monopoly to the holder to dictate terms to, and coerce others in ways that are not public - and, as such, (depending on the ethics of the holder), is always potentially dangerous - and to corporate entities is profoundly unwise. In the absence of better re-assurances in assignment agreements, it should only be socially acceptable among developers for projects that can be effortlessly avoided.

Of the current clutch of copyright assignments I've seen, I personally think the SCA is one of the most thorough, although I have yet to see an assignment that guarantees the level of fairness and transparency necessary when sub-licensing.

When assessing whether to assign rights pragmatics needs to rule; in some cases there will be an alternative that is simply more expensive or difficult. I encourage you to, wherever possible choose that - it is likely to be cheaper in the long run. Failing that, trying to ensure that your contributions to the project (at least) are made available to all under fair terms seems a reasonable thing to achieve, perhaps by getting a commitment to using an extremely liberal license for a certain element - from the vendor. Ultimately, adding copyright assignment to your project is a huge burden to bear (while swimming) - but to a reasonable non-profit, with good governance - this can (apparently) be compensated for by building an even better, more open and healthy culture.

Finally it is just common-sense - don't move into your partner's ramshackle house, and pour years of your life into renovating it to a level of re-sale-able perfection, without taking your partner to the altar first, and you don't get married in a confidential ceremony. Civil law is there to help prevent the injustice of having your love and support sold out. A refusal to consider protecting the rights of the weaker party tends to confirm an intent to exploit. This of course applies as equally to trademark as to copyright ownership.

Further reading

Of course, many others have written on this topic, no doubt far more lucidly than I, and with different perspectives.

Navigo card decoding

After buying the card reader, I had to use it, so I had a look at the content of my Navigo card.

Navigo is a French transport card, implementing Calypso standard

« The European standard for ticketing data (EN1545) has also been contributed by the actors of Calypso. After a few years of trials, the system has been generalised in the early 2000's in major European cities such as Lisbon, Paris, Venice, later followed by Milan, Porto, Marseille, Lyon, Turin, and many smaller cities. Calypso is extended now in other countries such as Belgium, Israel, Canada, Mexico, Colombia, etc. »

So, hopefuly this info will be useful to people in these places. Please let me know :)

Seat in Bois-Colombes
Seat at my train station under snow on December 17

EN1545 is an expensive document, so I could not access it and had to reverse the data in my card, here is what I found. All of this is based on data collected with Integrale subscription. Bits are numbered starting at 1 because I used cardpeek to work on the data, which uses lua where we count from 1.

Before looking at the real data, all the cards I saw where 3B6F0000805A080303000000CARD NUMBER829000 with CARD NUMBER using 8 hex digits.

Event log

Then we can start with event log which is the one I spent most time one. It contains the last 3 uses of the card. Reading it for a few days, and reading the ones of my colleagues gave me plenty of information (many thanks #bureau3).

I have split each record into two parts, the first part is 68 bits long, the second part is 38 to 86 bits

First Part

1    5    9    13   17   21   25   29   33   37   41   45   49   53   57   61   65
DDDD DDDD DDDD DDTT TTTT TTTT T001 0000 0000 00BB CC10 1000 1010 0KKK KRRR RXXX XXXX
  • D: Days since 01/01/1997
  • T: Minutes since midnight
  • B: 3 for Bus, 0 for other
  • C: 1 for Metro and some Bus, 0 for Train and some Bus. Related to length of second part
  • K: 1 for Bus, 3 for Metro, 4 for Tram 5 for Train
  • R: 1 for Entry, 2 for Exit, 6 for a Metro to Train transition, 7 for Train to Metro transition
  • X: 58 found on a non RATP bus, 1 for everything else

Second Part
This part is varying a lot.

  • For SNCF trains it is 38 bits long, for metro or RER, 54 bits.
    69   73   77   81
    YSSS SSSS AAAA A??? ... ?000 01
    • S: Sector
    • A: Station

    For a list of sectors and stations, refer to this page and this one.

  • For bus, if C=0, it is 68 bits and Bus number is 13 bits starting at bit 121. Else it is 86 bits and Bus number is 13 bits starting at 137. Bus line is sometimes 8 bits starting at 110.

Contracts

  • 14 bits stating at 86: Days since 01/01/1997 for the contract subscription date
  • 14 bits stating at 100: Days since 01/01/1997 for the contract end
  • 8 bits starting at 114: Allowed zones as a bitfield, 114 is zone 8, ..., 121 is zone 1

Miracle

After finding all of this I found a PDF on Google describing coding for interoperability in France between cities using EN1545. I was happy because it both confirmed that I was right on most things, and because it contains a lot of information, like the additional codes for other kinds of transport, the complete coding for event log (the changing length indeed depended on some previous bits, each structure has a bitmap telling which fields will be there).

All the result of my findings, and some from that PDF are in a fork of cardpeek that I maintain there (because the author did not reply to my email with a first patch one month ago). I fixed a few issues in cardpeek but not the main one: it copies lua decoders to ~/.cardpeek/scripts the first time you run it and they never get updated. To workaround that, I replaced that directory with a symlink to my git checkout...

While collecting data I had some surprise, for example Tramway T4 in one station (La remise à Jorelle) did not write anything to event log, does this mean that they can't control you? T3 does, and T4 did in the next station Les coquetiers.

More expected, using your navigo card to get a Velib bike does not write anything, they just use the id of your card.

Plenty of Velib available today
Velib station under snow on December 17

I also happily could not find some data that I would not have liked to find as they are accessible via RFID too, like my name, birth date, ...

2009-12-21: Monday.

  • Up lateish; J. off to counsel a young lady. Sandy arrived with presents - how kind; had a cup of tea.
  • J. back for dinner. Hannah & Joni popped over, and talked / played world of goo - ah, holidays are wonderfuly refreshing things.
  • Snow fight in the garden with J. and H. and latterly N. - with M. stumping around making a snowman and complaining when it got caught in the cross-fire: much fun. E. sleeping.
  • Tony & Janice popped around, good to see them too. Veggitales DVDs for the babes.
  • To work - plenty to catch up on, chewed over the E-mail pile, Clarity, polished up copyright thoughts, and posted them. Bed.

How to use interruptions to your advantage

When I first started public speaking, I loved when I had technical difficulties. Focusing on them - problems I was familiar with - calmed me down.

Today started out with quite a few technical difficulties. First my SO's car broke down on the way to work and I had to go get him. (About an hour roundtrip.) Then an hour later daycare called and said my 3 year old was sick so I had to go get him. (About 40 minutes roundtrip.)

So I'm trying to use these technical difficulties to my advantage. I'm squeezing all important work into nap time. I have to be very productive for two hours!

And then I think I might just take the rest of the day off. (Which can be productive in a different way.)

What do you do when you run into too many "technical difficulties"?

I'm back!

I finally got a new server and moved my blog to it. So hopefully all my hosting problems should be fixed now. :)

Reconciling

The discussion with Richard Stallman ended with requested silence.

I’d like to ask people to use the following meme for future such discussions:

<quote>La pensée ne doit jamais se soumettre, ni à un dogme, ni à un parti, ni à une passion, ni à un intérêt, ni à une idée préconçue, ni à quoi que ce soit, si ce n’est aux faits eux-mêmes, parce que, pour elle, se soumettre, ce serait cesser d’être.</quote>

Henri Poincaré, University of Brussels (1909-11-19)

As for the conclusions. I believe this survey result and this analysis are conclusive.

WebKitGTK+ Hackfest – Day G_MAXINT

Haven’t blogged about the hackfest since the day zero (although others have done a great job), but I guess I have a good excuse since we have been working all day every day, no time for blogging!
inmocoruna-torre

A lot of progress was made in many areas, but I can try to give a brief summary:

  • Gustavo and myself focused on fixing the form password saving regression during the first days. We wrote the basic code to hook into the webviews using JSC, store the auth data in the keyring, and refactored the Epiphany codebase a bit to be able to show infobars with the available options when submitting a form, like most browsers do. I know this was one of the most painful shortcomings of the browser for a lot of people, so I’m happy to put it behind us.
  • Dan, Benjamin and others spent a lot of time hacking on libsoup. The Content-Encoding support landed in both 2.28.2 and master, which should make us render correctly some pages instead of showing gzip compressed garbage. Other things of interest were the GIO Socket support, a requisite for better SSL support (including certificate management) and the so-called URI Loader, which was redesigned and advanced enough for Dan to be able to write basic about: support for Epiphany/WebKit and that is a blocker feature for mail clients that need to support CID URIs (like Modest or Evolution).
  • I spent some time finishing my refactoring of the work previously done in the DOM bindings bug. We are not there yet, but I have a quite smaller patch with a reduced scope (supporting a subset of the Node API) which already works and that I’ll try to upload when I’m back in Finland and that should be easier for the reviewers to accept as a first step.
  • We enabled the page cache support in WebKitGTK+, which uncovered a series of interesting issues that we spent some time fixing up. On top of that Álex also landed the first step of better cache management APIs, a widely requested feature. Both will be important in providing faster and more responsive web content for our users.
  • Oh my God, have to catch a train to Vigo soon and this list is endless!
  • Evan and Behdad teamed up to work on a Harfbuzz font backend for WebKit. They were able to make it show some fancy text by the last day, and the current plan is to land this at some point in the first half of the next year, with WebKitGTK+ and Chromium/Linux sharing most of the implementation.
  • Cody worked on using the new offscreen rendering support in GTK+ for our theming code. He made great progress, identifying some new features we need on GTK+, and hacking up a GtkOffscreenContainer that might end up going into GTK+ upstream and that was enough to make his proof of concept implementation actually show properly themed widgets in a page.
  • Philippe kept rocking on the media side of things, as usual. He finished the HTML5 media control patch which Zan started, and made other nice fixes to our increasingly awesome media support.
  • Martin Robinson worked on various cool things when he managed to find some free time in between his eternal quest to find his missing luggage (which AFAIK he only managed to get back on his way back to Barcelona!), including transparency support for WebViews and his improved DnD code.
  • What else! We worked on accessibility bugs and random epiphany issues (custom User Agent support by Vincent Untz, send the proper Accept-Language HTTP headers on requests by Mario, respecting web context menus including a way to override them to show the epiphany menu…), making our regression list shorter and shorter, and other invisible but important things like making our binaries both smaller and faster to build.
  • We reached some important conclusions on the gaming side of things, like Smash Brothers for the Wii being an insane game, proving that my Ryu totally owns Gustavo’s Ken or finding out about Benjamin’s past life as a Starcraft quasi-progamer. Speaking of games, don’t forget to enter the Konami Code in Epiphany 2.30 :D .

00016324

I’m probably forgetting things, but that’s about it. I think we all had a great time, great progress was made, and I want to thank Igalia, Collabora and the Foundation for their sponsoring and support for the event. Let’s do it again next year!

GNOME 3 platform cleanup

Long time no update on the Cleanup part of GNOME 3, hence if somebody wants to spend the Christmas days with hacking a bit on boring stuff, here’s the ToDo list! ;-)

And in the extended basket:

  • Killing libsexy: vino
  • Killing HAL: See the open dependencies of the meta bug
  • Killing XULRunner in favor of WebKit: yelp
  • So, what did I forget in this quickly written list? :-)

    Nice to see more and more modules getting Introspection Support.

    Worth to consider: XDG config folder implementation.

    Also I’d like to give a big “Thanks” to Javier Jardon for working like mad by both filing bugs and providing patches in these fields, especially when it comes to GTK+.

    Zomg EtherPad


    So my friend Corey told me about how Google recently bought a company called AppJet who makes this sweet online shared notepad called EtherPad. Google went ahead and released EtherPad under an Apache 2.0 license. Sweet; thanks Google!

    I could see Gobby hooking into EtherPad – could you?
    Enjoy some screenshots – and I apologize for being so short with this post; I am a busy bee today finishing things up before I leave for the holidays tomorrow.

    Main Document Window

    Document Versions Timeline

    You can play forward through versions…. so sweet…. I so wish MediaWiki had this!

    Import/Export Options

    Would be super sweet if you could import a MediaWiki page then export it back out from where you got it. Or at least export in MediaWiki format.

    Document Revision History

    What do you think? Could this app be useful for Fedora?

    Posted in Uncategorized

    Mon 2009/Dec/21

    • A note about hackfests

      If you are a proprietary software company and you don't employ all the world's experts in a certain domain, well, you are essentially screwed.

      But if you are a free software project, you can gather most of the domain experts at a hackfest, and be totally awesome.

    Javascript support in anjuta

    Finally, after far too long time I found time today to merge Maxim’s great Javascript support patches from GSoC 2009 into master. You find screenshots and more information on his blog.

    Thanks a lot for your work Maxim!

    P.S.: Remember that scipting is for loosers while C is for real (wo)men ;)

    Stormy's Update: Week of December 14th

    This is my update for work done for the GNOME Foundation, reprinted from the GNOME Foundation blog. For a higher level overview for what I do as the Executive Director, see What do I do as Executive Director of the GNOME Foundation? or my earlier updates.

    Pinged some people to help with GUADEC by mentoring or creating timelines.

    Had a great conversation with Sara Crouse from Wikimedia about how they find, apply and manage grants, like ones from the Hewlett Foundation and the Ford Foundation. Looking forward to implementing some of that at the GNOME Foundation.

    Spent an inordinate amount of time and energy on the threads that started on the Foundation list and the side conversations that came out of it. Assuming that even a fraction of the people involved spent that much time and energy, we need to figure out a more effective way to have those conversations. Obviously we could use some more self moderating earlier in the thread. Perhaps other forums, like the all hands IRC meeting we discussed in the past could also help.

    Took some time off to deal with a personal crisis. (And no, it wasn't Christmas shopping - I haven't done that yet!)

    Provided a quote for OpenDesktop.org and OpenSuSE build press release.

    Pushed a number of things and projects along from missing payments to projects with no activity to potential partnerships. Hopefully they'll all be moving forward soon and I'll be able to report on their success.

    Posted a couple of blog posts. I really hope GNOME is able to set an example for how free software projects can transition to the web services world. And I think we all need to keep not copyright law in mind, but the fact that we are representing GNOME in public in mind as we post to public forums. Quite a few press folks and journalists picked up on the Foundation list threads and we certainly didn't look like we were following our own Code of Conduct.

    Had a really good weekend with the family.

    My first (major) GTK+ patch

    Matthias Clasen cleaned up and pushed my first major patch to GTK+ yesterday. It allows the underlines on menus and buttons to be hidden until the alt key is pressed, similar to behaviour seen on recent versions of Windows. It is configurable via GtkSettings and off by default.

    I’d like to thank Matthias and Emmanuele Bassi for their help reviewing the patch and to Intel for allowing me to develop it at work (it was a requirement for Moblin from our illustrious interaction designer, Nick Richards).

    The final menu behaviour is a little different from my original idea and I’ve attached a patch to the bug that implements my preferred behaviour. You can try out the new setting by grabbing GTK+ from git and adding gtk-auto-mnemonics=1 into your ~/.gtkrc-2.0 file.

    kbd stuff going threezero

    Since gnome panel and applets are about to go relatively soon, I feel obliged to do something about that. Here are some unchained thoughts related to it…

    The gnome keyboard indicator applet is to become deprecated. What’s instead? The only viable option, no matter how I dislike it, is the notification area icon. Or, in terms of gtk+, GtkStatusIcon. What’s important is that it cannot be a Gtk widget (there are good reasons for that). So the widget gkbd-indicator that was used by the existing applet, is getting useless – I just can reuse some of its code.

    But there is an issue with gnome-screensaver. The screensaver is using gkbd-indicator widget in the unlock dialog (and it has to stay there!). That means I’ll still have to support the widget (unless, hehe, gnome-screensaver would suddenly start implementing notification area conventions). As the result, libgnomekbd will have two “layout indicating UI elements”. Somewhat ugly, on my taste…

    PS The GSwitchit Plugins will not be supported by the new GtkStatusIcon-based indicator.

    PPS It is still a shame that one cannot easily produce GdkPixbuf using cairo. The way through GdkPixmap does not work correctly because the transparency is lost by gdk_pixbuf_get_from_drawable (”If the specified destination pixbuf is not NULL and it contains alpha information, then the filled pixels will be set to full opacity”). The only way is using some code snippets. Why cannot that be put into gtk?

    gdk_pixbuf_get_from_drawable

    Translate Simple Scan

    One of the great features of Launchpad is how easy it is to translate applications. Make sure you have a Launchpad login and then you can translate Simple Scan from your browser!

    Your translations need to be reviewed by the Ubuntu Translators team, if you are a member please review any translations!

    Here is the UI for 0.7.6 in Romanian:

    this is what goes around; and this.. this is what comes around

    It used to be that no applications would compile for 64-bit architectures, because everyone was trying to cram pointers into ints. Today I had the opposite. The head of some code I'm working on wouldn't compile in a 32-bit environment, because someone was trying to store 5 bytes in a long.

    We have truly come full circle.

    December 20, 2009

    Last day of the hackfest

    Today was the last day of the hackfest. We’re all tired.
    Behdad and Evan did more work on Harfbuzz integration and can now show kerned text created by the new text layout engine.

    Kerning in WebKitGTK+ with Harfbuzz

    Dan made a patch so that WebKit no longer eats about: URLs.
    Diego worked on reinstating the safe/unsafe mime types code.
    Gustavo enabled the Page Cache that we thought was enabled all along (!).

    Mario worked on fixing a regression with regard to toggling image zoom. I’ll leave the last word to him this time!.
    Sponsored by Gnome foundation

    » Plaats op eKudos

    WebKitGTK+ Hackfest is over

    I’m writing this post from the Igalia office on Sunday at 23:00 pm and no… I’m not crazy nor I was expulsed from home or something like that. It’s just that today is the last day of the WebKitGTK+ hackfest and, although  was not officially an attendant to the hackfest (see http://live.gnome.org/WebKitGtk/Hackfest2009), I don’t regret to have joined yesterday, since it was a quite productive and interesting experience, and hopefully quite productive as well for the projects that most of the people here were working on during the whole week.

    To be honest, in my case there’s not much stuff to to talk about but just a couple of patches for libsoup, epiphany and WebKit that I was working on (some of them already got in), but I can assure you guys that the rest of the people (I mean, the “official” attendants) didn’t stop working even for a second during these days, getting awesome results, if you ask me (although of course they also found some time for relaxing going out for a walk, or even playing the Wii here in the office):

    Martin and Xan

    Martin and Xan, with the blackboard behind (and some results already visible)

    And that’s all folks, now I’d better go home to have a (deserved) rest since tomorrow I’ll still have to wake up early in the morning and to make the most of my first day of holidays (yes!!!).

    See you guys and hope next year we can repeat this experience (and I hope next time I’ll be able to join full time, also). Just a last picture to finish this post…

    WebKitGTK+ Hackfest 2009

    WebKitGTK+ Hackfest 2009

    Categories

    Just added categories to my blog just for blogging, in English, about GNOME and GUADEC 2010. Followed the RSS 2.0 spec. This should work. And now for some nice English content...

    Bluetooth DUN in packaged form for N900

    Over the last couple of weeks, I’ve been working on packaging up my bluetooth dun script for easy consumption. It’s been through a few iterations and is now in the ‘extras-testing’ repository and should be ready to go into the main ‘extras’ repository once it has enough testing feedback.

    The extras-testing repository is not intended for un-adventurous users, but if you’re interested in getting my package, it’s the place to go to. In the latest version, it will correct start the service when you install the package, and shut it down when you remove it. The actual upstart script hasn’t needed to change since I fixed the dependency issue.

    And on an unrelated note, there’s now a way to reliably trigger the portrait mode ‘hack’ - if you want to try out portrait mode browsing, etc. You can find that here.

    Gobby 0.4.93 is available

    We have released libinfinity 0.4.0 and Gobby 0.4.93, the latest version of our collaborative text editor. If you ask me, the killer feature in this release is with the user list - see for yourself:

    Before: After:

    However, there have been other major changes as well, including:

    • Chat: The chat known from the stable Gobby 0.4.x series has been reintroduced. It now also features a backlog.
    • HTML Export: Documents can be exported to HTML, keeping both user colors and syntax highlighting.
    • Color Reset: User Colors in a document can be reset via Shift+Ctrl+C, or via the menu. This allows to get an overview of most recent changes when you want the user colors to be an indicator of activity, rather than of history. Colors are reset locally on the client-side only.
    • Password Protection: infinoted 0.4.0, the dedicated server, allows to set a simple global password for the server. More sophisticated authentication mechanisms are planned for the next release, libinfinity 0.5 and Gobby 0.4.94.
    • UI Enhancements: The user interface is nicer at many points: The browser list is now sorted alphabetically, the file chooser allows to open multiple files at once, the direct connection entry has a history to easily connect to hosts to which one connected before already, on a failed connection one can easily retry the connection via double clicking the host in the browser list, user colors adapt to the background color of the text view (read: text is readable with dark themes) and several more enhancements and bug fixes.

    The very rough plan for the future is to add authentication support in 0.4.94, and displaying carets and selections of remote users (all protocol code is already there, all that is left to do is to draw them into the TextView) and self-hosting for 0.4.95 and 0.4.96.

    The downside of this release is that it breaks compatibility with earlier releases, especially libinfinity 0.3.0/Gobby 0.4.92. I am sorry; I know this is annoying, but maintaining backwards compatibility is as well, and my excuse is that Gobby 0.4.9x is still experimental. It allows me to concentrate more on the interesting stuff. However, if it is not too much trouble then I will try to keep compatibility for the next release. For Windows users it should be very easy to upgrade since an all-in-one installer is provided. Philipp Kern promised to get it soon into Debian and Ubuntu, so I hope the compatibility break will not cause too much trouble. Our public server on gobby.0x539.de runs 0.4.0 already. None of us is using Mac OS X (anymore), so we can't provide binaries for it, unfortunately. If anyone creates a nice bundle, we would be happy to distribute it on our website.

    I also want to express my thanks to Gabríel A. Pétursson and Benjamin Herr who made significant contributions to this release. Beginning from January, Benjamin Herr will be paid by the KIT's department of computer science to implement SVN integration and LDAP authentication, so these are likely to be available in one of the future releases as well.

    In a related note, we have also launched http://infinote.org, a Wiki to host information on the infinote protocol used by Gobby via libinfinity's implementation. I plan to migrate our protocol site to it, and to add more detailed and more structured documentation there. Also, Kobby, Gobby's KDE counterpart, has had another release recently. It still uses libinfinity 0.3.0, but I expect that to change rather soon. Simon Veith has implemented the adOPTed algorithm and the transformation functions as used by infinote in Javascript, in the scope of the jinfinote project. It passes all of libinfinity's tests, and he is now working on a web-based editor which can talk to Gobby and Kobby. An experimental demo is available already, however it does not interfere with Gobby yet.

    In another related note, the Gobby IRC channel moved to #infinote on freenode.org a few weeks ago. Actually we wanted #gobby which was already registered earlier though, however #gobby was registered earlier already even though it is unused nowadays. We try to get through the FreeNode bureaucracy to get the channel dropped, but it seems that this may take several months, if not years.

    The Great Australian Internet Blackout

    The Great Australian Internet Blackout is a combined online and offline demonstration against imposed online censorship. We’re collaborating with Electronic Frontiers Australia (and hoping to bring on similar organisations soon) to make sure every Australian knows why this draconian policy is unacceptable.

    We’re gathering steam within the online community opposed to this policy, then broadening our audience with offline outreach efforts. Our first big demonstration will be during the week of Australia Day… websites across the country with go dark for the week, and we will celebrate the national holiday by joining the traditional Australia Day public parties across the nation, wearing black (hey, it’s a “blackout” after all!) and informing our fellow citizens about the threat of imposed online censorship.

    Check out the website for more info, follow @OzNetBlackout on Twitter and get involved via Facebook group and events.

    Click here to view the embedded video.

    WebKitGTK+ Hackfest!

    This was a nice week in A Coruña, Spain. The WebKitGTK+ Hackfest went well I think :) The Igalia office is great for such an event, the organization/infrastructure was well done, the hotel near the office was awesome, what else? We managed to get things done code-wise! Even though the list is big some items were proudly erased from the TODO like:

    • Dan started a rewrite of some parts of libsoup with gio and worked on the soup URI loader
    • Xan and Gustavo worked on form authentication saving in WebKitGTK+/Epiphany
    • Xan did some good progress on DOM bindings
    • User-Agent support in Epiphany
    • Page cache control support in WebKitGTK+ was merged by Alex
    • Alex did some work on a11y support for WebKitGTK+
    • Bedhad and Evan worked on Harfbuzz integration (instead of Pango)
    • Reinout built a list of Epiphany regressions to fix
    • Evan and Benjamin worked on improving the build process (size and time)
    • Martin and Cody worked on ARGB windows support for WebKitGTK+
    • Benjamin went on war against crashers
    • and I worked on merging HTML5 video controls and the media tests
    • probably a lot more I didn't catch during all the discussions ;)

    Still some work to do though, like for HTML5 audio/video support:

    • on-disk buffering
    • fullscreen view
    • closed captions (this seems very specific to SMIL)
    • pitch control (when scrubbing)
    • enhanced controls UI
    • and still some media tests to fix :)

    So thank you to Igalia, Collabora and the GNOME foundation for sponsoring this event!

    Melbourne Tram Tracker for the N900

    So Collabora's robotic and non-robotic overlords very graciously bought everyone on staff an N900 for Christmas. In my opinion, it's actually a very nice phone (although possibly a little on the large side); but the let down is there just isn't the same host of applications for it. Still, possessing both the tools and the skills, I figured I should do something about this, rather than complain.

    One of the most useful iPhone applications in Melbourne is the real-time tram tracker. For stops without a display board, you can type in the stop ID and get the upcoming arrivals at that stop. You can also find nearby stops via GPS and a bunch of other things. It turns out that Yarra Trams offer a SOAP WSDL web service that is reasonably well documented, so I've spent a few days putting together a basic tram tracker for Maemo 5 (even if only two people will ever use it).








    It currently can show upcoming trams for a stop by ID or by searching for stops by road names. Could possibly also do things like search for stop by route. There is a lot of information available. It doesn't yet do searching by location; the information is in the database, I've just not yet looked at how the location APIs work yet. Also need to add support for storing favourites.

    I also want to add support for tracking a tram by tram ID. I'm wondering if it's possible to use the GPS to detect periods of immobility and check the upcoming tram stop after the tram starts moving again. I habitually miss stops; so what I think would be neat is to dial in a stop number or cross road you're looking for, and have your phone notify you when you're approaching it.

    The web service uses python-suds, which is unfortunately not packaged for Debian, so I can't just rebuild it for Maemo (if anyone wants to package this up for me, that would be really awesome). Then I'll find out how well my app actually runs on the device.

    In case anyone cares, the source code is here.

    December 19, 2009

    A week at LATU (part I)

    LATU stands by Laboratorio Tecnológico del Uruguay and is in charge of the technical aspects of the Plan Ceibal, as well as the overall coordination of the project. As part of their responsibility in making the software deployed on the OLPC machines as useful as possible in their context, they are spending considerable resources in the development and support of Sugar.

    Because they were until recently on the initial phase of the project, they have carried that work on their own, without participating in the community nor working together with other deployers of Sugar. Now that every child in public primary schools in Uruguay owns a XO laptop, they are starting to think about optimizing their Sugar work by pooling resources with others and coordinating with upstream's schedules. The LATU also has a big interest in that their work benefits other countries using Sugar, as part of Uruguay's participation in international cooperation. I have the hope that most of what they learn when working with Sugar Labs will help them take the most of other free software projects they use, such as GNOME (chances are the laptops that will be given to high schoolers in the next years will be running that).

    The concrete issue they found when developing on their own was that every time that their upstream (OLPC) produced a new image build, in order to benefit from the improvements in that release they had to apply the customizations made locally, solve any conflicts and retest everything. If they had contributed those modifications to Sugar, OLPC images would have come with them and no further work would be needed. Reaching the point in which they can directly use the OLPC images as-is is still a bit far away, but every bit that they integrate upstream is a step in the right direction and reduces their development and support costs. Also, when their employees work within the communities that maintain their software, they work directly with the most qualified engineers in those technologies, increasing local capacity.

    Through Walter's mediation, Miguel Brechner contacted me a few weeks ago proposing visiting them in Uruguay to explore ways for integrating their processes with those in Sugar Labs and other free software projects they use such as OLPC. I was lucky enough to be able to take the chance to also attend Ceibal '09, a wonderful event about which I blogged here.

    Another happy circumstance is that the LATU took the initiative of inviting Paraguay Educa, the deployers of OLPC and Sugar in Paraguay, to send one of their engineers to Montevideo in order to participate on the sessions. It was a big pleasure meeting Martin Abente (tch in #sugar), who wrote about his trip (in Spanish) here and hope I can meet in person the rest of his team soon.

    This is the first time since a year ago that I have been paid for working on Sugar, so I have high hopes in that we are going to find soon business models that will sustain further development of Sugar. Would be a sad situation if upstream development of Sugar had to halt because the core developers had to end their involvement in order to fulfill their real-life responsibilities. If that happened, deployers of Sugar (responsible for more than a million installations) wouldn't have a place where to pool resources and coordinate their work.

    I'm going to blog two more parts about the week I spent there, one focusing on the technical aspects and the other will be about the management and community side of things.

    WebKitGTK+ hackfest days 3 & 4

    Welcome to the third report right from the WebKitGTK+ hackfest!
    A nice walk along the windy coastline of A Coruña blew some fresh air into the heads of the hackers gathered here. Behdad made a photo of the group next to the Torre de Hercules:

    WebKitGTK+ hackers standing next to Torre de Hercules

    WebKitGTK+ hackers standing next to Torre de Hercules


    Notable hackfest results since the last blogpost include:

    • Diego fixed opening the user manual from Epiphany, even though he isn’t physically present at the hackfest
    • Benjamin has been busy fixing crashes
    • Christian worked on the context menu in text areas, and spell checking
    • Alejandro worked on middle button paste within the same page
    • Philippe merged the HTML5 video controls
    • Xan added support for customizing the User Agent string
    • Yours truly compiled a list of regressions from Epiphany/Gecko and triaged loads of bugs

    Sponsored by Gnome foundation

    » Plaats op eKudos

    Status Popup for GWT (Google Web Toolkit)

    For my English Book price comparison site CheapRiver.com I wanted to use similar status popup panels as Google is using for it’s web applications. For example the small yellowish block which says “Loading…” in the middle at the top of your browser window when you use gmail.

    The GWT widget PopupPanel can be used for this. Hence I made my own StatusPopupPanel class which inherits from PopupPanel. I thought I just put a cell of a table in there with the background color I want and center it at the top of the page.

    Not being an expert in CSS the difficulty I had is getting it positioned at the top of the page in the middle.
    I asked how to do this on the GWT mailing list, but did not get the answer I was looking for. Hence I asked a colleague at work how to do it (Thanks Roy) and he came up with the solution for the CSS. The trick is that you have to put the table in a <div> container of which you set the width to 100%. So now CheapRiver has nice status popups. I only use them for errors so I hope you never see them. Here’s a screen shot. (Click to enlarge)

    Screenshot

    And here is the code
    StatusPopupPanel.java

    import com.google.gwt.user.client.ui.Grid;
    import com.google.gwt.user.client.ui.PopupPanel;
    
    public class StatusPopupPanel extends PopupPanel {
     private Grid grid = new Grid(1, 1);
     public StatusPopupPanel() {
     super(false);
     grid.getRowFormatter().setStyleName(0, "statusPopupPanelText");
     grid.setStyleName("statusPopupPanel");
     setWidget(grid);
     setStyleName("statusPopupPanelContainer");
     }
    
     public void setText(String str) {
     grid.setText(0, 0, str);
     }
    }
    

    The CSS for StatusPopupPanel.java

    .statusPopupPanelContainer {
     position: absolute;
     top: 0px;
     width: 100%;
    }
    
    .statusPopupPanel {
     background-color: #ff3535;
     margin: 0 auto;
     padding: 0 5px;
     -moz-border-radius-bottomleft: 5px;
     -webkit-border-bottom-left-radius: 5px;
     -moz-border-radius-bottomright: 5px;
     -webkit-border-bottom-right-radius: 5px;
    }
    
    .statusPopupPanelText {
     font-weight: bold;
     font-color: #333;
    }
    

    Looking for Leftfield

    I bought Leftfield's “Leftism” audio CD a couple of weeks ago, and managed to scratch it to death trying to put it in the tray of a vertical CD drive (and closing the tray with the CD falling out of it).

    Does anyone have some rips of the CD for my legitimately purchased music?

    java-gnome 4.0.14 released!

    This blog post is an extract of the release note from the NEWS file which you can read online … or in the sources from Bazaar.


    java-gnome 4.0.14 (16 Dec 2009)

    You have to compose in order to enchant

    Access to Enchant spell checking API

    Coverage of the Enchant spell checking facade (which was already an implicit dependency arising from our GtkSpell coverage) is now included in java-gnome. It’s a lovely library with a simple to use API which in turn fronts for various back end spelling providers.

    More detailed input handling

    GTK’s handling of complex input methods is extraordinarily powerful, and of course present by default in the Entry and TextView text entry Widgets. If you’re doing your own text based work, however, you might need to capture the results of an input method being used to compose characters or words. InputMethod.Commit is where the result of a compose sequence is captured and delivered to the application.

    We’ve also made numerous improvements down in GDK where events are processed; as a Java library we represent many naked low-level native entities with strongly-typed classes, and have improved our coverage here, notably with new Cursor constants representing the common use cases of changing the pointer.

    Improved text rendering

    Other minor improvements are present across the text rendering stack, notably with the ability to introspect where a Pango Layout has made its line breaks when wrapping via LayoutLine’s getStartIndex() and getLength() methods.

    Guillaume Mazoyer finished up a work by Serkan Kaba resulting in us having coverage of the special LinkButton subclass of Button which can be used to present clickable URLs.

    You can now create custom PaperSize objects, which is handy if you need to use Cairo to output PDF documents with a non-standard paper format.

    Other changes

    You can now use gdk-pixbuf to query an image on disk for its dimensions via Pixbuf getFileInfo() function calls.

    There were of course miscellaneous improvements to various long established core classes, mostly fixing typos.

    We now have the methods necessary to have ImageMenuItems actually show images (there’s a GNOME bug whereby suddenly icons are not showing in menus. So you need to either explicitly tell an ImageMenuItem that it should always show its image, or use the global Settings to say that Menus and Buttons should always have their icons showing).

    The internal initialization sequence has been tweaked to ensure that GLib’s threads are initialized before anything else. This means java-gnome apps will work if glib 2.22.3 is installed; this is a workaround for bug 603774.

    Headless testing

    For a long time we’ve had to be careful in our test suite not to do anything that would cause a Window to appear or otherwise popup while the tests were running. But some for some test cases this is unavoidable, especially if the main loop needs to cycle. We now run the java-gnome test suite within a virtual X server (ie Xvfb), and as a result distros packaging the library can run the test suite on their headless build servers if they wish. There’s a new base class for java-gnome TestCases needing to run in this environment.

    Looking ahead

    What’s ahead for java-gnome? That’s always a good question. At this point java-gnome provides a comprehensive API for development of user interfaces suitable for the GNOME desktop, and it’s incredibly rewarding to see people using the library for their own programs.

    Development of java-gnome has continued pretty steadily, driven by people finding they need additional features from some of the underlying GNOME and FreeDesktop libraries we already expose. As a community we also work to fine-tune the performance and quality of the library through continuous improvement of the code base and its algorithms. There are also people quietly working on experimental coverage of more unusual libraries such as GStreamer and Clutter which is pretty exciting to see.

    Anyone using java-gnome are always welcome to join us in #java-gnome to ask questions or just hang out! So happy hacking, and see you soon.


    You can download java-gnome’s sources from ftp.gnome.org, or easily checkout a branch frommainline:

    $ bzr checkout bzr://research.operationaldynamics.com/bzr/java-gnome/mainline java-gnome
    

    though if you’re going to do that you’re best off following the instructions in the HACKING guidelines.

    Enjoy!

    AfC

    Lovely Inconsolata

    Federico writes that he’s recently discovered Raph Levien’s Inconsolata font and really liking it.

    I likewise discovered Inconsolata not too long ago (it was packaged in Gentoo and I was really pleased to discover it also available in Debian), and have been using it as the constant-width font in Quill and Parchment for program listings in technical writing.

    Parchment output showing Inconsolata font used for program code

    We all tend to get obsessive about things we like, and so of course I tried it as a terminal console font. Interestingly, after a brief consideration, I decided not to use it for terminals and gedit and such. Deja Vu Sans Mono is still the king for that, especially if you’re using Deja Vu Sans and Deja Vu Serif for the rest of your UI; it means that the visual consistency across your desktop is really awesome.

    gnome-apperance-properties Fonts tab, picking Deja Vu fonts

    Inconsolata doesn’t have anything even remotely close to the kind of Unicode coverage you need in your full-time constant-width font, and when fontconfig does a fallback it looks awful because Inconsolata’s metrics are so different from others. That’s all ok; Inconsolata is meant for program listings, and looks incredible on paper.

    Anyway, Federico noted that bold didn’t work when specified as a FontDescription, it doesn’t come up bold me either when I tried to do so via a bold PangoAttribute. But if you use Wouter’s Specimen, it shows up fine (on an Ubuntu box, anyway):

    gnome-specimen showing Inconsolata font medium and bold

    So there’s a bug in our stack somewhere.

    AfC

    Update

    Or not. I just reread the JavaDoc for the FontDescription constructor, and lo and behold relearned what I wrote there in the first place. To get bold Inconsolata you just need to use a ',' in the right place:

        desc = new FontDescription("Inconsolata, Bold 8.0");
        ...
    

    and my Cairo drawing code on screen in a XlibSurface shows bold. Great.

    But nothing is ever simple. I ran the same drawing code out through to a PdfSurface, and no bold. What the hell? There’s a bug in our stack somewhere. {sigh}

    In my original post, I misattributed Specimen… it’s written by our very excellent Wouter Bolsterlee, uws on IRC. Sorry!

    December 18, 2009

    Why (and how) I left Gmail

    After reading Luis Villa’s concerns about how little Google values our privacy, I decided leaving Google services as well. In fact, I already left Gmail 1.5 year ago, not (only) because of privacy concerns, but because I was simply not satisfied with its IMAP support.

    Call me old fashioned if you will, but I want an offline cache of my email. IMAP is of course the best protocol for that, but Google doesn’t support IMAP very well. They use folders as an offline implementation of their tagging feature, instead of actually using IMAP tags. This means downloading messages multiple times, which is not funny at all. And then, there is Philip Van Hoof saying [Google's] IMAP server is probably one of the poorest imaginable.

    I don’t feel like running my own email server, either on my computer or in the cloud, so I went with FastMail. Their email solution is not free of cost, unless you keep very little email, but I find their price and service fair enough. I think that the only catch is, I didn’t go for the most expensive plan, so I can’t keep arbitrarily large attachments from many years ago. That’s OK for me. I download the important attachments to my hard drive and use a proper backup instead; and I can still keep the respective email messages.

    I still have my Gmail account, which redirects to the FastMail one. I just don’t give my Gmail address to anyone these days, and I don’t use their SMTP server any more. I miss the conversation view, but I found out I can live without it. I reimplemented Gmail’s archive with an “Archive” folder inside Inbox, but latter it grew too big and I split it in one folder for each year.

    I gave up most of my tags during the migration, because I didn’t really use them to search the email archive. For the remaining tags, I used the extended tagging feature of IMAP. I have less then 5 tags, so I could use the standard ones, but Evolution and Thunderbird have the same default names for these 5 tags, so I decided to just not change them. FastMail doesn’t really provide a way for me to set or see tags in their web interface, but that’s OK for me, because I use primarily Evolution to check and manage my email. The important thing is that FastMail doesn’t brake my tags. Sadly, OfflineIMAP doens’t implement these arbitrary tags yet, so I can’t use it to relay my email.

    I’d love to know about other email solutions. I’m not really considering a move any time soon, but the discussion could be very interesting for current Gmail users.

    “skid mark”

    A new phrase that I learned yesterday. Probably something that’s been around computers for a long time, but it was my first exposure to it.

    Basically a “skid mark” is something you leave in code as a signature so that in a crash dump you can figure out how you got there.

    We’ve been doing a huge amount of work in Firefox 3.5 and 3.6 to reduce the number of crashes. For some crashes we have stack traces, but not a lot of data on how we got to that stack. So we’re actually checking in code changes and and shipping them into the wild to see how they change crash signatures.

    The interesting thing is that it’s working – we’ve been able to track down some pretty serious crashes with this technique.

    Early Christmas!

    My employer, Collabora was very generous and bought N900s for the entire company!

    Mine just arrived:
    N900: a Christmas present

    I was lucky enough to pick one up at the Maemo Summit in October as well. However, those have to be sent back to Nokia in a few months, so now I won't have to dread the deadline :)

    In related news, I'm (slowly) working on a handy Maemo 5 app of my own. More details as soon as it's usable (and uploaded to Maemo Extras [Devel]).

    Fri 2009/Dec/18

    • Dear lazyweb,

      I just started using Emacs 23 in openSUSE 11.2. Running this:

      emacs -fn "Inconsolata Bold 12"

      actually gives me Inconsolata, but definitely not Bold. Why? How do I make it Bold?

      (Inconsolata is a truly beautiful font for programmers, by our GNOME Emeritus Hacker Raph Levien.)

    Ceibal '09

    In the days 7th and 8th of this month took place in Montevideo the First International Event on Experiences about the Ceibal Plan, with the objective of sharing the experience gained during the first years of the implementation in Uruguay of the OLPC initiative, with the name of Plan Ceibal. On the afternoon of the second day, some schools displayed the work they had done with the XOs.

    Project Rayuela and LATU were kind enough to invite me to assist, which has been an unforgettable experience for me. The plan was that after the event I would stay for one week discussing with the software team at LATU how they can work most profitably with the Sugar community upstream, will talk about this in a later post.

    I still can hardly believe that it has been the second time that I have been close to a child learning with Sugar, and the previous one was more than 2 years ago, when attended FISL in the company of J5 and Jim Gettys. We visited then a school where the Federal University of Rio Grande do Sul was conducting a pilot with very early prototypes of the XO and Sugar.

    Visit to the school Luciana de Abreu

    This time was very different because we were seeing the results of at least one year of continuous work with Sugar, something that was happening in the mainstream of that society, backed by the government, the families, several volunteer organizations, and the primary education system.

    Students displaying their work with Scratch

    Though I was ecstatic seeing each school presenting their work, I felt a bit sorry for the developers and non-developers that made that possible with their work but received very little feedback to date on the actual use of it. I have plans to improve that.

    Learning geometry with Turtle Art

    It has been a very enlightening and encouraging experience, wish all my colleagues at Sugar Labs could have this opportunity as well in the near future. The last 2 years have been pretty hard for the Sugar developers, always having to stand the never ending stream of naysayers, even from OLPC itself, and with the last year having to live off our savings. I'm very proud that we have managed to stand firm on our believe that the best GUI for learning hadn't been invented yet and that we could contribute something valuable to the mission of giving more learning opportunities to children around the world, specially those from less favoured backgrounds.

    After having seen children and teachers involved together in learning with Sugar, I can finally say that we were right. Though of course, this is just starting...

    The program of the event can be found here and I expect that soon will appear video recodings of the sessions and of the school presentations. Will post here when they are made public. Also will post soon about the week that followed, in which I had the luck of being a small part of a process in which an organization (and in some sense, a country) moves from being a consumer of FOSS to being a full rights member of the FOSS community.

    Thanks to J5 and Gonzalo Odiard for the photographs.

    December 17, 2009

    Mono Developer Room at FOSDEM 2010 CFP: Deadline near!

    Are you a developer hacking on or using Mono? Coming to FOSDEM 2010? Be sure to submit your talk for the developer room and do it quickly: the deadline is nearing!

    More info.

    Europe, here I come!

    I wrote my last exam on Tue night and We afternoon headed to the airport to get to Spain for the WebKitGtk hackfest at the Igalia offices. At this time, stuck in Frankfurt airport after missing my connection.

    After the hackfest I'm doing a mini tour of Western Europe, thanks to RailEurope. Mostly visiting friends and family. Currently looking like: Frankfurt -> Kassel -> Paris -> Brussels -> Amsterdam -> Hanover -> Berlin -> (Maybe) Zurich.

    Looking forward to meet GNOME guys. And will blog about hackfest progression. Before you ask, my mandate for the event is to port WebKit to use the new HarfBuzz API.

    Banshee Grid View

    Spread out over a number of hours over the last few weeks I’ve been working on making our high-performance list view in Banshee also handle grid-style layouts.

    That is, all the benefits of our polished list widget with a new look and feel! I’m making the widget more clever and abstracted about layout and rendering so it can be more inviting to users. In fact, you can even live-toggle between layout modes (e.g list and grid) without losing any state.

    This effort provides a more visually immersing and space-conscious view for your collection of albums. This big step forward is the groundwork for presenting other media collections in a friendlier way — video thumbnails, movie box covers, audio books, photos…

    Screenshot of Banshee's new grid layout for albums
    The new grid layout mode for the album browser.

    There are still a few quirks to work out regarding interaction (different maths for keyboard navigation), and some more polish to add on the rendering side of things, but I am very close to merging this work to master in git.

    The code is being developed in the grid branch in git for those who would like to try it out. Poke me on IRC if you find problems, but I probably am already well aware — again, it’s not merged to master because it’s not RC worthy yet — but it’s ready for some broader testing!

    Thoughts? Suggestions? Complaints? Hopes and dreams? I’m super excited about finally having this land, and it’s been an enjoyable hack to implement.

    Unpackaged Open Font of the Week: Blackout


    Blackout is a decorative typeface that uses filled-in sans-serif lettering with a little bit of a grungy feel. Tyler Finck developed the font for his homepage / portfolio which is also a great example treatment of the typeface. It’s a bold and attention-grabbing typeface but it also has some attitude, so you could use it on posters to get people’s attention from far away, or you could also use it in limited doses around any kind of composition for a little bit of spice. :)

    Blackout and is licensed under the Open Font License.

    So, you want to package Blackout?

    Schweeeeet! You’re rad! You’ll want to follow the first steps here next to the ‘if you intend to do some packaging’ header:

    Our fonts packaging policy, which the above refers to, is documented here:

    And if you have any questions throughout the process, don’t hesitate to ask on the Fedora Fonts SIG mailing list:

    Last Week’s Font

    Last week’s font was Selfism by Jan Sonntag. Nobody has picked up the font package request yet! Would you like to?

    And on that note:

    Now people. Let’s be frank here. This is week #11 of the Unpackaged Font of the Week blog posts. It’s been over a month since these fonts I’ve been highlighting to you have gotten some packaging love. So talk to me:

    • Am I posting these too often? Should I go bi-weekly? Monthly?
    • Do you want to help and don’t know how to package? Or is the font-packaging process too complicated?

    • Do you just not care about fonts and would rather see packaging needs highlighted for some other type of thing? (I recently discovered there’s a lot of great audio apps that we don’t have packaged yet!)

    Please let me know, because I need to decide if I should keep doing this. My goal here is twofold: I want folks to see what awesome openly-licensed stuff is out there to use, but I also want to show folks very specific, well-defined, and not too-overwhelming ways they can jump in and help further free & open source projects – packaging content for use in a distro is a great way to do that.

    When you package fonts and help me out, I see the love, but when I don’t see stuff getting packaged, I feel sad!

    One idea I had was maybe to do fonts on week 1, do a wishlisted audio package on week 2 (omg do I want pure data packaged), do maybe something interesting off of the general package wishlist on week 3, and wishlist games on week 4, then rotate. That’s just one idea I had though. I want to hear what you think I should do!

    Posted in Unpackaged Font of the Week

    December 16, 2009

    udisks and gnome-disk-utility - past, present and future

    The past year or so I’ve been working a lot on gnome-disk-utility and DeviceKit-disks^Wudisks and I haven’t really blogged anything about it. Time to fix that!

    First of all, given that the main DeviceKit daemon was killed by libudev (if you are using GLib you want GUdev instead!), we (that is, myself and other contributors) decided to rename the project - most people were cool with that; see this message for more reasons. One improvement in this area is that we’re committing to limited ABI stability - hopefully this will make things easier on distributors.

    Most of the work in the udisks project been triggered by features we want in the Palimpsest Disk Utility - which is the most prominent part of gnome-disk-utility - the other part includes various desktop shell integration bits that I don’t have room to cover in this post.

    On the Disk Utility side, we’re now using an user interface that scales better with many disks - in the old user interface, we used to show all partitions as children of each disk in the tree view. In the new user interface, we’re using a grid to convey the partitioning:


    Also see bgo #579211

    While this interface isn’t optimal for the laptop or the machine-with-only-a-single-disk cases, it works a lot better for machines with many disks


    Bunch of disks

    The Disk Utility also have somewhat complete support for MD RAID (aka Software RAID on Linux) - we’ll show you running arrays and also arrays not yet running. There’s a way to add/remove components and currently I’m working on creating the UI for Online Capacity Expansion (e.g. dynamically expand a RAID-5 array from 4 to 8 drives).

    The SMART stuff has been revamped and there is now a way to turn off the desktop shell notifications. As it turns out a lot of our current users are like “I’m fine with using a disk that is about to fail” - kinda shocking but there you go. Another change is that we don’t consider a disk to be failing if there’s only a couple of bad sectors - the disk will have to have many bad sectors to be considered failing. This helps with all the false positives the SMART warnings generated in e.g. Fedora 11. Anyway, it seems like the desktop notifications are useful - more than a handful of people have pinged me on IRC thanking me and Lennart for this feature. Woot!

    We also now have rudimentary support for showing the SAS topology - this is really useful since machines with many disks typically use dedicated disk enclosure shelves. Currently we only show SAS Expanders but the plan is to peek (and poke!) SES-2 Enclosures (and in the future also SGPIO ones) to get information about bay numbering and to toggle the FAILURE and LOCATE leds. Ideally we want to turn on the FAILURE led if a member of a RAID array is kicked or if SMART status indicates that the disk is about to fail. Ideally, we also want to do (sensible!) things like starting a RAID rebuild if a kicked RAID member is replaced by a uninitialized and same-size disk. As always, we have to be really careful about automatically doing policy things like this - especially on Linux where it is hard to make any assumptions on how the system is used.

    The goal of the udisks effort is to be useful to users. For example, if you do video editing or a lot of photography, you care about IO performance. So I added a way to easily benchmark drives and RAID arrays:


    RAID-6 FTW!

    This feature was inspired by various similar Windows programs (a pastime on big storage forums such as this one involves trading benchmark screenshots) and the zcav program. It’s pretty cute, actually; I learned that my Intel 80GB G1 SSD only gives me 140MB/s on my Lenovo Thinkpad X61 while I get a full 270MB/s on my workstation (connected via SAS). Looks like the X61 is only using SATA 150 MB/s - a search on the tubes confirmed this. Gee! This feature is also useful when experimenting with RAID setups - for example, a 10K 36.7GB SAS disk benchmarks like this but when you put three of them together in Software RAID-5 it turns that writing is really slow. I know it’s supposed to be slow when writing to software RAID… but, really, this slow? Maybe I need to tweak some kernel tunables (which would be bad - we should be fast out of the box etc etc).

    Finally, I did a really cute hack a few weeks ago - I made Palimpsest use D-Bus over TCP/IP over SSH to speak to udisks:


    Bunch of servers

    With the way this works we leverage SSH for authentication and Avahi for Service Discovery. The feature is still a bit rough - it needs some UI additions (such as a disconnect button) and we probably want to replace the stock OpenSSH dialog with something, uhm, less scary. There’s also the question of authorization - right now you have to connect as root in order to modify things (udisks uses polkit for authorization) - we need to figure out how we want this to work. Right now I’m thinking udisks will ship some polkit configuration bits (a .pkla file) and a udisks-admin group that administrators can add users to. We’ll see.

    I’m kinda excited about the remote connectivity bits - it means that the Disk Utility is now very useful for headless server setups and said servers won’t have to run any graphical UI in order to be managed through the Disk Utility. We probably want to make the Disk Utility (e.g. Palimpsest) run on Win32 and OS X too (for users not running Linux on the desktop) for remote connectivity to e.g. Linux boxes running udisks. It shouldn’t be too hard to do this as we mainly depend on GTK+ and D-Bus.

    There’s a bunch of plans for both udisks and Palimpsest. We want to add support (meaning: udisks API and Palimpsest User Interface bits) for the following things: LVM, iSCSI (both target and initiator), Enclosure Management (as mentioned above), Event logging (e.g. “raid5:md2: read error corrected”), Multi-path, Btrfs and many other useful storage technologies. There’s also a few rough edges and missing features (e.g. file system resizing) but we’ll get to that in due time.

    FWIW, it’s important to realize that udisks and the Disk Utility complements, but does not replace, existing command-line tools. This means that you can go ahead and keep using your command line tools and scripts and the udisks/GNOME/Palimpsest/etc. stack will react to changes. For example, you can start/stop RAID arrays using mdadm(8), partition your disks using parted(8), create filesystems using mkfs(8), mount/umount filesystems and so on - the UI is kept up to date. While it’s approximately ten times harder (with the 10x figure being 65% accurate) to write software this way, it’s really the only way to do things on something like Linux where our set of users is as diverse as it is.

    Modest with BODYSTRUCTURE support

    These last weeks Dape and me have been working really hard fixing bugs in Modest and Tinymail here and there. Best Modest ever is coming.

    But today, I don’t want to talk about fixes but features. I want to talk about BODYSTRUCTURE. This is one of the coolest features we could have added to Modest. Tinymail had some initial support, but due to the many bugs it had and the fact that some use cases were not supported forced us not to use it so far. But thanks to the time Igalia gives us for hacking we managed to get it working.

    Oh wait! I didn’t tell you what BODYSTRUCTURE is about. Email messages are made of a group of MIME parts. One of them could be the subject, another one some footer and some others could be attachments. Without BODYSTRUCTURE support we were forced to download all those MIME parts when you wanted to see a message. This meant that if the message had some heavy attachments and you only wanted to see a small body with just a couple of words, you had to wait until the full message was downloaded,

    With this new feature, we can download every MIME part one by one, and thus saving you time, disk space and specially if you’re using a mobile device like N900, money in your GPRS connections. Do you want to read only the body? No problem we’ll show you that you have some attachments but we won’t download them until you request us to do so. Do you want to forward the full message? No problem, we properly detect that and include the full original message whether or not it was completely downloaded before.

    This will most likely be included in the next N900 software update that will be eventually delivered by Nokia. In the meantime, if you don’t want to wait just download packages and build it by yourself. Remember that you can find us in #modest channel @ Freenode.

    Having a tidy systray

    For too long on the Free Software desktop there has been a systray that is simply out of control. It provides for an arbitrary window with arbitrary behavior that developers have used arbitrarily. This provides a difficult interface for users as they don't know what to expect. It provides a difficult situation for applications as there is no way that they can give the user a good experience as there are no standards to follow. And it makes the whole support system for software more difficult through documentation and long explanations of how to use the computer.

    Enough bitching. We're going to fix it.

    We're starting with the KDE Status Notifier spec. It has been proposed to XDG and is currently used inside of KDE. It provides a DBus based interface for applications to advertise systray style information over DBus thus eliminating the need for XEmbed. The way that the Notifier Item spec is written it's designed to make the application expose information independent of the visualization, and we'll be one visualization of this data. A restricted visualization specifically focused on panels. We're going to have each application represented by a menu.

    So with every application being a menu, and the panel knowing this and being able to assume it, then the whole systray can be a menu bar. This makes it easy to browse application's menus, find what you want, and work with it. It also improves keyboard navigation and increases the accessibility. Menus are a nice mix between simplicity and expandability as they're easy to use and yet provide a way for applications to provide a rich set of functionality.

    <video autoplay="true" loop="true" src="http://gould.cx/ted/blog.pix/AppIndicatorsDemo.ogv">
    I'm sorry, your browser doesn't seem to support embedded video. The video is available to download.
    </video>

    To do all of this, and make it easy for application developers to implement, we've built a small convenience library. That library is libappindicator. Basically it takes an icon and a menu and magically sends it over DBus and puts it in the panel for you. No fuss. It will also handle the cases where you're on KDE and want to use the older KDE Status Notifier API. Or, even the old XEmbed way of doing things. We've tried to make it pretty easy for application developers:

    /* Indicator */
    indicator = app_indicator_new ("example-simple-client", /* ID   */
                                   "simple-client-icon",    /* Icon */
                                   APP_INDICATOR_CATEGORY_APPLICATION_STATUS);
    
    indicator_menu = gtk_ui_manager_get_widget (uim, "/ui/IndicatorPopup");
    
    app_indicator_set_status (indicator, APP_INDICATOR_STATUS_ACTIVE);
    app_indicator_set_attention_icon (indicator, "simple-client-att-icon");
    app_indicator_set_menu (indicator, GTK_MENU (indicator_menu));
    

    Some of this stuff is, unsurprisingly, not yet complete. The menu support basically only works with labels today, and the fallbacks aren't yet in place for the legacy protocols. I promise I'm working on it. We'll be getting better menu support here probably this week and then I'll start working on the legacy support. I believe that there's enough here to get started, and to start understanding what we're thinking.

    This post is getting long, let me just finish up with some links:

    • Indicator Application Launchpad page -- this is where all the code and bugs are. Feel free to branch and do what ever you want with it.
    • Documentation Wiki -- To understand what we're doing and how to port applications we've tried to put up some documentation. We'll continue to add as we get language bindings up and running here shortly.
    • PPA for Karmic -- We've backported the modules required so you can develop and play with what we're doing today. There's also a preliminary patch to Rhythmbox that you can look at, which will, of course, be submitted upstream when it's cleaned up.
    • Ubuntu Developer Week -- I'll be talking about this there, if you want to come and ask questions or just lurk.

    USDT probes in Gnome – a prototype

    This sounds like it could be the start of some kick-ass GNOME performance work. Dtrace ftw :)

    Advances of freedom

    I’ve been hacking at gnome-color-manager for a few weeks now. We had a first release last week, and we’ve since been adding in more new features and a couple of nice bugfixes. But that’s not what this blog post is all about.

    A typical target

    A typical target

    To calibrate a device, you need to scan in (or take a photograph of) a very accurately printed image. These printed images are called “targets” and usually come with a CD-ROM of the calibration data for that batch, or a URL of where to get the calibration data from.

    Now, the number of people wanting to use a calibration target is going to increase in the future, as they’ll want to have a color-calibrated workflow whilst using Linux. These random CD-ROMs that get lost and URLs that might vanish don’t seem so easy to use when you’re calibrating 200 workstations, maybe using a different type of target for scanner, camera and film. And re-calibrating them all two years later doesn’t look like fun either.

    So, ideally, we would ask the manufacturers of the calibration data (just big text files of numbers), and we could ship it in a shared package, that the distros can ship. Unfortunately, a lot of the targets in existence have NDA or horrible licensing terms shipped with them. This makes distros like Fedora that can only ship free software and content sad.

    Enter Wolf Faust. He’s the bloke that’s been shipping high quality IT8 targets all over the world for the last few years. He’ll ship you just a single target (there is no minimum order), so it’s enthusiast friendly. The pricing is cheap (25 Euro) and postage and packaging is reasonable (5 Euro), which makes him the obvious choice for someone that just needs a target or two to calibrate their photographic or graphics workflow.

    Wolf Faust has just released his entire set of calibration data under a free license. This means we can ship it in a distro package, so that calibrating a scanner is as simple as borrowing a target from a friend and taking a photo of it and then selecting a target name from a GUI drop-down. No need to fumble about with CDROMs or downloading the correct target from a website, now it just works.

    Now, I guess Wolf has realized by making the calibration data “free” content, he’ll sell more targets; and I hope he does. If you make it easier for people to use your product more people will buy it for sure. It makes no sense keeping this data secret and wrapped up in legalese. It might not be much, but this for me is an advance of freedom, as much as just-another-package in a repository.

    Note, gnome-color-manager will install shared-color-targets automatically using PackageKit if you try to calibrate a device and it’s not already installed. We’ll do the first official release of shared-color-targets just after Christmas.

    Mono 2.6 + MonoDevelop 2.2 on openSUSE 11.2

    Fantastic news — Mono 2.6 and MonoDevelop 2.2 are out! Be sure to read the detailed Mono 2.6 release notes.

    If you are running openSUSE 11.2, it’s quite simple to safely update your entire Mono stack and stay up to date with any 2.6 updates:

    • zypper ar -f -n Mono http://ftp.novell.com/pub/mono/download-stable/openSUSE_11.2 repo-mono
    • zypper mr -p 10 repo-mono
    • zypper refresh
    • zypper dup -r repo-mono

    Those instructions will change the Mono distribution from openSUSE to the upstream Mono repositories.

    I highly recommend installing MonoDevelop 2.2 (zypper in monodevelop), though if you already had it installed, the above will update you to the 2.2 release.

    MonoDevelop 2.2 Integrated Debugger
    Integrated Mono 2.6 soft debugger in MonoDevelop 2.2

    A huge congratulations and thank you to the Mono team for another exceptional major Mono milestone!

    UPDATE: apparently there is a problem on the x86_64 build in the openSUSE Build Service. This appears to be an OBS bug, but for this reason, it’s recommended that you use the repositories at http://ftp.novell.com/pub/mono/download-stable.

    Public vs. Private Forums

    Many people feel that in the open source/free software community openness is about letting everyone become privy to any comment/decision/situation that may happen within a project. For the most part this is the goal we strive for but at the end of the day there are things that should remain private until they become real concerns for the public at large.

    The issue is we have diverse communities that rarely agree 100% even among the best of friends. It becomes counter productive when these disagreements become the source of misinformed news articles without general consensus allowed to form within the community itself.

    Forget the community for a minute. As individuals we often need to think before we speak in order to make sure we say exactly what we meant to say. The same goes for diverse groups. They must work out the directions they go in in an atmosphere that is free from the chilling effect of having to watch what they say because their intentions may be misinterpreted.

    There is also the point of being able to freely express an opinion without fear of outward reprisals. There is a reason Democracy subsists on the concept of private ballots. It allows even the most unpopular opinions to at least be registered while individuals can be sure they won’t be ostracized. The openness comes from being free to dispute results, protest and influence them through public action, not from knowing the individual’s specific vote.

    We strive to be inclusive with everyone who has a stake in what we do but when it becomes counter productive one needs to redress the situation so that we continue to move forward. It does no one any good to dwell on speculation from quarters which are ill informed and just looking for traffic grabbing sensationalism. It is perfectly fine to exclude them from discussions they have no stake in and let them feed on the results, judging the community based on its final actions and consensus.

    [read this post in: ar de es fr it ja ko pt ru zh-CN ]

    Freezing Totem with text subtitles?

    Then ask your distribution to backport the patches.

    This has been fixed in Fedora about a month ago.

    Save MySQL: Getting Some Facts Straight

    I’d like to clear up some misconceptions that have appeared in comments on various blogs, social bookmarking, and other fora in response to Monty’s recent call for action. Many of these have been repeated ad infinitum and it’s time to set the record straight.

    1). Monty sold MySQL to Sun.

    Monty founded MySQL Ab along with David Axmark. True. He and David then decided they’d like to pay their bills, and so to generate revenue, they decided to seek outside investors. As neither of them is highly experienced with such matters, they hired Mårten Mickos, who became CEO of MySQL Ab. At that point, Monty’s role in day-to-day business decision making dropped to almost zero. If any one person “sold MySQL to Sun,” it was Mårten.

    This is not to say Mårten acted against Monty’s express wishes. Monty had great hopes for Sun’s stewardship of MySQL, and only became disillusioned after some time actually working for Sun. Mårten did the job Monty and David hired him to do, and in no way should my sentiments be construed as “blaming” Mårten. But saying it was Monty’s decision to sell to Sun is patently wrong. It was Mårten’s decision in concert with the outside investors he had courted.

    2). Monty should live with the consequences of the sale.

    Let’s use an analogy here.

    Let’s say that Red Hat’s investors and stockholders approve a deal to sell Red Hat to Google. (And just for fun, let’s assume one of the results is rebranding Red Hat Enterprise Linux as Red Goo. It’s my analogy.)

    I think Mark Ewing and Bob Young would probably be satisfied with that decision, especially if it paid them well. This was the case with Monty and the sale of MySQL Ab to Sun.

    Now let’s say that after a few years, Google is acquired by Microsoft.

    Would Mark and Bob be pleased about the possible prospects for Red Goo? Would you tell them, “This is what happens when you sell a company?” Or would you, quite rightly, start being concerned about the future of one of the most important Linux distributions? And would you feel that Mark and Bob had reason to voice their concerns and try to do the best they could to serve their community of users, developers, and customers?

    Even if you thought Microsoft had the best intentions for Red Goo, it doesn’t mean Mark and Bob have to agree with you. Even if you believe that Oracle has the best intentions for MySQL, that doesn’t mean Monty has to agree with you.

    3). Monty got PAID! He’s rich, man! He sold MySQL Ab for a BILLION!

    Monty has more money than me. He probably has more money than you. But remember, MySQL had investors. MySQL Ab had Mårten. To think Monty received US$1Bn is foolish. To think he received even a tenth of that is a mistake.

    Yes, Monty got a payday. But it wasn’t what a lot of people think it was.

    4. Monty sold his business, and now he just wants to get it all back for free!

    Part one? True. Part two? Not so much.

    Monty’s business was sold. He has now started a new business (Monty Program) and employs a lot of the former MySQL coders, as well as strange Free Software wonks (that’s me, for those of you keeping score at home). We all work on MariaDB, a fork of MySQL that does not have a pay-for “Enterprise Edition” and that actively seeks great contributions from the wider open source community.

    Let’s say that Oracle buys Sun, and completely trashes MySQL. Who benefits?

    Monty Program.

    MariaDB will immediately become the de facto replacement in most Linux distros and in the minds of most people.

    Let’s say Oracle has to sell the MySQL unit as part of the remedy for the EC’s concerns. Monty can’t buy it. As I said, he’s got money, but not that kind of money. And who would get hurt if Oracle sells it or does a great job stewarding the project in the future?

    Monty Program.

    Now, I like getting a paycheck. I want Monty Program to succeed so that my wife and I aren’t eating out of cat food tins at the bus station. But I also like principled people that care, really care, about the projects they love. And that’s why Monty is potentially hurting our business prospects by raising his concerns with the European Commission. He cares about the code he has worked on for 27 years. He cares about the companies that have built a business on that code. And he cares about the users of that code; be they Fortune 50 companies or your teenage brother running WordPress.

    Part of me wants to slap Monty. I want our company to become the standard. Now.

    Part of me wants to kiss Monty. He’s the kind of dedicated and thoughtful steward any project deserves.

    Paul McCullagh discussed this thoughtfully on his blog this week.

    5. Anyone can fork. You did. What’s the big deal?

    Two words. “Embedded solutions.”

    A lot of companies have embedded MySQL into products they sell. They’re not interested in releasing their entire software stack under the GPL. They buy licenses from Sun (and formerly from MySQL Ab) to allow them to embed MySQL into their products and not have to release all their source.

    MariaDB, or any other fork, will never be able to offer this. Ever. We forked the GPL version, and so will anyone else. Your code depends on our code and you’re selling it to any interested party? You have to GPL every bit of code you sell.

    I said it in my last post, and I’ll say it again. Forking saves the code. It does not save the business.

    And as I also said before, and will reiterate, if Oracle is so certain that forks are the answer, why didn’t they just do that months ago and save themselves the headache? They could have forked MySQL, taken MySQL off the table, and avoided this whole EC inquiry. They didn’t. Think about that.

    6). Just use Postgres.

    Sure. Go tell Google that. Or Amazon. Or eBay. Or Craigslist. Or any ISP depending on cPanel to ease customer usage of internals.

    Postgres is a great project. It delivers a a fantastic database server. I have a lot of people in the Postgres community I consider friends. So does Monty. And you know what, those people would be among the first to tell you that migrating to or from one database to another is not an easy task for large deployments with thousands of lines of customized code. They’d also tell you that Postgres is superior to MySQL for some things, and MySQL is superior to Postgres for others.

    “Just switch to Postgres,” is not a solution for many people, primarily businesses that have built their products on top of MySQL.

    Now, can we move beyond these issues to the larger issue at hand? If you think Oracle’s proposed acquisition of MySQL is probably going to turn out badly for the Free Software and open source communities, read Monty’s blog post and act. If you think it’s a tempest in a teacup, I respect your right to your opinion. Please respect Monty’s right, and if you wish to engage people on the subject, keep the preceding points in mind.

    December 15, 2009

    google is your butler- the tension between utility and privacy

    I’ve often defended Google’s thirst to know things about people with a butler analogy. Good software should, like a butler, try hard to understand your preferences and act on them for you without you even realizing they are there. That means learning and remembering things you’ve done in the past, and using that to base recommendations on. When you tell your butler ‘bring me desert, please’, he should remember that you usually like chocolate, and that all this week you’ve been experimenting with different cakes, and therefore bring you another variant on chocolate cake. If he suddenly forgot you liked chocolate and you’ve been having cake all week, you’d be irritated when he asked you those things again, or if he just brought you a canoli out of the blue.

    Ideally you want your butler to know at least something about what your friends and co-workers are doing too- if I say ‘bring me a shirt’, and the butler knows I’m going out with the cool kids tonight, then I want my trendiest shirt based on what my friends think is trendy. But if I am going to the office and say ‘bring me a shirt’, I want the butler to know that my workplace is casual, but not too casual, and so on. I could of course tell him all these things every time he brought me a shirt, but it is easier for everyone if he just remembers, and perhaps does some outside research on his own.

    Like a butler, you want your tools to work intelligently based on context and history, and Google is without doubt one of those tools- for many of us, the most important single tool in our computing lives. The problem, of course, is that your butler has a lot of incentives to keep your private information private. Surely the butler can be bribed, but therefore you pay him well and treat him like a human being, and you try to avoid these sorts of problems. Google’s incentives run at least partially the other way- they have strong incentives to mine that data extensively, to share it with others, and to collect well more than most people might think is useful, in the name of being the ultimate butler. And these incentives lead to risks- incentives to share with third parties that you might not trust; risks that things might be subpoenaed; risks that they might leak to Google employees or even outside Google; risks that effective advertising might use such information to manipulate your political views. On balance, most of us are going to look at these issues and decide that we’re OK with Google knowing these things, because the risks are remote and the benefits tangible. So we acknowledge there is a tension between privacy and functionality, and move on.

    I wish that at this point I could announce some deep new insight about the balance between these two competing forces. I can’t; most of what there is to be said has been said already. The thing that makes me write about it right now is, of course, Eric Schmidt’s recent comment. The thing that bugs me about it is that he doesn’t seem to realize there is a tension. These words don’t speak of ‘we’re wrestling hard with this question every day’ (a reasonable compromise position) or ‘we’re doing everything we can to collect as little data as possible’ (the pragmatic civil libertarian perspective). They speak of a company (or at least a CEO) which doesn’t realize or doesn’t care that there are balances and compromises to be struck and continuously re-considered. And that, to me, is very, very troubling; more troubling than any particular policy position could be.

    So I’m experimenting this week with other search engines, and once I finish moving I’ll be looking again at other mail and rss readers. I really don’t ask much of Google in return for trusting them; I’m not an absolutist, I just need to know that they are continuing to treat privacy as a difficult, multi-faceted issue that constantly has to be evaluated and considered. And if Schmidt is any indication, that isn’t what Google is doing right now.

    Feeds