Subscribe to future posts here 
Sailfish OS update from Jolla
It’s been another busy Sailfish OS fortnight, the main news being the release of Sailfish OS Verla 4.2.0. The release had a longer-than-expected gestation period, due to a blocker bug surfacing during the Early Access period. But it got there, and hopefully has already made its way on to your phone. We already covered the Early Access release in some detail last time, and if you want to know more I recommend Pekka’s excellent Verla blog post. It’s also been wonderful to see all of the content go up from the community on the release. More on that later.
Every release marks a milestone, but next month we’re also celebrating a different kind of milestone. For it will in fact be ten years since Jolla started as a company. And what better way to celebrate the fact than with a birthday bash to which you’re all invited! Right from the start the community has of course been an integral part of the Sailfish OS project, and so we really hope as many of you as possible will be able to join us. The party will take place in Berlin on 14th October, exactly ten years to the day since the founding of the company. As well as celebrating this important milestone it will also be an opportunity for us to meet friends, to put faces to names, to share some of the latest developments with you, and to discuss all things Sailfish.
Places are necessarily limited, so if you’d like to join us please register for the event as early as possible to avoid dissapointment. If you can’t make it to Berlin in person, then we also hope to enable online participation, and there will also be a further celebration to take place in Finland at a later date, with details to be announced. We’re really looking forward to celebrating with you.
The other — maybe not-so-big — news is that this community newsletter now has its own category on the forum. If you have a topic you’d like to see covered, then please add it in the comments below, we’re always looking for ideas, as long as they’re likely to be of interest to the Sailfish OS community. And if you’d like to contribute a complete article yourself, or to work with us to produce an article, then that’s also something we can make happen. Send a direct message to @flypig on the forum in this case so we can discuss it further.
And this time we have a great example from community member Damien Caliste. You may recall that we covered Damien’s work improving the Sailfish OS PIM applications back in August. It’s worth reiterating that Damien’s considerable contributions to Sailfish OS are all done in his spare time, and purely for the benefit of users.
Back then we noted that Damien planned to write a special article about Buteo, and I’m really pleased to say that we can share it with you here today. Buteo is the beating heart of synchronisation on your phone, the conductor of the PIM orchestra, so it’s importance can’t be overstated.
The Quiet Power of Buteo
Let’s talk about the synchronisation framework in Sailfish OS.
What is “a synchronisation framework” actually? It’s a set of libraries and code that run on your device and that is responsible to keep some device content up-to-date (in other words: in sync) with the same content stored elsewhere. You may want your Firefox bookmarks shared between your devices, your calendar data appearing also on your desktop computer or your last vacation pictures uploaded to your home NAS automatically.
All these actions require some code responsible for synchronisation to be run at one moment or another. This code can be part of the application (Firefox handles its bookmark sharing on its own), or delegated to a dedicated system background process. Buteo is of the latter kind.
Having a centralised synchronisation framework is a huge advantage in terms of platform consistency: all emails, calendar data… are using the same interface to schedule synchronisation and they are proposing the same capabilities (you can ask not to fetch emails during the night because it’s implemented in Buteo while you cannot do it for WhatsApp discussions because it is a per application synchronisation). But it’s also beneficial in terms of code maintainability: the contact application is only responsible for displaying contacts and interacting with the user, the storage backend is responsible for keeping the contact details on the device and the synchronisation framework is responsible for the coherence of the data between server and device.
Buteo is the result of internal development by Nokia for Harmattan (the system powering the N9 and N950). It was released as open source as part of the Meego initiative around mid-2010, see this mail archive. It seems that SyncEvolution which also provided a way to synchronise calendar and contact data, was discarded or overlooked at the time and a new synchronisation framework was designed. Here is an extract from the wiki page of that time describing Buteo’s capabilities:
- Device to device synchronization of PIM data (Contacts and Calendar for
now) based on SyncML- Device to cloud synchronization of PIM data (ovi.com, Scheduleworld.com
etc.) based on SyncML- Synchronization of media content to PC based on MTP
- Device to PC synchronization (currently only OviSuite, but this requires
contract with OviSuite to support a particular device)
From the start, Buteo was architected with the following components:
- a background process responsible to orchestrate synchronisation (see the
msyncdprocess), - a library to handle synchronisation profiles,
- a plugin system to implement various synchronisation mechanisms.
A synchronisation profile is a set of information concerning a service to be synchronised. It is serialised on disk using XML. Here is an example of profile for a public calendar on the Internet:
<?xml version="1.0" encoding="UTF-8"?>
<profile name="vacances" type="sync">
<key name="destinationtype" value="online"/>
<key name="enabled" value="true"/>
<profile name="webcal" type="client">
<key name="allowRedirect" value="true"/>
<key name="remoteCalendar" value="https://fr.ftp.opendatasoft.com/openscol/fr-en-calendrier-scolaire/Zone-A.ics"/>
</profile>
<schedule interval="720" days="" time="" enabled="true" />
</profile>
The profile is a sync profile (there are some other kinds of profiles like the storage ones, but they are not used much nowadays), with a single service using the webcal client plugin.
In fact several services can be attached to the same profile, but this feature is also seldom used; it’s more common nowadays for multiple separate profiles to be created, one for each service.
The client part stores some data required by the webcal plugin to actually operate the sync. In this case the rather essential address of the Internet resource to be fetched. The last part is the <schedule> element used to describe when the client plugin should be called. Here it is an interval="720" setup, meaning discrete synchronisation every 720 minutes (twice a day). This <schedule> is quite versatile and offers to synchronise:
- at a given time or by interval,
- on specific days of week,
- with a different interval or time during some given period of time.
The account page in the Settings application provides a UI to actually set up such a schedule. Use the “Custom” choice in the list of preset schedules to access the complete set of possibilities offered by Buteo.
The profile files are located* in $HOME/.cache/msyncd/sync/ which is a privileged directory, so you’ll need to use devel-su -p to access it. All profiles are read by the msyncd background process and they are dispatched according to their schedule. If the schedule specifies a particular time for sync, then the device will even be woken up at that time. In the case of interval scheduling, the device is woken up periodically and profiles are grouped to save battery. In case the device is out of reach from any network, and a profile requires a sync via the network, this profile will be synced as soon as the device reconnects to the Internet. When the device is in low battery mode (below 10%), all synchronisation is disabled.
The msyncd daemon offers a DBus interface for interact with it:
-
startSync()andabortSync()can be used to start and stop the synchronisation of a given profile, using the profile name as identifier, -
syncProfile(),updateProfile()andremoveProfile()can be used to manage profiles, -
msyncdalso emits asyncStatus()signal whenever a change is done to the status of a profile or a change is made to the profile itself.
The architecture of Buteo is modular and based on plug-ins to provide synchronisation for various service. Plugins are run as separated processes (for better insulation between the background process and the plugin code). Here is an incomplete list of currently supported services:
- Calendar data synchronisation with generic CalDAV servers, Nextcloud (using the same plugin as for CalDAV) or Google calendars, but also shared calendar resources.
- Contact data synchronisation with generic CardDAV servers, Nextcloud or Google contacts.
-
Emails by periodically asking the
messageserver5background process to fetch mails, or by delegating it to an “always up to date” status, whichmessageserver5then handles fully itself. - Exchange sync, which is part of the proprietary package offered by Jolla. This provides EAS email, calendar and contacts synchronisation, similar to
messageserver5except using thesailfish-easbackground process instead. - …
Apart from Exchange, all of these plugins are open-source and contributions are welcome.
Besides existing plugins, writing a new plugin for Buteo is quite simple. It requires basically to implement five methods:
-
init()which is called at plugin initialisation and where information can be retrieved from the associated profile (uninit()is called before finalisation of the plugin process to release any resource if required), -
startSync()which is called immediately afterinit()returns and where the actual syncing work should be implemented, -
abortSync()which may be used to cancel any pending synchronisation process, - and
cleanup()which will be called only once when a profile is permanently deleted, to clean any saved resources on device.
When the sync operation is finished, the plugin should emit either success or failed signal and some additional logging information could be stored in a Buteo::SyncResults object (this information is then saved into logs by msyncd).
The simplest plugin you could look at is the webcal service plugin, and particularly its src/webcalclient.cpp file. I wrote this plugin from scratch to allow to sync “static” calendar resources from the Internet (that’s why it’s quite simple, it just downloads the public calendar as ICS data, scratch the existing entries on device for this calendar and add the received ones).
As we can see from these, various protocols for synchronising data can be used. The SyncML one, which is kind-of historic now, also has a plugin to handle various services including calendar and contact data. But this plugin has not been actively maintained for a while (once again contributions are welcome, don’t hesitate to contact me if you want to look into its sources).
Nowadays, Buteo is already pretty stable and mainly in maintenance mode, receiving bug corrections or code cleanups. For example, here are a few recent changes:
- Commit 5983eb7 correcting wakes-up during user-defined sleeping periods.
- Commit f1f7d9d removing custom logging functions in favour of Qt ones.
- Commit 36e1cf9 reworking plugin builds to avoid exposing internal
.cppfiles. - …
So what about the future? I’m working on QML bindings for Buteo. Currently, Sailfish OS uses QML bindings for Buteo from the sailfish-components-accounts package which is closed source and mainly related to accounts (see libaccount-qt). The main reason for adding QML bindings would be to allow feedback in the UI on the synchronisation process. For example it would be useful to know that a calendar was last synced on a particular date, or that during the last sync two new events were retrieved from the server and one was uploaded. This is the purpose of my demonstration app harbour-logbook. It’s not yet ready for widescale use, since it’s waiting for various pieces like the Buteo QML bindings, and a new list model in the calendar QML bindings, to be accepted. But other pieces have already been accepted (mainly for CalDAV services):
- Commit ad52cb4 in Buteo to expand the log format to be able to save per-item information during a sync process.
- Commits 485ec0c and ce2b652 in buteo-sync-plugin-caldav to save the
instanceIdentifierof touched events during a sync to the Buteo log. - Commit 74adcde in mKCal to be able to load incidences from an
instanceIdentifierstring.
So stay tuned!
* The $HOME/.cache location is a bit strange and historical, it may be moved later to a more consistent location; something like $HOME/.local/share/system/Synchronisation or equivalent.
Energy from the Community
As always, Sailfish developers have been busy creating and updating their apps over the last two weeks, and as always there were more releases than we’re able to cover here. Nevertheless we have three apps that were recently updated and one that had its first release only recently. All of them provide some really nice functionality.
Foil Pics
As well as being a Jolla developer, Slava Monich (slava) is clearly also one of the more prolific Sailfish OS app developers and we’ve looked at several pieces of his software in previous newsletters (Foil Auth, QR Clip, Books, Value Logger, Counter). Foil Pics is part of slava’s “Foil” series of apps that also includes Foil Auth and Foil Notes. All three of these apps have a strong focus on privacy, with the ability to encrypt some or all of the data stored by them. Foil Pics, as the name suggests, is for managing image files. It’s design is similar to that of the Jolla Gallery app (very similar in fact to earlier versions of the Gallery app where the sharing options were hidden behind a split-pane view). The main additional feature is, of course, the ability to store and manage encrypted image files.
The encryption and decryption process is handled in a very straightforward manner. The app view is essentially split into two: one for showing normal files and the other for showing encrypted files. It’s a bit like having two tabs split across two screens. Files can be selected individually to view them, and at which point the pulley menu offers the option to view info about the file, delete the file, or encrypt the file. Select the encryption option and the image will be moved over to the “Encrypted” photos page. Both encryption and decryption are swift — taking less than a second each — and there’s also the option to select multiple files to encrypt or decrypt all in one go. Storing encrypted image files this way becomes straightforward and intuitive.
Slava’s triplet of Foil apps all use the same encryption mechanism underneath and so all share the same encryption password. The Foil apps very intentionally favour security over convenience, so one thing to be careful about is that if you forget your password, the options for recovering your data are limited (brute forcing may be an option, depending on how secure your password is). In short: don’t forget your Foil password if you don’t want to lose access to your data! The latest version allows image details to be copied to the clipboard, and also fixes the app so that it still works with the latest sharing changes in the Verla release. If you want to store encrypted images on your phone, then Foil Pics is a great option. It’s available from the Jolla Store and OpenRepos.
SailSeries
SailSeries by Jonne Pihlanen (joonne) is a fascinating app. It’s not the sort of app that it would occur to me that I might need, but now I’ve installed it, I think it’s going to run my life. The idea is simple: feed the app info about your favourite television series, and it will tell you when and where you have to be sat in front of a screen to be the first to see it. This might all seem a bit archaic in the age of on-demand streaming services, and selecting programmes this way certainly brings back memories of flicking through television listings in the 90s. Back then it was all about programming the video recorder to come on at just the right time (I say “programming” but it was more like arcane incantation, and I say “just the right time” but it was more commonly an hour too late… but I digress). Nevertheless, even streaming services have release dates for programmes, and if you plan to be able to discuss them with your friends the next day, you’ll need to watch them as soon after they’re released as possible. SailSeries is a great way to make sure you don’t miss out.
What really impressed me about SailSeries is the breadth of programme coverage. The data itself comes from TheTVDB, a community-driven online service that claims to include over 129 000 TV series and 266 000 movies. It’s also kept well up-to-date. I found most of the programmes I was expecting to be released this week had details available (including broadcast shows from Finland, Germany, the UK and USA; as well as shows from the big streaming services). It neatly populated my upcoming week with gruelling TV-watching tasks and can generate daily notifications of upcoming shows. In addition to structuring your life, it’ll also provide summaries and thumbnails of shows so you know you’re getting exactly what you expect. Having entered programme details you can leave it populated this way so that you’ll immediately know when the next episode is due to be released, even for series that are still way in the future. Another nice feature is the fact that data is stored locally and periodically updated, so you don’t have to be online to use the app (although you won’t get search or images without a connection).
The latest SailSeries update involved a big refactoring of the app to introduce worker threads, which should make everything work more fluidly. You can get yourself a copy of SailSeries from the Jolla Store.
Wunderfitz
I’d originally planned to write about Wunderfitz after its previous update back in June, but had difficulty getting all of the features working and so decided to hold off. It recently received another update, and although there are still some kinks to iron out, it’s such a useful app that I couldn’t hold off any longer. Wunderfitz is another app from Sebastian Wolf (WerkWolf), who’s Piepmatz app we looked at in the last newsletter. Wunderfitz provides an offline translation dictionary, allowing translation of words between different languages based on the dictionaries you choose to install.
There are other apps and ways to translate on Sailfish OS of course. I used to make extensive use of the Yandex service through the Translator app, but this no longer seems to be maintained. There are also several online translators which will do a great job. But a native offline app is not only easier to use, it also offers a sense of reassurance if you’re heading to another country with uncertain connectivity and unclear roaming charges.
However, it’s also important to understand the scope of Wunderfitz. In its offline form it’s not intended as a way to translate phras es . It really is much closer to a translation dictionary in terms of functionality. Give it a word and it will do a good job of translating it. Give it more than one and it will refuse to cooperate. As long as you keep your expectations and process in line with this, it does an excellent job. Personally I find it much more efficient to use than an online service.
WerkWolf has also chosen to add another tab, entitled “Curiosity”, to the Wunderfitz UI, which is bound to catch your attention. Selecting this allows you to perform translation of any block of text you point your camera at. This capability does require an Internet connection, since it uses the Microsoft Azure backend services. Because of this it’s also not a continuous process: after capturing an image it gets uploaded to the service, and this process can take a second or two. I found it was able to copy well even with quite dense blocks of text.
As I mentioned there are some kinks. The dictionaries come from Dict.cc, and although the quality of the material is pretty good, the process for getting them is pretty bad. They have to be downloaded manually, via an intricate process involving multiple emails. WerkWolf can’t be blamed for having to follow Dict.cc’s process, but it certainly makes the setup process more complex. Similarly to get image translation working, various Azure keys have to be provided. Azure is a bit of a maze for the uninitiated, and depending on how you create your services you may end up with an endpoint address that Wunderfitz won’t work with. The latest version already includes a host of UI improvements and bug fixes, and WerfkWolf has an excellent track record, so hopefully some of these issues can be ironed out in future releases. In the meantime, if you’re willing to put in the effort to set it up, Wunderfitz provides the best offline dictionary for Sailfish OS. It’s available from the Jolla Store and OpenRepos.
Speech Note
Like Wunderfitz, Speech Note is another single-task, super-simple-on-the-surface tool, but that’s built on much more complex foundations. On first appearance it seems to have a one-button UI, mimicking the voice assistant buttons you might find in other operating systems. Press and hold the button, speak into the microphone, and the text of your voice will be automatically transcribed into the text field that fills the remainder of the screen.
As I say, it’s a really simple concept, built on highly-complex foundations. Developed by Michal Kosciesza (mkiol), the app makes use of Mozilla’s open source DeepSpeech models for converting audio speech into text. This is the same technology that can be used with the open source Mycroft voice assistant, and in English at least, during my testing it proved to work remarkably well. The app also supports Germain, Spanish, French, Italian, Polish and Chinese, although sadly I wasn’t able to test these. If anyone else has tried them, please post your experiences in the comments.
Like Wunderfitz there’s also an initial setup stage that needs to be completed before the app can be used. Happily this only involves selecting the relevant language followed by a brief pause while the relevant data are downloaded. The English model took around 3 minutes to download for me on a very average internet connection. It’s pretty painless, and worth the effort, because in contrast to the major commercial voice assistants, using a local model means there’s no need to send your data to a cloud service. From a privacy-perspective, this is an important feature. With all of the processing happening locally, it had no problem keeping up with the speed of my speech, and the quality of results was excellent as long as I enunciated clearly. For transcribing general conversation it’s not likely to fare so well. and either way some editing will likely be needed afterwards. Even given this it’s still likely to be far swifter than using your phone’s keyboard for getting large pieces of text onto your device.
Overall, I must admit I was impressed. The transcription works really well, the app provides a clean and effective interface. The privacy balance is likely to be appreciated by many Sailfish OS users. In addition to transcribing speech through the microphone, the latest update also offers audio file transcription and various UI improvements. It’s available from OpenRepos.
Community Shout-Out
With the release of Sailfish OS Verla 4.2.0 we saw a whole host of articles about the new features and functionality, many from community members, which we wanted to pay tribute to. There are several country-specific sites that share Jolla and Sailfish news, so for example we were pleased to see it covered by the community site in the Netherlands. We also saw the news aggregated by linuxleón, liliputing (requires scrolling), Tweakers (also based in the Netherlands) and OSNews. As we already mentioned in the previous newsletter, Leszek Lesner covered the features in one of his excellent YouTube videos. Last but not least, Nico Cartron (cartron) wrote a nice overview written in both English and French. We love to hear about community coverage of Sailfish OS releases, so if you know of others, do let us know.
Please feed us your news
This is a community update, and frankly we can’t always keep up with all the exciting stuff happening in the Sailfish community. Plus, the less of this we have to actually write ourselves the better. So please help us out by posting your Sailfish news updates to the forum as a reply to this post. We’ll collate as much of it as possible into one easily digestable post for the next update.
And don’t forget to join us at the community meeting every other Thursday on IRC. It’s a great place to discuss any of the content you see here, or to share your ideas for future updates. The next meeting will be on the 30th September, full details here.
