Sailfish Community News, 1st July, Calendar Stack

Subscribe to future posts here :bell:

Sailfish OS update from Jolla

In Finland we’ve been enjoying midsummer, or juhannus as it’s known here. That means bonfires, saunas and the start of the holidays for many. It also means there’ll be a gap until the next community meeting. The next one is scheduled for the 5th August. So you have plenty of time to clear your calendars and think of super-interesting questions.

At the last meeting before the summer on the 24th there was plenty of lively discussion about granting Android apps access to Bluetooth capabilities, dual-booting Sailfish OS and Android on the same phone, and reviving a non-booting Jolla 1. In case you weren’t able to make it you can read the full discussion in the logs.

Another important change that happened last week was the move to the OFTC.net IRC platform after having used Freenode for over a decade. The three Sailfish OS channels have now all moved: #sailfishos for general Sailfish discussion, #sailfishos-porters for discussion about porting to different hardware and #sailfishos-meeting where we hold the fortnightly meetings. There are bridges for Matrix and Telegram if you prefer to access them through those channels. Please do join us in our new home if you haven’t already.

If you’ve been following previous newsletters you’ll know that Damien Caliste is one of our most prolific community contributors. He’s put a huge amount of effort into maintaining and advancing the calendar capabilities of Sailfish OS over the years. You’ll have noticed some of the improvements he’s made as features directly visible in the Calendar app, such as the ability to set timezones for events, or greater flexibility with how events recur. I’m going to hand over to Damien now for a more detailed look at the calendar stack in Sailfish OS, how all of the pieces fit together, and some of the hidden — but crucial — changes he’s been contributing to the calendar code recently. Thanks go to Damien for this fascinating insight.

The Sailfish OS Calendar Stack

Let’s have a look at the software stack that supports calendar events in Sailfish OS. This stack is based on KCalendarCore, a Qt abstraction for representing events, todos and journal entries developed in the KDE project. Nowadays KCalendarCore is part of the KDE framework and serves as a foundation for the calendar-related operations in KDE. It is a small and stable library, but it started its life as a part of the old KCal, a huge piece of code from the KDE4 era. Here’s the first commit back in 2010 from Allen Winter, who is still the maintainer today:

The KCalCore library deprecates and mostly replaces the KCal library.
KCalCore is free of any relation to the old Calendar resources and
focuses entirely on iCalendar and vCalendar storage and data manipulation.
KCalCore used QSharedPointers for safe memory access, is free of i18n
strings and contains no methods for user interaction: KCalCore is all
about the calendar data.

From this we can see that KCalendarCore is also used to write and read calendar resources to and from iCalendar format (the vCalendar format is now deprecated), allowing events to be exchanged with servers or other devices. To do this, it relies on the libical library, which is used in many environments, KDE of course, but also Gnome. Here is an example on how to read an iCal serialisation:

#include <KCalendarCore/ICalFormat>

    KCalendarCore::ICalFormat format;

    // This represents the serialised iCal string data received from 
    // another device
    const QString serializedCalendar = QLatin1String(
        "BEGIN:VCALENDAR\n"
        "PRODID:-//IDN nextcloud.com//Calendar app 2.0.4//EN\n"
        "VERSION:2.0\n"
        "BEGIN:VEVENT\n"
        "DTSTAMP:20201103T161340Z\n"
        "UID:bd1d299d-3b03-4514-be69-e680ad2ff884\n"
        "DTSTART;TZID=Europe/Paris:20201103T100000\n"
        "DTEND;TZID=Europe/Paris:20201103T110000\n"
        "SUMMARY:test recurring\n"
        "RRULE:FREQ=DAILY;COUNT=4\n"
        "END:VEVENT\n"
        "BEGIN:VEVENT\n"
        "DTSTAMP:20201103T161823Z\n"
        "UID:bd1d299d-3b03-4514-be69-e680ad2ff884\n"
        "DTSTART;TZID=Europe/Paris:20201104T111500\n"
        "DTEND;TZID=Europe/Paris:20201104T121500\n"
        "SUMMARY:test exception\n"
        "RECURRENCE-ID;TZID=Europe/Paris:20201104T100000\n"
        "END:VEVENT\n"
        "END:VCALENDAR");
    // Create an empty calendar event to store the data into
    KCalendarCore::MemoryCalendar::Ptr calendar(
        new KCalendarCore::MemoryCalendar(QTimeZone::utc()));
    // Parse the data into the event
    format.fromString(calendar, serializedCalendar);

This code snippet creates a memory calendar to store events and populates it with the serialized data containing a four-day recurring event starting on the 3rd November with an exception on the 4th November as specified in the iCal string.

Events can then be retrieved from the memory calendar with the following code snippet:

    // Reference the event using the unique ID from the original iCal data
    const QString uid = QString::fromLatin1(
        "bd1d299d-3b03-4514-be69-e680ad2ff884");
    // Access the main event
    KCalendarCore::Incidence::Ptr parent = calendar->incidence(uid);
    // Access the exception to the main event
    KCalendarCore::Incidence::Ptr exception = calendar->incidence(uid,
        QDateTime(QDate(2020, 11, 4), QTime(10, 0), 
        QTimeZone("Europe/Paris")));

The calendar can contain events, todos or journals, which are generically called incidences. They are indexed by their UID and potentially their recurrence ID (which is the starting date that the exception is replacing). The calendar API also provides methods for fetching incidences based on dates.

The KCalendarCore API is quite complete and supports all the RFC5545 requirements and a good portion of its updates, such as those in RFC7986 which brings support for colour per incidence.

Coming back to the creation of KCalendarCore, as stated in the initial commit KCalendarCore only deals with a memory representation of calendar resourcess, which brings us to mKCal. This second library is responsible for the permanent storage of incidences on your device. As far as I can tell, this piece of code was also separated from the KCal monolithic library in 2010. As visible from the master repository location (namely SailfishOS in GitHub), this library is not used by KDE. Actually KDE relies on a wider storage daemon: Akonady. In contrast mKCal targets embedded environments with constrained resources, and so relyies on a small SQlite database to store incidences for all of its calendars. I think the “m” letter in the name stands for “mobile” extension to KCal.

The library consists mainly of a SQlite driver to read and write incidences into a central database file. It also overloads the KCalendarCore::MemoryCalendar class to associate storage backends to it so that modifications can be transparently propagated between the storage and the memory representations. By default, the storage file is located in $HOME/.local/share/system/privileged/Calendar/mkcal/db. One can inspect the file with the sqlite3 tool in the sqlite package, but note that the folder requires privileged access, so you’ll need to use devel-su to open it.

devel-su -p sqlite3 $HOME/.local/share/system/privileged/Calendar/mkcal/db

There are two main tables:

  • the calendars table that lists the different calendars and their properties (associated colour, local or synced with a server…),
  • the components table that stores all the incidences for all calendars. The schema for this table looks a bit frightening, but it’s basically storing the description of the event, its summary, its starting and ending dates and so on. The dates are stored in three fields each, so that they can be stored as either a local time or a time with an associated time zone. Local times are always the same whatever the timezone, and might be used for things like a daily exercise which always happens at 8pm independent of whether you’re in Paris or London, say.

On top of these two libraries, the nemo-qml-plugin-calendar library provices QML bindings to the calendar resources. It also adds threaded read and writes to ensure that database accesses won’t block the UI.

For example, this QML page uses the AgendaModel from nemo-qml-plugin-calendar to display all of the events within a given timespan in a simple list. Here we use the Format.formatDate() function to display the times and dates in a human-readable form. For each itme in the AgendaModel we can use model.event.displayLabel to get the title of the event, model.event.startTime to get the start time and so on. You can see all of the available properties in the calendarevent.h header.

import QtQuick 2.0
import Sailfish.Silica 1.0
import org.nemomobile.calendar 1.0

Page {
    property date start: new Date(2021, 5, 28)
    SilicaListView {
        anchors.fill: parent

        header: PageHeader {
            title: "Week beginning " + Format.formatDate(start, 
                Formatter.DateLong)
        }

        // Read the data from the calendar
        model: AgendaModel {
            startDate: start
            endDate: QtDate.addDays(start, 6)
        }

        // Display the data in a list
        delegate: ListItem {
            contentHeight: Theme.itemSizeLarge
            Column {
                x: Theme.horizontalPageMargin
                width: parent.width - 2 * x
                anchors.verticalCenter: parent.verticalCenter

                Label {
                    color: Theme.highlightColor
                    // The display title of the event
                    text: model.event.displayLabel
                }

                Label {
                    color: Theme.secondaryColor
                    // The start and end times of the event
                    text: Format.formatDate(
                              model.event.startTime, 
                              Formatter.DateMediumWithoutYear)
                          + ", " + Format.formatDate(
                              model.event.startTime, 
                              Format.TimeValue)
                          + " - " + Format.formatDate(
                              model.event.endTime, 
                              Format.TimeValue)
                }
            }
        }

        VerticalScrollDecorator {}
    }
}

As noted above, access to the calendar database is privileged, so this QML code must be run with privileges as well, otherwise it won’t find any of the events. This is also one of the reasons why apps using nemo-qml-plugin-calendar aren’t currently allowed in harbour.

All of the calendar stack is still actively maintained:

  • I started in 2019 to move from the old KDE4-era KCalCore library to the latest upstream version (renamed KCalendarCore). This work included a move from the deprecated KDateTime to the Qt5 QDateTime. This last move resulted in many changes to the API which had to then be propagated to all of the software that made use of it. I thank Bea Lam (but also Pekka Vuorela, Chris Adams and David Llewellyn-Jones) for all the help they provided with the migration. It finally landed in 4.1.0, removing the remaining bits of KDE4 from your device \o/.

  • Using the upstream KCalendarCore also means that Sailfish OS development contributes to KDE with bug fixes, such as this wrong UTC export bug, or features, such as allowing a colour property per event. But also the reverse, allowing Sailfish OS to benefit from bug fixes and improvements coming from the KDE developers.

  • mKCal received a lot of love in the last months with major code cleaning, for example with commit 1d842e302915d removing 1200 lines of duplicated code shared with KCalendarCore (after some tuning here and there to rule out all of the minor variations that existed in the duplicated code). Additions are also regularly done to improve its API:

    • I can cite commit 007fcd1a6 adding the possibility to fetch all incidences deleted on device but not yet synced.
    • Or this other commit which allowed a lot of code from the sync plugins to be simplified by not always updating the lastModified of a saved incidence.
    • The mKCal build systems were recently switched to use cmake instead of qmake, thanks to a contribution from @PureTryOut.

As previously mentioned, the support for calendar capabilities is reasonably complete in KCalendarCore and mKCal. It’s often in the QML bindings that glue code is missing when one wants to add support for this or that functionality. Here are some of the more recent changes bringing support to the UI for various new improvements:

An important topic that I’ve not covered here concerns the synchronisation framework. This is also an area that has received various contributions from the community and from sailors at Jolla. For instance, Google sync and generic CalDAV sync now flag faulty events individually during a sync so that a warning can be displayed in the calendar for an events that didn’t sync correctly. But let’s keep the details of the sync framework for a future newsletter.

Energy from the Community

As always we have a fine selection of new and updated apps for you to enjoy. If you’ve released a new app, or are updating an established app, don’t forget to let us know in the comments below.

Ielig Web

Browsers are amazing pieces of software, practically operating systems in their own right. But sometimes the rich feature set they come with, coupled with the suitably complex user interface to match, is more than is needed to quickly skim through the latests forum posts or news items. This is where Ielig Web (starts with an “i” not an “L”) comes in. Written by hanswf, Ielig Web is designed to get you access to those sites you visit often as swiftly and unobtrusively as possible. Rendering is performed by WebEngine, so you won’t get the same level of compatibility as with the main browser, but to some extent that’s the point. If you’re willing to streamline your experience even further you can deactivate javascript, block image loading and switch to a different user agent so that only the info you really need gets rendered. Obviously whether you can access what you need this way will depend on the site, but one of the key features of Ielig is that you can build up a list of bookmarks on the main page, each with their own optimised site-specific settings. Once you’re viewing a site, don’t expect a flashy interface. Quite the reverse in fact. The Ielig interface is as unobtrusive as possible, rending a translucent rectangle in each of the four corners of the screen which you can press in order to (starting top left and working clockwise) go back, go forwards, return to the bookmark list and reload. These, combined with a four-entry pulley menu represent the entire browser UI. This leaves the entire screen available for viewing content. As the author notes on the help page, you wouldn’t want to browse the web this way all of the time, but for access to specific information quickly, it may be just the approach you need. The latest update provides 64-bit ARM support and a new “Download” entry in the pulley menu, and it’s available from the Jolla Store.

Piperka

The Piperka (“Pepper” in Bulgarian) website indexes literally thousands (4887 at last count) of regularly released webcomics. The site allows you to browse the index and bookmark the strips that interest you, ultimately allowing you to keep track of those you’ve read and any new ones that are published. The Piperka Client app by Kari Pahula (kaol) does pretty much the same thing, but in Sailfish style. That means you can zip through the long list of comics by scrolling down or entering a search term, find the comics you like and bookmark them for future reading. You can do all of this without having to create an account, but if you do supply your Piperka credentials you’ll find your bookmarks and viewing state are neatly synced with the Piperka website. Once you’ve selected a few comics you enjoy you can start to get recommendations for other comics the app thinks you might appreciate as well. Helpfully the more risqué comics are clearly marked to avoid anyone stumbling across something that might offend. When it comes to reading the comics themselves the app renders the original site of the comic in a web view. Some sites are better than others: they’re often optimised poorly for a mobile display and I found I had to do a fair amount of panning and zooming to properly appreciate the stories and artwork. Thankfully the app provides its own consistent controls for moving forwards and backwards through the strips, so you don’t have to wrestle with the fiddly controls on the sites. The fact Piperka is just an index rather than hosting the content itself is important, because it means it can sidestep some otherwise difficult copyright issues. This allows it to maximise the list of supported comics. It’s a decent and practical compromise. The latest update provides 64-bit ARM compatibility and is available from the Jolla Store. It’s an app that’s well worth installing if you’ve not tried it yet.

DartsScore

I’ve often pondered the wisdom of having dartboards in pubs. Sure, darts is a fun game, but throwing spiked projectiles around after a few pints just never seemed that sensible. Using the sharp end of a large stick to propel rocks across the table (I’m talking about Pool) was always more my thing, but I digress. Actually, darts can be a very serious pastime and keeping careful track of the scores is important if you’re to avoid falling out with your friends. DartsScore by rgrnetalk allows you to do just that. What’s more, it has a super-streamlined interface to avoid the need to type in any numbers. Just press the button corresponding to the target you — or one of your opponents — just hit, prefixed with a Triple or Double modifier as appropriate. The app does all of the score-keeping for you, keeps track of whose turn it is and will even tally multiple games across multiple sets. It supports between two and four players and you can set it up with player names to rule out any confusion. Ultimately, it’s an app to do a very specific task, and that task it does well. I can see a future in which Sailfish OS devices equipped with DartsScore are pinned up next to dartboards in pubs everywhere, replacing all those traditional but unreliable chalkboards used today. The latest update to DartsScore provides improved icons and an ARM-64 build. If you’re a dartist, this is definitely one to have installed, and it’s available from the Jolla Store.

ZirkelTraining

Okay, so I have to admit I don’t have the best track record when it comes to exercise. Basically I’m more interested in technology than breaking a sweat. So maybe testing an app that’s designed to help with circuit training is exactly what I need? ZirkelTraining by Sikarjan is a great way to feed your technology needs while at the same time incentivising exercise. Opening the app for the first time is a bit overwhelming. There are buttons and sliders everywhere, menus at the top and bottom of the screen, and some ominous looking “plus” symbols that look very much like they’re going to result in having to do even more exercise. But once you get the hang of the configuration steps it all starts to make sense. First choose a training mode such as pyramid (where the exercises crescendo to a peak before tapering off again, with the app mirroring your exercises automatically), choose the duration you’ll spend exercising and the time needed for recovery (gasping for breath) afterwards. Set the number of cycles you’ll do for the exercise. Now move on to the Exercise List and add the name of the exercises you’ll be performing, add them to the list, reorder them as needed and Accept the results. Save this for later user. If you’ve still got energy after all that, hit the Start button to really get to work. The crucial point is that while the configuration can be a bit fiddly, when the exercises start the app puts all that effort to good use by carefully leading you through the cycles. It will clearly tell you what to do at each stage. When you’re mid-cycle and pushing yourself to the limit, it’s good to have your phone dealing with the secondary work of keeping track of times and activities. There’s a lovely big countdown display dominating the screen and the countdown audio makes you feel like you’re in the middle of an action movie. Unfortunately I wasn’t able to get the music playlists to work, but if you can it’ll allow you to play Eye of the Tiger on repeat. The latest update improves profile saving and is available from the Jolla Store. Now it’s time for me to have a lie down.

Please feed us your news

This is a community update, and frankly we can’t always keep up with all the exciting stuff happening in the Sailfish community. Plus, the less of this we have to actually write ourselves the better. So please help us out by posting your Sailfish news updates to the forum as a reply to this post. We’ll collate as much of it as possible into one easily digestible post for the next update.

At this point I’d usually also invite you to join us at the community meeting on IRC. However, as noted above, during the summer there’s a bit of a hiatus while we all restore our batteries in the summer sun. The next meeting will be on the 5th August, so please do proposed topics and keep an eye on the post to find out what we’ll be discussing. In the meantime, don’t forget to let us know about any news or updates to include in the newsletter in the comments below.

18 Likes

Thanks for the insight.

From community perspective, this might be important:

coderus is passing over private projects

Biggest impact for most of us - patchmanager.

Biggest impact for me:
No personal ringtones for aarch64. This keeps my new X10 II in the drawer.
(Else I could use my old Nokia 6310 - my last phone without contact-specific ringtones. :weary:)

Probably this is the moment to thank coderus for his contibutions to this community.
He took care of so many issues, even smaller ones, to make life with Sailfish OS so much easyer.
e.g. add a search engine to the stock browser - this functionality finally found its way to one of the last Sailfish updates, but all the years up to that point there only was ‘sailfish-browser-searchengines’ from coderus.
Thanks for that!!!

15 Likes

Yes indeed @coderus deserves thanks for everything he’s brought to the community over the years (as well as everything I expect he’ll continue to bring in the future, even if in different ways). I’ll try to include something in the next newsletter too.

8 Likes

Thank you, @dcaliste, for your love spent to this part of the stack and the very interesting technical summary!

4 Likes

Thanks to Coderus indeed. I rebooted my devices many times and each time the first I did was installing this ‘search engines for Sailfish’.

3 Likes

Oh, regarding Calendar app.
How can I make it not only display Birthdays in Calendar for contacts that have it set but also notify me of them?

2 Likes