Migrating configuration and data files for sandboxed apps

[vige]

Unfortunately we don’t yet support launching apps in sandboxed mode directly from the SDK.

[martyone]

Good question - could you please post it as a separate topic? There might be some partial answer at least.

[wetab73]

Hi Martin,

Sure, I’ll create a separate topic for it. Even a partial solution would be welcome.

[Thaodan]

@vige

Starting with Sailfish OS 4.3.0, if the .config/harbour-appname directory exists, it will also be visible in the sandbox. So basically, this is what you should do in order to migrate your configuration:

@Ygriega ^

Here’s my way of migrating my userdata as I did in Communi-Sailfish:

void SettingsProxy::migrateSettings()
{
    QString oldConfigFileStr =
        QStandardPaths::standardLocations(QStandardPaths::ConfigLocation).first() +
        "/" + "harbour-communi/IRC for Sailfish.conf";
    QString newConfigFileStr =
        QStandardPaths::standardLocations(QStandardPaths::AppConfigLocation).first() +
        "/" + QCoreApplication::applicationName() + ".conf";

    if((!QFileInfo(newConfigFileStr).exists() && !QDir(newConfigFileStr).exists()) &&
        (QFileInfo(oldConfigFileStr).exists() && !QDir(oldConfigFileStr).exists())) {
        QFile::rename(oldConfigFileStr, newConfigFileStr);
    }
}

This might help other struggling with the issue.
With this example the app doesn’t need to know anything about being migrated, we just call this before initializing our QSettings instance.

E.g. in the case of communi-sailfish:

SettingsProxy::SettingsProxy(BufferProxyModel *bufferModel,
                   IgnoreManager* ignore, QObject* parent)
{

    this->migrateSettings();

    m_settings = new QSettings(
        QStandardPaths::standardLocations(QStandardPaths::AppConfigLocation).first() +
        "/" + QCoreApplication::applicationName() + ".conf",
        QSettings::IniFormat,
        parent);

    m_ignoreManager = ignore;
    m_bufferModel = bufferModel;
    this->restoreSettings();
}

The file quoted can be found here: https://github.com/communi/communi-sailfish/blob/master/src/app/settingsproxy.cpp

After this we can also spawn DBus services that match the tld.orgname.appname.

All code shown here is license as GPL-2.0.

[poetaster]

Thanks! I wish I’d had this before I migrated

[poetaster]

@vige

As long as you are using libsailfishapp , the values are set automatically, using the values from the .desktop files.

This is not exactly the case. If the settings backend is a custom class that declares orgname and appname the settings from desktop (or main, for that matter) won’t apply.

Settings.h, sailjail default locations

Believing the default paths to be automagcially available when declaring Sailjail org/name, I hadn’t done anything to explicitly declare them in machines vs. machines main. This seems to be in error. The machines-vs-machines-sfos.cpp includes: QObject* createSettings(QQmlEngine *engine, QJSEngine *jsEngine) { return new Settings(); } qmlRegisterSingletonType<Settings>("harbour.machines.vs.machines.sfos.Machines", 1, 0, "SettingsBackend", createSettings); The sailjail has org set (de.poeta…

It’s a bit sordid, but basically if the class delegated to handle settings for QML declares org and app in the global namespace it won’t jive with the desktop file.

To save you from navigating, if you do something init a settings class like:

#define SETTINGS(group) QSettings settings("com.github.mzanetti", "harbour-machines-vs-machines-sfos"); settings.beginGroup(group)

That corresponds with the orgname and appname in your .desktop file. It won’t work.

What does work is using the Settings init that specifies path:

#define SETTINGS(group) QSettings settings("com.github.mzanetti/harbour-machines-vs-machines-sfos/harbour-machines-vs-machines-sfos"); settings.beginGroup(group)

[Thaodan]

@poetaster

To save you from navigating, if you do something init a settings class like:

#define SETTINGS(group) QSettings settings("com.github.mzanetti", "harbour-machines-vs-machines-sfos"); settings.beginGroup(group)

That corresponds with the orgname and appname in your .desktop file. It won’t work.

It doesn’t because that path will be still outside of the sandbox using this, e.i. it will resolve to $XDG_CONFIG_HOME/org.foobar/foobarapp.conf.
But what @vige was saying that libsailffishapp will set the values from the desktop file to the QApplication object and thous all instances that the meta data in it like QSettings.
The issue is that the default path for QSettings lies outside of the sandbox.

PS: The organisation name doesn’t have to include the full url to your github page, its just org.name.

[poetaster]

My point was that the settings files were not visible within the jailed version of machines vs machines with

#define SETTINGS(group) QSettings settings(“harbour-machines-vs-machines-sfos”,
“harbour-machines-vs-machines-sfos”); …

Although that created the config directory as per (~/.config/harbour-machines-vs-machines-sfos/harbour-machines-vs-machines-sfos.conf):

@vige

Starting with Sailfish OS 4.3.0, if the .config/harbour-appname directory exists, it will also be visible in the sandbox. So basically, this is what you should do in order to migrate your configuration:

This was not the case. The config file was not visible some number > zero, 4.4 users. I got complaints. Worked for me, but I’m still on 4.3.

It get’s more hairy with stuff like:

fahrplan2.pro: DEFINES += FAHRPLAN_SETTINGS_NAMESPACE=\“harbour-fahrplan2\”
main.cpp: QSettings settings(FAHRPLAN_SETTINGS_NAMESPACE, “fahrplan2”);

That get’s us .config/harbour-fahrplan2/fahrplan2.conf should, might still be visible in 4.4?

The solution here is? Looks like writing an ifdef for Sailjailed sailfish. Or?

As for the PS, I’m trying to render the only ‘authoritative’ original org there is. And that’s zanetti’s github repo. He’s merging PRs so that seemed like the logical orgname to me?

[Thaodan]

@poetaster

#define SETTINGS(group) QSettings settings(“harbour-machines-vs-machines-sfos”,
“harbour-machines-vs-machines-sfos”); …

Although that created the config directory as per (~/.config/harbour-machines-vs-machines-sfos/harbour-machines-vs-machines-sfos.conf):

It can be accessed with the default profile, for the explicit profile the path in ~/.config/appname can be accessed if it already exists.

The issue lies here as I explained:

@Thaodan

But what @vige was saying that libsailffishapp will set the values from the desktop file to the QApplication object and thous all instances that the meta data in it like QSettings.
The issue is that the default path for QSettings lies outside of the sandbox.

@poetaster

fahrplan2.pro: DEFINES += FAHRPLAN_SETTINGS_NAMESPACE=\“harbour-fahrplan2\”
main.cpp: QSettings settings(FAHRPLAN_SETTINGS_NAMESPACE, “fahrplan2”);

That get’s us .config/harbour-fahrplan2/fahrplan2.conf should, might still be visible in 4.4?

Same case as for all apps as I explained for your first case in this post. You can access the path in the default profile always and in the explicit profile only if it already exists.

[poetaster]

@Thaodan

You can access the path in the default profile

Ok, I think I’m stumbling on this point. I’ll go RTFM. I had been using an explicit profile in the belief that this would be a harbour requirement.

[Thaodan]

It is not however you should move to one. In case the app had data from before having an explicit profile you can still access the data, you should just move the data to the new paths to it is same for old and new users.

[rinigus]

On the basis of reading sailjail code, if we don’t define OrganizationName and ApplicationName in X-Sailjail section, the app can have the same storage arrangement as before. Manuals are not clear on whether it is intended or not. At the same time, I can still set permissions and those would be taken into account. Such .desktop file passes sfdk harbour check as well.

Is this considered as a feature which will stay or there is expected break for it in future?

PS: unless use_compatibility in daemon/sailjailclient.c is false - couldn’t decipher with a fast glance

[pawel.spoon]

i did copy over the databases. i can see that copy was successfull, but when app starts it says it is a fresh install

can it be that i have screwed the appname/pkname/orgname settings in main.cpp ?
as usual the issue was between the ears
but should appname contain the displayname or harbour-blba-bla ?

[direc85]

My go-to method is to clear up a test device (or emulator), make a new app installation with supposed old-and-new folder access settings, and see where the new datafile actually lands. It’s a bit more labor to do it that way, but it takes the guesswork off it

[pawel.spoon]

sql db works now, but config not
harbour-olive-goes-shopping.conf
[General]
categories=true
categorizeItems=true
categorizeShoppingList=true
migrated=true
[defaultuser@VollaPhone .config]$

you can see that migration did happen but the values are not loaded, nor are manual changes persisted ?

solution: the problem was not the path or appname but the usage of settings

i did copy it from somwhere years ago and it always creates an instance of setting, obviously with the default location which is now wrong

[dcaliste]

I was hoping the same as you at first glance, but there is a caveat : if the directories already exists under the before-sailjail-paths, then they are mounted inside the jail. So far so good, we keep the same behaviour. But you cannot create directories inside the jail that will be persistant. So, for all new users, their data are lost at each restart of the application.

[Thaodan]

Can’t you copy over the old db to the new paths? The app should be able to access the old “harbour-foobar” directories when using the new sandbox profile.

[nephros]

@dcaliste

But you cannot create directories inside the jail that will be persistant. So, for all new users, their data are lost at each restart of the application.

I guess this matches up with the Firejail man page for a whitelist entry:

@man firejail

–whitelist=dirname_or_filename

Whitelist directory or file. A temporary file system is mounted on the top directory, and the whitelisted files are mount-binded inside. Modifications to whitelisted files are persistent, everything else is discarded when the sandbox is closed.

Now if you consider using a whitelist in your application profile, be aware that:

https://wiki.archlinux.org/title/Firejail"

Whitelists block everything what is not explicitly whitelisted. They should not be used in profiles for applications that need access to random locations (e.g., text editors, image viewers/editors).

[dcaliste]

Indeed, it has been designed for migration and it helps to copy data from the old directory to the new location. My remark was more about ‘was it possible, by not giving an organisation name, to get the old behaviour ?’ The answer is yes, but only for existing installations, which means ‘no’ in general for this specific question.

[direc85]

I was implementing migration of the storage folders of Whisperfish (Rust application), when I noticed that I could not simply move (i.e. std::fs::rename) the folders from the old to the new location if started with Sailjail (i.e. normally from the launcher icon). I had to copy-and-delete the files in them recursively instead. Luckily there’s a crate that provides fs_extra::dir::move_dir which does exactly that!

I believe this is because Sailjail mounts the old storage folders separately from the new ones, and as such they appear to be in different filesystems (mount points).

Hopefully this piece of information helps with others struggling with failing moves - although it looks like QDir::rename tries to move, and falls back to copy-and-remove automatically.