Index for media files & documents not updating

[rozgwi]

REPRODUCIBILITY: sporadically
BUILD ID = OS VERSION: 3.4.0.24
HARDWARE: Xperia X
UI LANGUAGE: DE
REGRESSION: unknown

DESCRIPTION:

Tracker does not update the index for media files and documents anymore.
I.e. despite having taken photos, dowloaded files during the whole week, the latest files available im Gallery and Documents are from Tuesday. In between a reboot has happened, without changes.
The problem occurs independently from using the internal (unencrypted) storage and an encrypted SD card.
Reindexing from the Sailfish Helper Tools does fix the problem but only for so long.

PRECONDITIONS:

Any media files and/or documents stored on the phone. (Possibly an encrypted sdcard plays into the issue as well.)

STEPS TO REPRODUCE:

1. Take a photo, download or copy any image/document file from an external source
2. Open gallery
3. Open documents
4. Check if new media is displayed

EXPECTED RESULT:

New photos, image files and documents show up in the gallery and documents app respectively.

ACTUAL RESULT:

Only files already present up to a certain point in time are displayed.

[dmz]

You’re not alone .

I experience the same problem on a XA2 for some time now (at least since 3.4.0.24). New media (e.g. music files and cover art copied to the SD card, photos shot using the camera) won’t show up in the respective apps any more after it has been added. Re-indexing via “Settings”/“Utilities”/“Clear Tracker database” only helps for so long.

However, I noticed that the “Media” app starts “re-calculating” the number of files/albums/artists etc. whenever I add a new album. Unfortunately, once it’s done, the new files aren’t accessible via the app (as they don’t show up in search, lists etc.).

I did some more research using the “tracker info” command and indeed when a new file is added,
some metadata ends up in the data base but less than what’s reported for a file that’s “seen” by the “Media” app (e.g. all “nmm:”-metadada is missing).

BTW: the number of files (MP3s, JPEGs, documents) on my device that may be eligible for indexing is close to 17.000.

[rozgwi]

If anyone could advise how: I’d like to collect some logs fron tracker and see if I can provide more info for reproducing the issue.

Thanks for any hints!

[dmz]

For me, the problem persists in 4.0.1. After having forced a complete rescan of my music library, a number of albums are still missing. For some other files, the meta data is partially unavailable (e.g. artist and duration of some MP3s are absent - even when they were present before the last scan (!)).

After digging into some documentation about tracker, I wonder if the state of the tracker-store is supposed to be “Unavailable”:

[nemo@Sailfish ~]$ tracker daemon
Store:
28 Feb 2021, 20:01:11:  ✗     Store                - Unavailable

Miners:
28 Feb 2021, 20:01:11:    1%  File System          - Crawl finished for directory 'file:///home/nemo/Downloads/mail_attachments/6/2436-2' 
28 Feb 2021, 20:01:11:  ✗     Extractor            - Not running or is a disabled plug

If I start tracker-store manually, it will quit after a few seconds:
[nemo@Sailfish ~]$ /usr/libexec/tracker-store &

Forcing a rescan of a directory that is missing from the data base doesn’t appear to be working at all:
[nemo@Sailfish ~]$ tracker reset -f /run/media/nemo/495C-4E0B/Music/Some Artist/Some Album/
(data base at /home/nemo/.cache/tracker/meta.db will not be updated)

Any hints?

[nephros]

If that path is a real one you will need to escape or quote the spaces.

[dmz]

I sure did. (But unfortunately decided just before submitting my post to use a path as an example that I just made up instead of a real one). So in fact, the path I submitted looked like:
[nemo@Sailfish ~]$ tracker reset -f /run/media/nemo/495C-4E0B/Music/Some\ Artist/Some\ Album/

But thanks for pointing this out.

Anyway, my main concern is why the data base of my music collection is incomplete when using the “official” way to force a reload (via Sailfish’s “Utilities” tool), not the question how I can work around this problem.

[inta]

This got worse to the point photos and music is not usable at all. Since 4.2 photos and music on an encrypted sdcard are never completely indexed. Sometimes only a few hundred files show up, sometimes thousand, sometimes a few thousand but never all of them. Rebooting does not help and resetting the media database from the settings does neither.

[JoOppen]

I had some success with stopping and restarting the tracker:

stop with tracker daemon -k in the terminal and
restart with tracker daemon -s
Eventually all files were found

[TMavica]

It happened when I unlock the sdcard mostly, systemctl --user restart tracker-miner-fs will fix the issue

[inta]

Thank you @JoOppen and @TMavica at least one of the two suggestions did help. I had no time for testing yet, so I just entered both at once and after a while all pictures appeared in the gallery.

Edit: tracker daemon -k && tracker daemon -s did actually work

[inta]

Maybe this bug is fixed with 4.3 (early access), after the last reboot new photos show up immediately in the gallery.

[jovirkku]

Good to hear. At least, Tracker (the indexer) was upgraded from version 2.3.6 to 3.1.2.

[meegouser]

I ran into the same issue today with my Xperia XA2 (with 4.3.0.12), when a picture (and a subsequent screenshot) wouldn’t get added to the gallery after receiving it via Bluetooth. I was able to (at least temporarily) reset and re-ignite the indexing with the Sailfish Utilities “reset button” and a phone restart, however when I originally tried to use the aforementioned Terminal commands (also as documented in the official Wiki: https://jolla.zendesk.com/hc/en-us/articles/360006687453-Clearing-the-media-database), I only get the reply bash: tracker: not found, regardless if I’m devel-su’ed or not. I’m pretty certain I’ve typed this (rather simple) command in correctly, so what is differentiating my XA2 from (as far as I could see) everyone else, that could use the commands without problems?

Edit: I just saw that in the 4.3.0 changelog there’s an entry about “Tracker3” now being in use, did this change the tracker “handle”, possibly?

[jovirkku]

@meegouser

Edit: I just saw that in the 4.3.0 changelog there’s an entry about “Tracker3” now being in use, did this change the tracker “handle”, possibly?

Yes, you guessed that right. The command is now (from OS release 4.3.0 onwards):

tracker3 reset -s

This document is up to date now. Sorry that we forgot to do that earlier.

[direc85]

@TMavica

It happened when I unlock the sdcard mostly, systemctl --user restart tracker-miner-fs will fix the issue

At least with Sailfish 4.3, the correct command is

systemctl --user restart tracker-miner-fs-3

Still experimenting how to refresh one directory only, I keep getting timeouts with tracker3 reset -f /home/defaultuser/Pictures/Screenshots…

[dmz]

It seems that this issue has finally been resolved in 4.4.0 (which included an update of tracker to version 3.2.1, according to the Changelog)

New audio files show up quickly in the “Media” app, as one would expect.