Index for media files & documents not updating (Xperia10 II, 4.1.0.23)

REPRODUCIBILITY (% or how often): 100
BUILD ID = OS VERSION (Settings > About product): 4.1.0.23
HARDWARE (XA2, Xperia 10…): Xperia 10 ii dual
UI LANGUAGE: English
REGRESSION: (compared to previous public release: Yes, No, ?): yes
DESCRIPTION:
Index for media files & documents not updating.
After mount a encrypted sd-card, the tracker daemon reindex stuck.
tracker daemon
Miners:
16 May 2021, 12:20:12: 25% File System - Processing… 02m 57s remaining

there is no process that shows an activity („ps -ef“ and „top“)

10 minutes after the mount of the SD-Card the tracker directory ~/.cache/tracker shows no activity
2228228 drwxr-xr-x 2 defaultu defaultu 4096 May 16 12:11 .
2228227 drwxrwx— 19 defaultu defaultu 4096 May 16 11:04 …
2228484 -rw-r–r-- 1 defaultu defaultu 10 May 16 12:11 db-locale.txt
2228373 -rw-r–r-- 1 defaultu defaultu 2 May 16 12:11 db-version.txt
2230780 -rw-r–r-- 1 defaultu defaultu 5 May 16 12:11 first-index.txt
2230779 -rw-r–r-- 1 defaultu defaultu 10 May 16 12:11 last-crawl.txt
2228381 -rw-r–r-- 1 defaultu defaultu 6410240 May 16 12:16 meta.db
2228767 -rw-r–r-- 1 defaultu defaultu 98304 May 16 12:16 meta.db-shm
2228758 -rw-r–r-- 1 defaultu defaultu 10240000 May 16 12:16 meta.db-wal
2230776 -rw-r–r-- 1 defaultu defaultu 363086 May 16 12:11 ontologies.gvdb
2233828 -rw-r–r-- 1 defaultu defaultu 1 May 15 22:09 parser-version.txt

PRECONDITIONS:
STEPS TO REPRODUCE:

  1. Reset Tracker with „tracker reset -e“
  2. Wait till tracker reindex is finish (without SD-Card)
    ([defaultuser@X10II-p1rat tracker]$ tracker daemon
    Store:
    16 May 2021, 12:12:08: 0% Store - Idle
    Miners:
    16 May 2021, 12:12:08: ✓ File System - Idle
    16 May 2021, 12:12:08: ✓ Extractor - Idle)
  3. mount encrypted SD-Card

EXPECTED RESULT: Index for media files & documents finish
ACTUAL RESULT: stuck in the middle of nowwhere :slight_smile:
ADDITIONAL INFORMATION:
[defaultuser@X10II-p1rat tracker] tracker daemon Store: 16 May 2021, 12:16:11: 0% Store - Idle Miners: 16 May 2021, 12:16:11: 19% File System - Processing… 04m 05s remaining 16 May 2021, 12:16:11: ✗ Extractor - Not running or is a disabled plugin [defaultuser@X10II-p1rat tracker] tracker daemon
Store:
16 May 2021, 12:16:12: 0% Store - Idle
Miners:
16 May 2021, 12:16:12: 25% File System - Processing… 02m 57s remaining
16 May 2021, 12:16:12: ✗ Extractor - Not running or is a disabled plugin

[defaultuser@X10II-p1rat tracker]$ tracker status
Currently indexed: 1857 files, 357 folders
Remaining space on database partition: 99.7 GB (97.64%)
Data is still being indexed: Estimated 02m 57s left

[defaultuser@X10II-p1rat tracker]$ tracker daemon
Store:
16 May 2021, 12:18:03: 0% Store - Idle
Miners:
16 May 2021, 12:18:03: 25% File System - Processing… 02m 57s remaining
16 May 2021, 12:18:03: ✗ Extractor - Not running or is a disabled plugin

Am I the only one who has this problem?
Additional Information:
Is it perhaps due to the size of the SD card?
Mine SD-Card is 128GB in size.

Problably not: Index for media files & documents not updating

However, for me it currently works most of the time (XA2 / 128 GB SD card) as I did not have to reset the media data base for the last 2 months or so.

I didn’t have that problem on my old XA2 either.
Only since I got the new Xperia 10 II with fresh installed version
4.1.0.23 indexing no longer works :frowning:

I have encountered similar issues on X10 II but I have 32 Gb SD-card that isn’t encrypted. It seems that some photos & images are not indexed as there are usually thousands of photos that are missing on my side. It also seems that if I start re-indexing them on Utilities it loses everything and gets stuck somewhere. I don’t have any logs however.

Happened to me as well with an unencrypted 64GB card, so I just moved everything over to the main memory and now it works. Of course that isn’t a solution to the problem.

Agree, that is not the solution, to copy all the files from SD-Card to internal memory.

@vige, @jovirkku: could you tell me, is this a bug and is this in your internal bug tracker as well?

I have the same problem as well. Anyone knows how to debug tracker properly?

Okay, I found the offending sparql query, that makes tracker crash and burn for me. For me tracker dies with the following debug messages:

(tracker-miner-fs:1197): Tracker-CRITICAL **: 14:36:25.957: Could not execute sparql: GDBus.Error:org.freedesktop.Tracker1.SparqlError.Internal: Not a ISO 8601 date string. Allowed form is [-]CCYY-MM-DDThh:mm:ss[Z|(+|-)hh:mm]

In strace, the offending query is:

INSERT DATA {\nGRAPH <urn:uuid:472ed0cc-40ff-4e37-9c0c-062d78656540> {\n_:1 a nfo:FileDataObject , nie:InformationElement , nfo:Folder ; \n  nie:dataSource <urn:nepomuk:datasource:6b397c2211a096d778fae0a21cf10ea2> ; \n  nfo:belongsToContainer <urn:uuid:30e161e0-33ce-4ee1-8520-90ae893368e5> ; \n  nfo:fileSize 4096 ; \n  nie:isStoredAs _:1 ; \n  nfo:fileLastModified \"292278994-08-17T07:12:55.807Z\" ; \n  nfo:fileName \"With Love\" ; \n  nfo:fileLastAccessed \"2021-05-26T11:45:26Z\" ; \n  nie:mimeType \"inode/directory\" ; \n  nie:url \"file:///run/media/defaultuser/ec5e57e8-5cd2-4dd4-8a82-d0d2f274de59/Music/Christina%20Grimmie/Christina%20Grimmie/With%20Love\" ; \n  tracker:available TRUE .\n}\n};\n

As you can see, the modification timestamp is horribly wrong. So sparql rightfully chokes on it.

If I run stat from the command line manually, I get the following:

  File: /run/media/defaultuser/ec5e57e8-5cd2-4dd4-8a82-d0d2f274de59/Music/Christina Grimmie/Christina Grimmie/With Love/
  Size: 4096            Blocks: 8          IO Block: 4096   directory
Device: b341h/45889d    Inode: 9437904     Links: 2
Access: (0755/drwxr-xr-x)  Uid: (100000/defaultuser)   Gid: ( 1000/  system)
Access: 2021-05-26 13:45:26.000000000
Modify: 1970-01-01 00:00:00.000000000
Change: 2019-07-15 19:47:12.000000000

So I guess it just has no modification timestamp?

It seems like newer versions of tracker fallback to setting it to the unix epoch start: src/miners/fs/tracker-miner-files.c · master · mirror / tracker-miners · GitLab

But that is a new change: tracker-miner-fs: Fall back if no modification date is found (c6beebb1) · Commits · mirror / tracker-miners · GitLab

Maybe that causes some issues or that change isn’t in Sailfish yet?

It also explains why it only breaks on 64bit. The date is ~INT_MAX, which on 32bit has 4 digits (2038) and on 64bit has a lot more.

1 Like

thank you for your research. Very mysterious.
Sometimes a tracker reset -e works for a sucessful re-indexing.
Could you tell me how to find the debug messages?

tracker -e didn’t work for me. I think this is a bug with tracker interpreting no modification date as date(INT_MAX), which won’t cause issues on 32bit, but on 64bit the year gets very big.

For debugging I ran G_MESSAGES_DEBUG=all strace -s 1024 /usr/libexec/tracker-miner-fs after killing the tracker processes with tracker daemon -k and then starting the remaining ones with tracker daemon -s.