Index for media files & documents not updating (Xperia10 II, 4.1.0.23)

Bug Reports
,
#1

REPRODUCIBILITY (% or how often): 100
BUILD ID = OS VERSION (Settings > About product): 4.1.0.23
HARDWARE (XA2, Xperia 10…): Xperia 10 ii dual
UI LANGUAGE: English
REGRESSION: (compared to previous public release: Yes, No, ?): yes
DESCRIPTION:
Index for media files & documents not updating.
After mount a encrypted sd-card, the tracker daemon reindex stuck.
tracker daemon
Miners:
16 May 2021, 12:20:12: 25% File System - Processing… 02m 57s remaining

there is no process that shows an activity („ps -ef“ and „top“)

10 minutes after the mount of the SD-Card the tracker directory ~/.cache/tracker shows no activity
2228228 drwxr-xr-x 2 defaultu defaultu 4096 May 16 12:11 .
2228227 drwxrwx— 19 defaultu defaultu 4096 May 16 11:04 …
2228484 -rw-r–r-- 1 defaultu defaultu 10 May 16 12:11 db-locale.txt
2228373 -rw-r–r-- 1 defaultu defaultu 2 May 16 12:11 db-version.txt
2230780 -rw-r–r-- 1 defaultu defaultu 5 May 16 12:11 first-index.txt
2230779 -rw-r–r-- 1 defaultu defaultu 10 May 16 12:11 last-crawl.txt
2228381 -rw-r–r-- 1 defaultu defaultu 6410240 May 16 12:16 meta.db
2228767 -rw-r–r-- 1 defaultu defaultu 98304 May 16 12:16 meta.db-shm
2228758 -rw-r–r-- 1 defaultu defaultu 10240000 May 16 12:16 meta.db-wal
2230776 -rw-r–r-- 1 defaultu defaultu 363086 May 16 12:11 ontologies.gvdb
2233828 -rw-r–r-- 1 defaultu defaultu 1 May 15 22:09 parser-version.txt

PRECONDITIONS:
STEPS TO REPRODUCE:

  1. Reset Tracker with „tracker reset -e“
  2. Wait till tracker reindex is finish (without SD-Card)
    ([defaultuser@X10II-p1rat tracker]$ tracker daemon
    Store:
    16 May 2021, 12:12:08: 0% Store - Idle
    Miners:
    16 May 2021, 12:12:08: ✓ File System - Idle
    16 May 2021, 12:12:08: ✓ Extractor - Idle)
  3. mount encrypted SD-Card

EXPECTED RESULT: Index for media files & documents finish
ACTUAL RESULT: stuck in the middle of nowwhere :slight_smile:
ADDITIONAL INFORMATION:
[defaultuser@X10II-p1rat tracker]$ tracker daemon
Store:
16 May 2021, 12:16:11: 0% Store - Idle
Miners:
16 May 2021, 12:16:11: 19% File System - Processing… 04m 05s remaining
16 May 2021, 12:16:11: ✗ Extractor - Not running or is a disabled plugin
[defaultuser@X10II-p1rat tracker]$ tracker daemon
Store:
16 May 2021, 12:16:12: 0% Store - Idle
Miners:
16 May 2021, 12:16:12: 25% File System - Processing… 02m 57s remaining
16 May 2021, 12:16:12: ✗ Extractor - Not running or is a disabled plugin

[defaultuser@X10II-p1rat tracker]$ tracker status
Currently indexed: 1857 files, 357 folders
Remaining space on database partition: 99.7 GB (97.64%)
Data is still being indexed: Estimated 02m 57s left

[defaultuser@X10II-p1rat tracker]$ tracker daemon
Store:
16 May 2021, 12:18:03: 0% Store - Idle
Miners:
16 May 2021, 12:18:03: 25% File System - Processing… 02m 57s remaining
16 May 2021, 12:18:03: ✗ Extractor - Not running or is a disabled plugin

Community meeting on IRC 26th May 2022
Tracker stop working periodically
#2

Am I the only one who has this problem?
Additional Information:
Is it perhaps due to the size of the SD card?
Mine SD-Card is 128GB in size.

#3

Problably not: Index for media files & documents not updating

However, for me it currently works most of the time (XA2 / 128 GB SD card) as I did not have to reset the media data base for the last 2 months or so.

#4

I didn’t have that problem on my old XA2 either.
Only since I got the new Xperia 10 II with fresh installed version
4.1.0.23 indexing no longer works :frowning:

#5

I have encountered similar issues on X10 II but I have 32 Gb SD-card that isn’t encrypted. It seems that some photos & images are not indexed as there are usually thousands of photos that are missing on my side. It also seems that if I start re-indexing them on Utilities it loses everything and gets stuck somewhere. I don’t have any logs however.

#6

Happened to me as well with an unencrypted 64GB card, so I just moved everything over to the main memory and now it works. Of course that isn’t a solution to the problem.

#7

Agree, that is not the solution, to copy all the files from SD-Card to internal memory.

#8

@vige, @jovirkku: could you tell me, is this a bug and is this in your internal bug tracker as well?

1 Like
#9

I have the same problem as well. Anyone knows how to debug tracker properly?

#10

Okay, I found the offending sparql query, that makes tracker crash and burn for me. For me tracker dies with the following debug messages:

(tracker-miner-fs:1197): Tracker-CRITICAL **: 14:36:25.957: Could not execute sparql: GDBus.Error:org.freedesktop.Tracker1.SparqlError.Internal: Not a ISO 8601 date string. Allowed form is [-]CCYY-MM-DDThh:mm:ss[Z|(+|-)hh:mm]

In strace, the offending query is:

INSERT DATA {\nGRAPH <urn:uuid:472ed0cc-40ff-4e37-9c0c-062d78656540> {\n_:1 a nfo:FileDataObject , nie:InformationElement , nfo:Folder ; \n  nie:dataSource <urn:nepomuk:datasource:6b397c2211a096d778fae0a21cf10ea2> ; \n  nfo:belongsToContainer <urn:uuid:30e161e0-33ce-4ee1-8520-90ae893368e5> ; \n  nfo:fileSize 4096 ; \n  nie:isStoredAs _:1 ; \n  nfo:fileLastModified \"292278994-08-17T07:12:55.807Z\" ; \n  nfo:fileName \"With Love\" ; \n  nfo:fileLastAccessed \"2021-05-26T11:45:26Z\" ; \n  nie:mimeType \"inode/directory\" ; \n  nie:url \"file:///run/media/defaultuser/ec5e57e8-5cd2-4dd4-8a82-d0d2f274de59/Music/Christina%20Grimmie/Christina%20Grimmie/With%20Love\" ; \n  tracker:available TRUE .\n}\n};\n

As you can see, the modification timestamp is horribly wrong. So sparql rightfully chokes on it.

If I run stat from the command line manually, I get the following:

  File: /run/media/defaultuser/ec5e57e8-5cd2-4dd4-8a82-d0d2f274de59/Music/Christina Grimmie/Christina Grimmie/With Love/
  Size: 4096            Blocks: 8          IO Block: 4096   directory
Device: b341h/45889d    Inode: 9437904     Links: 2
Access: (0755/drwxr-xr-x)  Uid: (100000/defaultuser)   Gid: ( 1000/  system)
Access: 2021-05-26 13:45:26.000000000
Modify: 1970-01-01 00:00:00.000000000
Change: 2019-07-15 19:47:12.000000000

So I guess it just has no modification timestamp?

It seems like newer versions of tracker fallback to setting it to the unix epoch start: src/miners/fs/tracker-miner-files.c · master · mirror / tracker-miners · GitLab

But that is a new change: tracker-miner-fs: Fall back if no modification date is found (c6beebb1) · Commits · mirror / tracker-miners · GitLab

Maybe that causes some issues or that change isn’t in Sailfish yet?

It also explains why it only breaks on 64bit. The date is ~INT_MAX, which on 32bit has 4 digits (2038) and on 64bit has a lot more.

2 Likes
[Release notes] Verla 4.2.0
#11

thank you for your research. Very mysterious.
Sometimes a tracker reset -e works for a sucessful re-indexing.
Could you tell me how to find the debug messages?

#12

tracker -e didn’t work for me. I think this is a bug with tracker interpreting no modification date as date(INT_MAX), which won’t cause issues on 32bit, but on 64bit the year gets very big.

For debugging I ran G_MESSAGES_DEBUG=all strace -s 1024 /usr/libexec/tracker-miner-fs after killing the tracker processes with tracker daemon -k and then starting the remaining ones with tracker daemon -s.

2 Likes
#13

Seems to be fixed in SailfishOS 4.3.0:

1 Like
Community meeting on IRC 26th May 2022
#14

Thanks for highlighting that this was fixed @olf. Just to tie everything together, @jovirkku explains in the other thread that Tracker was upgraded from version 2.3.6 to 3.1.2 with the release of Suomenlinna (4.3.0), which is likely the source of the fix (also mentioned in the Suomenlinna Release Notes).

I’ve tagged this bug report as ‘fixed’. If anyone is still experiencing this, please do say.

#15

Erm, sorry, but no, I did not! I merely denoted that this “seems to be fixed in SailfishOS 4.3.0” and provided a reference for that. I have not tested anything, did you?

Hence I believe it is a bit premature to "tag this bug report as ‘fixed’ ", unless there is some proof that this is fixed (in contrast to just assume or guess that this bug is fixed).

#16

From what I understood of @KuroNeko’s nice analysis, the crash was fixed upstream by commit c6beebb1c5, which was then brought into the Sailfish OS trackier-miners repository with commit c67a65daac, which I can see was released in Sailfish OS 4.3.0.

As such, I don’t think it makes sense to create an internal bug report about this (hence the fixed tag), but I will do if anyone is still experiencing it on 4.4.0.

@p1rat, @KuroNeko: would you in a position to test this out again and confirm either way?

2 Likes
#17

The bug I experienced is fixed, yes. I guess I only posted about that in the update thread at the time…

1 Like
#18

Thanks @KuroNeko, that’s good to hear. It would still be good to get confirmation from @p1rat in case it turns out it was a different issue, but hopefully it was the same and the same fix will apply.

1 Like
#19

it was the same issue and the problem is fixed.

2 Likes
#20

Marvellous, thank you @p1rat.