don’t know if this is a terminal issue, but it is happening because the terminal has no idea that there is UTF-8 filename. It is using (obviously) ASCII.
try setting the terminal to appropriate locale if possible.
Confirmed. Anything beyond finnish non-ASCII characters gets replaced by question marks in the directory entries, regardless of obvious shell i18n settings. This is on 3.4.0.24, so it’s not a regression from this version.
It’s not dependent on terminal emulator: I can observe the same effect using ssh connection. (Actually, if you look carefully, you can see Cyrillic letters in the screenshot of the terminal emulator in a message above, just not in ls output.)
Running ls | od -c shows that ls indeed outputs ? signs instead of Cyrillic letters. However, bash (from gnu-bash package) shows Cyrillic letters without problems, in command line and in output of echo *.
On the other hand, Busybox shell replaces Cyrillic letters with ? signs when trying to input them in command line editor, but still preserves them: entering echo привет results in command line looking like echo ??????, but the output is still привет. And echo * in Busybox shell shows Cyrillic letters in filenames correctly.
As /bin/ls is a symlink to /usr/bin/busybox (through busybox-symlinks-coreutils), this looks to me like an issue in Busybox (or libraries it uses).
But I also can confirm that ls was replacing Cyrillic letters with ? signs in filenames in Sailfish 3 as well (up to 3.4), so it can be also a similar issue in coreutils package (or, again, in libraries it uses).
Should presence of locales matter? As long as a character is a valid UTF-8 sequence, it should be printable, no matter what locale is used.
For exampe, in Fedora I can type LANG=C.UTF-8 ls, and it shows me all filenames with Cyrillic letters correctly. On the other hand, LANG=C.UTF-8 busybox ls in Fedora replaces Cyrillic letters in filenames with ? signs. Coincidence? I don’t think so.
I think so (man locale) and what is the output of locale -m.
When I setup LC_ALL to i.e. bg_BG.UTF-8 and do locale --help I see it in the appropriate language. Same with ru_RU.UTF-8.
In my opinion it is a problem in “ls” - bug?!
I saved a perl script and it displays the dir listing also properly
opendir(FH, '.') || die "Can't opendir: $!";
while (readdir FH) { print "$_\n"; }
closedir FH;
I’ve installed GNU Coreutils instead of Busybox ones: zypper in gnu-coreutils, then select option 1 (remove conflicting busybox-symlinks-coreutils package). And now ls shows filenames with Cyrillic letters correctly! So, yes, this is obviously a bug (or missing functionality) in Busybox.
I’ve turned on my old Jolla-1 phone that still has Sailfish 3. It has the same problem with ls not showing Cyrillic letters in filenames. But what I did not know is that /bin/ls on Sailfish 3 is also a symlink to /usr/bin/busybox through busybox-symlinks-coreutils package. (For some reason I had a false impression that Sailfish has regular GNU Coreutils, not Busybox ones.)
But if you’re stuck with Sailfish 3 on your Jolla-1 phone, you can still install GNU Coreutils: zypper in coreutils (please note that the package in Sailfish 3 is called coreutils, not gnu-coreutils and has much older version than the one from Sailfish 4, presumably to remain under GPLv2 license).
As a sidenote, let me express a great pleasure about Jolla finally allowing up-to-date GNU packages under GPLv3 license into Sailfish 4 (although not pre-unstalled, but still available for installation using zypper).
