Offline language translator with Speech Note 4.0

Ahoy Sailors!

Just want to share a few words about an update I’ve made for Speech Note 4.0 app.

New version comes with Offline Translator which surprisingly works quite well for many languages! You can translate full-text between Catalan, Bulgarian, Czech, Danish, English, Spanish, German, Estonian, French, Italian, Polish, Portuguese, Norwegian, Iranian, Dutch, Russian, Ukrainian and Icelandic. To do so, Internet is not needed. It works, as everything in Speech Note, totally offline.

Speech Note might be useful if you, for some reason, don’t want to share your private data with online services or just happen to be in place with limited access to the Internet or, most importantly, want to be prepared for the inevitable apocalypse of our civilization.

Translator is built with code from amazing Bergamot Project and language models from Firefox Translations.

Short video presentation:
https://www.youtube.com/watch?v=88cdPpvBmmI

Live speech translation with me trying to speak Spanish using Speech Note + Papago:
https://www.youtube.com/watch?v=nTCwWwko7lU

37 Likes

thankyou, will give it a try.

Thank you mkiol for your efforts!
Any tips which STT and TTS models to use on armv7-devices?

2 Likes

Thanks so much, I love your speech apps.

I have one small (?) request though: harbour-dsnote is now probably the largest package for SFOS at ~150MB. Would it be possible to relocate the large files somewhere under /home/.system or even /home/<current_user>?

2 Likes

Can we have it in the official store please?

It is indeed awfully big. Sorry for that. The problem is that I need to ship a bunch of heavy python libraries needed to do only one task ‘Restore punctuation’. I didn’t find so far any good C/C++ library for this… You have opened my eyes. Yes, I could install those libraries to /home/.system! Thanks for the tip :slight_smile:

3 Likes

Everything depends for which language. For English, ‘Vosk Small’ (STT) and any Piper TTS model (my fav is ‘Jenny’ voice).

I really would like to do that but it is not possible right now due to official store restrictions. Unfortunately, Speech Note do few things which are not allowed in Harbour. Sorry :frowning:

OK i get it. In case it is API related there was a discussion a while ago about adding even more apis.

Maybe we can start the discussion again AND ask Jolla politely (or buy them a beer or two :stuck_out_tongue:).

1 Like

Please help with setting up Speech Note correctly. Right now have installed latest version from Storeman. The huge amount of options and possible settings puzzles me!

Would like to use german, spanish and english. Are there some suggestions for an adequate setup for lower CPU power? It should work on Xperia 10, unfortunately no ‘stronger’ device is available.

Thanks for any hint!

Unfortunately, I don’t own an X10, so I’m not 100% sure how powerful this device is. Nevertheless, I would recommend using the default settings and downloading the following models from the “Languages and models” menu:

English:

  • Speech to Text:
    • Vosk Small
    • WhisperCpp FUTO English-39
  • Text to Speech:
    • Any Piper voice
  • Translator:
    • English to German
    • English to Spanish

Deutsch:

  • Speech to Text:
    • Vosk Small
    • WhisperCpp Futo Multilingual-39
  • Text to Speech:
    • Any Piper voice

Espanol:

  • Speech to Text:
    • Vosk Small
    • WhisperCpp Futo Multilingual-39
  • Text to Speech:
    • Any Piper voice

In general, Whisper offers better accuracy and punctuation marks, but it might be slow on X10. The Vosk model is lighter, but has poorer accuracy and does not support punctuation marks.

4 Likes

Setup went fine, results are funny!
Thanks @mkiol !

I wanted to give it a try. Download was simple. Install worked.

Then it asked me for permissions.
Essentially access to every aspect of everything.
Includes location data, NFC, Networks, Media index, Camera, Internet, Bluetooth, all user directories, …
Total access.
Listening with the camera, submitting stuff via internet pretending not to do exactly this, filing data records local using networks of all kind, bluetooth not to miss Elon on Mars, NFC is probably to cure cancer and location service is to prevent child porn.
Joking!?

NO THANKs, not my device!
This is either hammering stupid or a targeted privacy breach siphoning my data.
A cosy Langley,VA device ?!
NO, THANKS!

The permissions dialog is confusing but it doesn’t actually access those things. The reason all of those permissions are requested is because the application hasn’t defined which defined which permissions it needs, and Sailfish has no mechanisms to enable permissions later on, so it has to request them all when you first launch the application to make sure the application works. You’ll see this sometimes with older applications, if they weren’t updated to define the permissions they actually need, Sailfish will ask you to give them all permissions, even if it’s just a ‘Hello World’ application. It doesn’t mean the application is actually doing that. The permissions dialog is definitely something that could use improvements in the future to make this clearer, although ideally the application would define the permissions it needs so the ‘Give me access to everything’ dialog doesn’t need to pop up.

3 Likes

The reason is exactly as @nthn explained. Speech Note currently has no defined permissions. The main reason is its integration with Speech Keyboard. Speech Note needs unrestricted access to the system keyboard to handle “speech to text” requests triggered from the system keyboard. Unfortunately, there is currently no defined SFOS permission that would allow communication between apps, so the only solution was to disable “sandboxing” completely. This is also the reason why I couldn’t submit the Speech Note to the official Jolla App Store - it is available only in Storeman.

4 Likes