I found out that ollama has arm64 binaries: Release v0.1.42 · ollama/ollama · GitHub
So I thought its worth a try.
I downloaded the binary, ran chmod +x Downloads/ollama-linux-arm64 and Downloads/ollama-linux-arm64 serve and then you can actually run models in another terminal session. (Downloads/ollama-linux-arm64 run llama3 "tell the history of linux on mobile phones")
I tried llama3, its quite slow, maybe a word every 15 seconds on my 10iii, but it works 
How much memory does it take? I’m guessing OOM problems aren’t exactly helped by doing that right?
A 4.7 GB download… 
Didnt check, but used the phone while it was genrating and it wasnt worse than else
Yes, also didnt check where it is downloaded to. Might also be worth checking the other models available with ollama
Trying phi3 right now, on my desktop.
And here’s your answer from phi3… 
$ ollama run phi3 “tell the history of linux on mobile phones”
pulling manifest
pulling b26e6713dc74… 100% ▕█████████████████████████████▏ 2.4 GB
pulling fa8235e5b48f… 100% ▕█████████████████████████████▏ 1.1 KB
pulling 542b217f179c… 100% ▕█████████████████████████████▏ 148 B
pulling 8dde1baf1db0… 100% ▕█████████████████████████████▏ 78 B
pulling f91db7a2deb9… 100% ▕█████████████████████████████▏ 485 B
verifying sha256 digest
writing manifest
removing any unused layers
success
Linux has a unique and evolving relationship with mobile phones, which began in
earnest with the advent of smartphones. Its journey into this market is marked by both
challenges and successes, as open-source developers aimed to provide alternatives to
proprietary operating systems like Android (based on Google’s own version of Linux).
Here’s an overview of Linux’s history within mobile phones:
Early Beginnings: PDAs & Palm OS/Linux
Before smartphones became ubiquitous, there were Personal Digital Assistants (PDAs),
which served as early forms of mobile computing devices. In the late 1980s and 1990s,
Linux was used to develop alternative operating systems for PDAs. One example is Palm
OS, an object-oriented real-time operating system designed for synchronizing personal
information with a handheld device.
Smartphones & Android: The Dominance of Proprietary Systems
The first smartphone was the IBM Simon Personal Communicator, released in 1994.
However, it wasn’t until the release of Apple’s iPhone and its subsequent operating
system iOS that mobile computing became widely popular. At this time, Android started
to emerge as a significant Linux-based alternative with an open-source kernel called
“Linux for Mobile.”
Android: Opening Up Possibilities
In 2005, Google acquired the company Android Inc., which was developing its own mobile
operating system. In 2007, they launched Android as a Linux distribution tailored for
smartphones and tablets, with open-source components that allowed manufacturers to
customize it according to their preferences while maintaining compatibility across
devices.
Linux Distributions in Mobile Devices: Post-Android Era
While Android dominates the mobile market today, other Linux distributions have also
made their way into smartphones and tablets, albeit less prevalently:
Challenges & Future Prospects
Linux on mobile phones has faced challenges due to the dominance of proprietary
operating systems like Android and iOS, which benefit from significant investments in
user-friendly interfaces, developer ecosystems, and strong brand identities. However,
open-source communities continue to innovate and explore new possibilities for Linux
distributions on mobile devices.
As smartphones evolve with the proliferation of IoT (Internet of Things) devices and
wearable technology, it’s likely that more customizable, secure, and privacy-focused
options powered by open-source systems such as Linux may find their place in this
expanding market.
I thought I should try the smallest model ( moondream) on my X10III, but I can’t even get through the model download. The binary (ollama-linux-arm64) constantly crashes after a few MB, saying it is searching for a better connection. Is there a speed requirement for the connection?
Living on the country side my 4G connection isn’t much to brag about. 
Likely, the post will be used to train another version of ollama, chat-gpt and others.
You could try to download from huggingface and just place it where ollama expects it (is it this one? moondream/moondream2-gguf at main)
I think it is.
The model may generate offensive, inappropriate, or hurtful content if it is prompted to do so.
“a tiny vision language model that kicks ass and runs anywhere”
Here is ollama built on device (XIII) https://files.catbox.moe/6c0l6n.gz maybe it will be less crashy? Seems to continue downloads at least for me
Thanks! I’ll take a look at it tomorrow. My spare time is out for today.
I had no problems with crashes and smooth downloads, sounds like a connection issue
Uh, nice. I wonder if there are a few easy to get speedups by building ollama for sfos and setting a few configs
Yes it is a connection issue. Also @throwaway69’s build is complaining about it and crash.
Well, I’ve known for a long time that I should re-flash my X10III, but as it is my one and only daily drive with some critical apps, I am waiting for the X10V image or the C2, whichever comes first.
So no AI on my phone for now.
Btw… utilities restart worked fine for me.
A bit OT, but I just bought another 32 GB RAM stick for my PC, to play with these things. I doubt it, but maybe I’ll find some real use for it.
Has someone tried the new llama 3.2 1B or 3B?
I see pretty decent results for such a small models…
