Camera2 API development

[mjun]

As I don’t see anyone working on this (maybe I’m wrong?) I’ll take a stab at it and attempt to implement Camera2 API support. As a developer with 10+ years of experience how hard can it be ?

All joking aside, I have substantial experience in Java and Python but almost none in C++ so any help to get me started with setting up the project is welcome as I don’t see any guidelines how one actually develops and debugs internal components for SFOS.

So let’s get started with basic information - we have release blog for version 4.4 that states the following:

Sailfish OS hooks into the Android Camera API to access the camera firmware. The glitch exists in this version of the API, but is fixed in the Camera2 API. Switching Sailfish OS to use the Camera2 API is a large, but not impractical, amount of work. What’s more, because the code that needs changing is open source, and we’ve had a number of excellent contributions to the camera pipeline from the community in the past, we’re hoping we can work together with you i the community to get this resolved in a future release.

I have created a fork of droidmedia project and, for starters, two bits of information would be extremely helpful:

1. Where are the other (non-main) lenses disabled in SFOS?
2. Are there any guides how to setup droidmedia project for development? If I open it in Sailfish IDE i get a bunch of dependency errors.

Thanks!

[vige]

@mjun

Where are the other (non-main) lenses disabled in SFOS?

That one is easy: /vendor/etc/camera/camxoverridesettings.txt, search for multiCameraEnable.

[mal]

@mjun

As I don’t see anyone working on this (maybe I’m wrong?)

I have already made a partial implementation of Camera2 API support and plan to continue adding the missing things.

[mjun]

Great! I’ll be happy to test and/or contribute (I own Xa2, xa2 plus and 10 iii).

Are your changes (publicly) available on github?

[direc85]

Thanks for picking this up, or at least trying to! I tried to tinker around Camera2 API earlier, but came out with nothing…

Please share your work to the community, no matter how early steps, the more eyes there is to it the faster it progresses! I’m happy to help in whatever I can, with all the zero knowledge I have about porting and Android inner workings

[mal]

I need to check the code and clean it up a bit before pushing it to public, it’s just too ugly right now because of the missing parts and commented out code.

[jojo]

if you need it, i volunteer to test it on 10 ii

[Raduprv]

Hello,
Any progress with this? I would really love to use the Camera 2 API on Droidian, the Camera 1 is very basic and can’t take decent photos with it.

[direc85]

Any progress anyone could share so others could join?

Taking a photo from a moving target is pretty much impossible with X10III at the moment… Extracting frames from videos it is

[mjun]

Just letting everyone know that I have given up on this (for now). There is a multitude of reasons and I will list them here because I feel some things regarding SFOS needs to be said:

• I received numerous messages from community regarding Camera2 api development - everyone wants to test. This is all fine but literally not a single person contacted me with actual information/help how to set up dev enviroment for development and testing camera. I have spent a lot of time to make this work but I have a company to run and family that I want to spend time with and just didn’t want to countinue hacking
• There is no documention provided by Jolla regarding core system development. There is not even a README file on github. To make things work, stock camera app is closed source. Providing a link to C++ header file is not enough if you want to attract people to help out
• Looking at last couple of releases I see only minor updates or updates that are largely handled by community. Android support was updated but aside from having “bigger version number” no real improvements regarding android support have been made (as a matter of fact, a lot of android apps stopped working due to the way android app rendering was handled)
• More and more apps (that I use regularly) on android rely on having bootloader locked and this is a trend that will over time make android support less and less useful
• android support for NFC, bluetooth (other than audio), biometrics is non-existent. I regularly use apps that require biometrics (fingerprint reader) that provide no alternative

All in all, I feel that the feature gap between android and SFOS has grown so much that it is nearly impossible to catch up with the current state of things. I don’t really see any real roadmap and plan made by jolla and a lot of OS (stock apps) are still closed source which makes it very hard for community to move this OS into brighter future.

For now, I’m keeping my 10 III in drawer and using Xperia 5 IV with android as a daily driver. Until I see some significant progress made regarding HW compatibility with 10 III (echo cancellation fixed, battery usage and other reported stuff) I will not be returning to SFOS.

As a person that runs business I feel what Jolla does currently will eventually kill off SFOS completely. The only way to get back on track is to partner with companies like Volla / Gigaset, fairphone, pinephone and similar which already have a vibrant community and support and opens up ability to ship phones with locked bootloader and overall better user experience.

I feel that sony open source program is just not worth it - by the time OS gets open sourced devices are already 1 year old and hard to buy…

These are just my 2 cents on the matter, I might be wrong but given the current state of things buying sony device + SFOS licence and then have so much bugs and reliability problems is just not worth it.

[vlagged]

Hi @mjun, I understand your opinion and partially share some of the frustration.

For this subject alone, the header linked in this post is from a piece of software called droidmedia.

My understanding from the steps to build a hardware adaptation is that droidmedia is actually an Android package with hybris interface.
There are at least two clients for it that I know of: gstreamer1.0-droid plugin (which makes the droid part available for gstreamer clients) and gecko-camera (before: gmp-droid) which makes droid parts available for the gecko browser.

So basically it’s a C interface to Android software, and you partly need to be an Android C++ developer to understand how to refactor it (but who is an Android developer and uses Sailfish…? yup you guessed it.)

So basically jolla camera needs to use droidmedia one way or another to work across devices with different android bases.

My frustration comes more in line with what I once heard rinigus saying: we can’t help with Camera2 api since we’re too busy with the proprietary VoLTE “solution” (sorry, I can’t find the exact place of the quote and I might remember it incorrectly)

(And sorry to be writing so late in this thread - I’ve postponed looking into this for some time. Only now I got to the point of my zenfone port needing Camera2 API too;)

[yajo10]

Hi @mjun ,
I really appreciate the efforts you put into this. From your last post I got the intention, that you actually managed to build the droidmedia package.
I just procrastinated from learning, looked into it shortly and I totally agree with you: the documentation is not that well. I

f you finally got it working could you imagine to share some of your insights? Maybe someone else can pick up your work and go on with that?

I don’t want to bother you and really appreciate the time you spend!

[rinigus]

Re VoLTE and Camera 2: not that I had a chance to work on VoLTE either, but it is priority for me. I expect that VoLTE would be a major obstacle in some time and there is no difference on whether you have a camera or not if you cannot use your phone as a phone. But priorities for other developers could be different…

[Raduprv]

To be honest, I don’t think the camera 2 api would make a huge difference for Linux based phones. I mean, for planning to use the phone as a… phone.

However, there are many old phones that are sitting in a drawer somewhere, or end up in the garbage dump. Having access to Camera 2 API (and a way to install a working Linux distro, such as Sailfish, Ubuntu Touch, Droidian, etc.) would give many of those phones a new life. They could be used for security applications (remote security cams in areas where there is no electricity), time lapse cameras, baby monitors, and so on.

So having access to the Camera 2 API would possibly bring more people to try the non android OSes.

[teleshoes]

Xperia 10 III has the ultrawide and optical-zoom lenses disabled because there is a firmware issue in camera1 api that makes the selfie lens inaccessible. camera2 api would solve that.

plus, features like (true) manual-focus (not what adv-camera calls manual-focus) and HDR are only in camera2.

[teleshoes]

but really, i came here to say that opencamera with camera2 api supports all 4 lenses in SFOS android app support, IF you are very careful (and you enabled them in camxoverridesettings.txt).
[+] button changes main => ultrawide
[+] button again changes ultrawide => zoom
[+] button switches to WEIRD SELFIE MODE, and then any further action CRASHES the app.

so INSTEAD, to return to main camera lens, press selfie-camera twice. this works from any of the 3 back lenses.

selfie toggle button works normally, as long as youre not in the weird duplicate selfie camera mode (at which point, every action crashes the app).

if you get into the weird duplicate selfie camera, the only way i know of to make opencamera work again is to restart appsupport. restarting cameraserver no longer works as of SFOS 4.5 (although it totally DOES work in SFOS 4.4)

NOTE: opencamera in camera1 does NOT work with all 4 lenses. it does something akin to what jolla-camera does, and breaks on selfie mode. it is somewhat alarming that opencamera lens switching doesn’t work perfectly even with camera2 api, though.

[fingus]

Maybe its time to release special versions of Jolla-/advanced-Camera-App which will adress this issues

[teleshoes]

this thread is about why you cant do that. the camera api used in SFOS is camera1. jolla believes (apparently correctly) that no solution aside from implementing camera2 api will work (although i STILL see weirdness in camera2 api, it is a weirdness that can very likely CAN be managed in software once camera2 is impld)

jolla ALSO decided that its not worth their devs time, and asked the community to do it for them for free. someone valiantly tried, found that jolla makes community devel of system stuff difficulty on purpose, and gave up.

this is the sonyxperidadev bug report:
https://github.com/sonyxperiadev/bug_tracker/issues/732

[erlend]

Camera 2.0 was released with Android 5.0 in November 2014, almost 9 (!) years ago.

Its a real shame that Jolla has never found the time to implement it

[olf]

@mal

I need to check the code and clean it up a bit before pushing it to public, it’s just too ugly right now because of the missing parts and commented out code.

@mal, did you progress with cleaning up your code in the past 7,5 months? Even if not, why don’t you just publish your work in order to enable any collaboration on Camera API 2 in SailfishOS?

P.S.: We have regularly seen such outcomes as this (frustrated developer ceases working on something, because sailors did never seriously reply, only formally), which is one crucial aspect of Jolla’s failure to foster and collaborate with a developer community.