Camera2 API development

hmm…

[thilo@waschmaschine git]$ ls -lah build/envsetup.sh
lrwxrwxrwx 1 thilo thilo 16 27. Sep  11:39 build/envsetup.sh -> make/envsetup.sh
[thilo@waschmaschine git]$ ls make
ls: cannot access 'make': No such file or directory
[thilo@waschmaschine git]$ ls build/make/

EDIT:
looking at this my initial repo sync probably failed…

I’ll try again from the start… (with repo sync

Did you fix your build setup?

Just as I said before, I am lucky to already have a such build environment, since I already built the HADK bits.

I imagine that to only make droidmedia one would “just” need to:

  • have the manifests for the specific device. I don’t think vendor or kernel matter, so it might just work with the AOSP manifests of the specific adaptation, say, Android 10. Find out by devel-su and grepping /system/build.prop for ro.system.build.version.release
  • use repo command, which needs to be installed separately, to sync the sources. Might require skipping submodules.
  • use source build/envsetup.sh or just the . for source
  • checkout droidmedia using git clone in external/droidmedia and make sure you’re on camera2 branch
  • breakfast $yourdevice or, in the case of AOSP, find a codename that is not emulated (maybe aosp_arm64-eng will work). Otherwise, $yourdevice is the codename you fetched the manifests for.
  • export PORT_ARCH="aarch64"
  • make droidmedia. That should find the makefile wherever so even in external/droidmedia. And should output some libdroidmedia.so, figure it out where from the output. (it should be out/target/product/$yourdevice/system/lib64/libdroidmedia.so)
  • scp the libdroidmedia.so to the device. Use devel-su to overwrite it in /usr/libexec/droid-hybris/system/lib64/libdroidmedia.so.
  • If the above fails zypper in -f droidmedia to have back your system
  • If you want to see logging, use /usr/libexec/droid-hybris/system/bin/logcat while operating the camera. ALOGI used today may not output, you can tryi # setprop log.tag INFO or just change the source to ALOGE

Warning: this is written away from my build system, take it with a grain of salt and duckduckgo it.

4 Likes

I actually fixed it today. I had to delete the folder and refetch the repositories. Make ran and then i ran out of time, but I built the .so :slight_smile:

5 Likes

I tried compiling for Xperia 10 III following the instructions above and on docs.sailfishos.org

make droidmedia
20:48:01 Build sandboxing disabled due to nsjail error.
============================================
PLATFORM_VERSION_CODENAME=REL
PLATFORM_VERSION=11
TARGET_PRODUCT=aosp_xqbt52
TARGET_BUILD_VARIANT=user
TARGET_BUILD_TYPE=release
TARGET_ARCH=arm64
TARGET_ARCH_VARIANT=armv8-2a
TARGET_CPU_VARIANT=generic
TARGET_2ND_ARCH=arm
TARGET_2ND_ARCH_VARIANT=armv8-2a
TARGET_2ND_CPU_VARIANT=generic
HOST_ARCH=x86_64
HOST_2ND_ARCH=x86
HOST_OS=linux
HOST_OS_EXTRA=Linux-6.2.0-34-generic-x86_64-Ubuntu-20.04.2-LTS
HOST_CROSS_OS=windows
HOST_CROSS_ARCH=x86
HOST_CROSS_2ND_ARCH=x86_64
HOST_BUILD_TYPE=release
BUILD_ID=RQ3A.211001.001
OUT_DIR=out
PRODUCT_SOONG_NAMESPACES=device/sony/common device/sony/lena vendor/qcom/opensource/core-utils vendor/qcom/opensource/display/sm8250 vendor/qcom/opensource/display-commonsys-intf/sm8250
============================================
external/droidmedia/Android.mk was modified, regenerating...
[ 23% 56/239] including external/droidmedia/Android.mk ...
external/droidmedia/Android.mk:48: warning: *** ANDROID_MICRO undefined. Assuming 0
20:48:23 Disallowed PATH tool "xmllint" used: []string{"xmllint", "--xpath", "string(/device[@name=\"Android\"]/item[@name=\"battery.capacity\"])", "device/sony/pdx213/overlay/frameworks/base/core/res/res/xml/power_profile.xml"}
20:48:23 See https://android.googlesource.com/platform/build/+/master/Changes.md#PATH_Tools for more information.
[ 50% 121/239] including hybris/hybris-boot/Android.mk ...
hybris/hybris-boot/Android.mk:71: warning: ********************* /boot appears to live on /dev/block/bootdevice/by-name/boot
hybris/hybris-boot/Android.mk:72: warning: ********************* /data appears to live on /dev/block/bootdevice/by-name/userdata
"xmllint" is not allowed to be used. See https://android.googlesource.com/platform/build/+/master/Changes.md#PATH_Tools for more information.
Could not determine android architecture. Please pass it as 2nd argument using gettargetarch.
/bin/bash: external/audioflingerglue/detect_build_targets.sh: No such file or directory
[ 51% 123/239] including hybris/mw/libhybris/libhybris/compat/hwc2/Android.mk ...
hybris/mw/libhybris/libhybris/compat/hwc2/Android.mk:8: warning: *** ANDROID_VERSION_MINOR undefined. Assuming 0
hybris/mw/libhybris/libhybris/compat/hwc2/Android.mk:13: warning: *** ANDROID_VERSION_MICRO undefined. Assuming 0
[ 51% 124/239] including hybris/mw/libhybris/libhybris/compat/input/Android.mk ...
/bin/bash: line 0: test: too many arguments
ninja: no work to do.

#### build completed successfully (38 seconds) ####

but I cannot find the droidmedia.so file:

ls out/target/product/pdx213/system/lib64/
android.frameworks.bufferhub@1.0.so                  android.hardware.graphics.common@1.1.so    android.hidl.token@1.0-utils.so  libbufferhubqueue.so   libfmq.so                liblogwrap.so                libsurfaceflinger.so
android.frameworks.vr.composer@1.0.so                android.hardware.graphics.common@1.2.so    android.hidl.token@1.0.so        libc++.so              libfs_mgr.so             liblp.so                     libsync.so
android.frameworks.vr.composer@2.0.so                android.hardware.graphics.composer@2.1.so  ld-android.so                    libcgrouprc.so         libgralloctypes.so       liblzma.so                   libtimestats.so
android.hardware.boot@1.0.so                         android.hardware.graphics.composer@2.2.so  libEGL.so                        libcrypto.so           libgraphicsenv.so        libnativebridge_lazy.so      libtimestats_proto.so
android.hardware.boot@1.1.so                         android.hardware.graphics.composer@2.3.so  libGLESv1_CM.so                  libcrypto_utils.so     libgsi.so                libnativeloader_lazy.so      libtinyxml2.so
android.hardware.common-V1-ndk_platform.so           android.hardware.graphics.composer@2.4.so  libGLESv2.so                     libcutils.so           libgui.so                libnativewindow.so           libui.so
android.hardware.configstore-utils.so                android.hardware.graphics.mapper@2.0.so    libSurfaceFlingerProp.so         libdexfile_support.so  libhardware.so           libpackagelistparser.so      libunwindstack.so
android.hardware.configstore@1.0.so                  android.hardware.graphics.mapper@2.1.so    libadbd_auth.so                  libext2_blkid.so       libhidl-gen-utils.so     libpcre2.so                  libutils.so
android.hardware.configstore@1.1.so                  android.hardware.graphics.mapper@3.0.so    libadbd_fs.so                    libext2_com_err.so     libhidlbase.so           libpdx_default_transport.so  libvintf.so
android.hardware.graphics.allocator@2.0.so           android.hardware.graphics.mapper@4.0.so    libandroid_runtime_lazy.so       libext2_e2p.so         libhidltransport.so      libprocessgroup.so           libvndksupport.so
android.hardware.graphics.allocator@3.0.so           android.hardware.media@1.0.so              libbacktrace.so                  libext2_misc.so        libhwbinder.so           libprotobuf-cpp-lite.so      libz.so
android.hardware.graphics.allocator@4.0.so           android.hardware.power-V1-cpp.so           libbase.so                       libext2_quota.so       libhwc2_compat_layer.so  libprotoutil.so
android.hardware.graphics.bufferqueue@1.0.so         android.hardware.power@1.0.so              libbinder.so                     libext2_uuid.so        libinput.so              libselinux.so
android.hardware.graphics.bufferqueue@2.0.so         android.hardware.power@1.1.so              libbinder_ndk.so                 libext2fs.so           libkeyutils.so           libsparse.so
android.hardware.graphics.common-V1-ndk_platform.so  android.hardware.power@1.2.so              libbootloader_message.so         libext4_utils.so       liblayers_proto.so       libsquashfs_utils.so
android.hardware.graphics.common@1.0.so              android.hardware.power@1.3.so              libbufferhub.so                  libfec.so              liblog.so                libstatslog.so

Any ideas?

(If I need to clean and rerun droidmedia, how would I do this? I have been switching between master and camera2 to trigger partial rebuilds)

I tried with make clean when i switched the branch. And i think libdroidmedia is the correct make target.

For the xa2 i needed additionally libncurses5, python2 and LC_ALL=C is required. (that is at least as far as i got). I am currently trying to build a dockerfile that includes all the requirements i needed.

3 Likes

I got the libdroidmedia.so now for the XA2:

build with this script in ./makelib

#!/bin/bash
# have jdk8 and the repo tool from android installed
export VENDOR=sony
export DEVICE=h4113
export HABUILD_DEVICE=pioneer
export FAMILY=nile
export BRANCH=8.1.0_r52
export HAVERSION="hybris-sony-aosp-"$BRANCH"_20190206"
export PORT_ARCH=aarch64
export LC_ALL=C

cd build
source build/envsetup.sh
export USE_CCACHE=1
lunch aosp_h4113-eng
make -j$(nproc --all) droidmedia

on this docker image in docker/Dockerfile

FROM python:2.7-buster
ENTRYPOINT [ "/bin/bash" ]
RUN apt-get update && apt-get install -y \
  software-properties-common  build-essential gcc-multilib make ccache libncurses5 \
  && wget -qO - https://adoptopenjdk.jfrog.io/adoptopenjdk/api/gpg/key/public | apt-key add - \
  && add-apt-repository --yes https://adoptopenjdk.jfrog.io/adoptopenjdk/deb/ \
  && apt-get update && apt-get install -y adoptopenjdk-8-hotspot \
  && rm -rf /var/cache/apk/*

Instructions

  • Setup the repositories with repo and clone droidmedia into build/
# have jdk8 and the repo tool from android installed
cd build
export VENDOR=sony
export DEVICE=h4113
export HABUILD_DEVICE=pioneer
export FAMILY=nile
export BRANCH=8.1.0_r52
export HAVERSION="hybris-sony-aosp-"$BRANCH"_20190206"
export PORT_ARCH=aarch64
repo init -u https://github.com/mer-hybris/android.git -b $HAVERSION -m tagged-manifest.xml
repo sync -c --fetch-submodules
git clone https://github.com/mer-hybris/droid-src-sony-nile droid-src
git clone https://github.com/sailfishos/droidmedia external/droidmedia
  • write the build script to makelib and the dockerfile to docker/
  • run docker build docker/ and get the image hash
  • run docker run -it -v(pwd):/camera2api -w /camera2api <imagehash> to get into the image
  • execute bash makelib
  • find libdroidmedia in build/out/target/product/pioneer/system/lib/ (or maybe lib64 if you are on another device)
  • if this works run cd build && make clean && cd ..
  • now you can switch the branch for droidmedia and rebuild with the camera2 api
  • deploy it to /usr/libexec/droid-hybris/system/lib/libdroidmedia.so on your phone (maybe lib64 on other devices.make sure you do not mix lib and lib64 :wink:
6 Likes

I don’t why make droidmedia did not include the libdroidmedia target but now I was able to successfully build the lib. Thanks @thigg !

I can confirm the inverted viewfinder colors, looks fancy actually. Photos have correct color and are taken as fast as in OpenCamera :+1:

@mal A list with known open todos would be super useful or maybe even an issue for each. Feel free to dump a list here and have the community turn it into proper issues. I assume we then open PRs against the camera2 branch and merge to master once ready?

5 Likes

The inverted color issue sounds like a wrong pixel format, but if it was just that, it would already be corrected… Let’s hope that’s not a firmware issue that needs to be worked around…

I’m excited about the progress! I wish I had more hours in a day so I could participate in this, but unfortunately I don’t…

3 Likes

We can certainly create new issues on the droidmedia project, maybe prefix them all with [Camera2]. What we miss is something of a milestone or label created for that, but that’s details.

We have esr91 milestone in Issues · sailfishos/sailfish-browser · GitHub and that looks very clear to me. Maybe having milestone “camera2” for each camera2 related issue. Some food for thought.

2 Likes

Hi @rainemak, I fully agree that a milestone/label with “camera2” would help, however I don’t think any of us can create it. Can you or @mal create this and we’ll just use it? Thanks!

3 Likes

Hi @vlagged ! Created camera2 label (for colored issues) and milestone

Please use flag both camera2 label and milestone when going forward.

8 Likes

So, if released, Camera API 2.0 will affect all SailfishOS-Units also the Ports?

1 Like

A small status update on Camera2 support. I managed to find a fix for the viewfinder color issue. I also have almost working video recording support which does record the video and I can see the first frame of the video when playing it but because of some issue with frame timestamps all later frames are skipped. These changes are not yet in the camera2 branch because I need to cleanup the code again after the a bit ugly code that was created during debugging of the issues.

As for the question in previous comment: current plan is that Camera2 support would be available for all devices which are built on Android 7 or newer base. Community ports need to have their adaptation packages updated to get the support so schedule of that depends on each porter when they will do that after the support is finalized and merged.

48 Likes

I just tried the camera2 branch on my xperia 10-iii. The nice thing is, that camera usage does not interfere anymore with android. I can use android and sfos applications using the camera and it just works :slight_smile:

although the quality seems to be better in sfos…

22 Likes

by any chance, does anyone have camera2 libdroidmedia.so built for 10iii?
i would love to not have OpenCamera and jolla-camera crash each other all the time, but not quiiite enough to undertake that build procedure

Unfortunately none of us had time to work on it the last few months so it is still not super functional (no video, no torch, no creative modes, strange exposure settings, …)

3 Likes

thanks. i know it’s not ready, and i was still interested in trying out the camera2 lib, to test it out, and see if it fixed the crashes in android OpenCamera.

however, after reading the issue you wrote more carefully, i see that the torch doesn’t work even outside the camera app, and you’re correct, i rely on the torch too much to use it as-is.

anyway, thanks so much for the work you folks have been doing on this!

no commits since november. has this hit an insurmountable block, or is everyone just too busy on 4.6/etc?

1 Like

As i understood it it was a private effort and not done by jolla, but maybe that changed now

2 Likes