There is an odd quirk with the Android SDK: while the source code is Apache-2.0 licensed, free software, the binaries that Google ships are not free software and even put substantial restrictions on the apps that are built with the Google binaries. @beuc has been leading the effort to make actual free software binaries of the Android SDK. The Debian Android Tools Team has been working to package the Android SDK Tools as part of Debian.
There has been a great discussion on the FSFE Android mailing list about doing free software rebuilds of all the Android SDK components where we have outlined what is left to be done to actually ship this binaries to app developers who care about free software.
I’m proposing to use the F-Droid resources to both run a builder, which is easy now that we have the gcc cfarm machines. Then I also propose hosting the results in an sdkmanager repo on mirror.f-droid.org as the official source of this repo so that it can be easily mirrored elsewhere.
If we can get those builds to be reproducible, that will make it much easier to manage building the official binaries since we won’t need to have some super locked down builder machine.
This effort definitely needs help, here are some specific things where people can jump in:
quite hard to even find the git tags that Google uses to make their builds
write the scripts to make the sdkmanager repos from our binaries
get sdkmanager packaged in Debian
creating an Ansible automation to create the buildserver for these machines
building all versions of the NDK based on the existing build scripts
create build script for the Android Support libraries
setup maven repo for all of the free software libraries
scripted setup to automatically publish diffoscope output comparing our builds to Google’s official builds
JFYI, ReplicantOS team did SDK builds but gave up for some reason (there’s a relevant bug in their redmine). Yet I’m sure you still can find people with that experience.
Replicant shifted focus off SDK/NDK because they thought Debian would do it - but providing several clean Debian package sets of complete SDK platforms turned out quite hard and time consuming, so possibly they could revise their position and help with this project
On other news while perusing git history I stumbled on the sdk_repo goal modifier in the SDK build system to create separate packages as well as XML package lists.
And also found out how to build system images for alternate architectures e.g. make PRODUCT-sdk_arm64-sdk.
More doc at https://android-rebuilds.beuc.net/SDK/
Two steps closer to drop-in rebuilds
Ok, so we can put this repo on mirror.f-droid.org without a problem. Before copying your existing binaries, I’d like to try a quick test of whether they are reproducible. It would be nice to launch this repo with reproducible builds.
If you mean reproducible as in https://reproducible-builds.org/ I highly doubt so, but independent binaries would still be nice.
If you intend to use the CI rules you added, beware that these differ from the normal build system (typically: not the same working directory, not the same UID, etc.). I was thinking of pushing android-rebuilds:* base images somewhere at dockerhub so they could be grabbed by the CI and so we could properly (re)build everything using the same recipes.
Something like this Docker possibly.
I didn’t expect to have to use a non-dash prefix (androidrebuilds/android-rebuilds:sdk-XXX) OTL
I’m open to better naming schemes.
I think we’re getting side-tracked here
We’ve been discussing this additional disk space for nearly 2 month, went from “create a sdkmanager” repo to “rebuild locally” to “rebuild reproducibly”, but I still need disk space
There’s a chance in 1 000 000 that the SDK buids reproductibly even with disorderfs, and from experience debugging a monolithic build for reproducibly issues takes an eternity and then some. You can hack the android-rebuilds Makefile locally to try and install disorderfs just for checking (I don’t know how precisely) but let’s do that aside from this particular sub-project.
Right now I think we need to focus on:
setting up the storage area
check how to provide a sdkmanager repo, and how to combine the bits of generated XML from the rebuilds
unify the CI / non-CI builds because right now there’s duplication and discrepancies between Dockerfile+Makefile and the .gitlab-ci.yml.
The disk space is there already! I didn’t know you were ready to
transfer binaries. How would you like to do that? I could set up a
dedicated account, then could SSH/rsync the binaries to it. You’ll have
to send me an SSH public key for that.
The sdkmanager XML format is easy. We can just use the Google ones as
an example. When I did it before, it was quicker to hand edit it rather
than even script it.
Ok, so now there is a android-rebuilds VM on one of our GCC boxes to provide somewhere to work. Now the question is: how should the repository look on the mirror box? It doesn’t yet have a webserver, but that’ll take me 15 minutes to setup.
I think we might actually want to find a better than android or android-rebuilds for the root name there. Perhaps android-sdk? But maybe just android works.
“SDK” may be too limited since we also target the NDK, the Gradle plugin and Studio. NDK and Studio may be downloaded separately but the Gradle plugin is distributed through a repo as well AFAIK.
android-free-dev (or android-freedev to avoid dashitis? ;)) then.
liberated → as opaque?
dfsg-free → somewhat redundant, debian-centric
Google uses plain “android” pretty ambiguously (“Android is open source”…) so we probably want to be a little more precise (“dev”), unless there’s a reason to stay compatible with their naming scheme (I wouldn’t know, I don’t use their repos ;))
