I am one of the maintainers of Project ARTist (GitHub) and we were recently asked whether we could include our app in F-Droid. While everything we use should be open source already, I am not sure if our app is eligible for inclusion in the store because of some complicated dependencies.
As a quick intro, ARTist is an instrumentation framework for Android apps that utilizes the new Android Runtime compiler dex2oat to re-compile and modify applications. We utilize it through our ArtistGui app that ships with and wraps our modified version of the dex2oat compiler to make it accessible through a user interface. Root is required for the whole process. If you know Xposed, ARTist has a somewhat similar goal.
Alright, so having this said, the guidelines for what is allowed for F-Droid apps state that:
The software should not download additional executable binary files (e.g. non-free addons, auto-updates, etc).
But this is exactly what we are doing/planning to do. We create our custom compiler from AOSP source code components, which carry an open source license, but still building and including them is not only complicated but it also seems to violate this constraint. And since those components are hardware and OS dependent, we would like to have an app that automatically downloads the correct native code bundle.
So before investing time in preparing our app for inclusion in the F-Droid store, I would like to ask for opinions on whether this would be allowed/wanted? I pretty much like the idea of having our app in the store but for me it depends on how strictly the above mentioned guidelines are enforced. Any feedback is appreciated.
While we have scripts to automate building them and putting them into the APK, the process actually has to happen from within a completely built AOSP version for a particular hardware arch & OS version. So it happens outside the app building process and a script is just baking the results into the final APK. Eventually, we want to get rid of shipping those binaries with the app (see this issue) and download ready-made binary packages from our GitHub repo.
So to answer your question, yes they end up in the APK and they are currently a part of the build process. It is just not the typical app build script.
How much of the AOSP tree do you need to run the build? It seems totally feasible to build your compiler from source in fdroid. We already run some really large builds, like VLC, LibreOffice, Fennec, etc. I don’t think it makes sense for the f-droid.org repo to allow apps that download and run arbitrary binaries.
I’m also part of the Debian Android Tools Team, so I’m familiar with building chunks of the SDK. An alternate approach would be to help us package up any of the AOSP components that you need, and get that included in Debian. I think we would allow using binaries from Debian/main since those are also vetted for being 100% free software.
The easy way to get started with your app in F-Droid it so make your own fdroid repo with your own APKs. There are no restrictions on what you put in your own fdroid repo. Izzysoft and MicroG are two examples of fdroid repos like that. Another thing you might be interested in is that you can distribute any kind of file in an fdroid repo, so you could also distribute your dex2oat compiler that way.
Concerning AOSP: Essentially we are building the art AOSP project plus some libraries that even out differences between the actual device and our AOSP build, hence by having a look at the build files we could find out what parts of AOSP are actually needed. However, my educated guess from working on this for some time is that quite a lot of the native AOSP projects are needed. So if we plan for building the compiler for the f-droid. org repo, we should plan with building major parts of AOSP.
Concerning alternatives: I think creating an own repository is a nice way to get started and get some experience while working towards getting ARTist into the f-droid. org repo eventually. Since you mentioned that repos are not only providing apps… distributing ARTist via f-droid might be an option to make the whole process a bit easier. What do you think about the following setup: We have ArtistGui and the ARTist compiler as dedicated entries. ArtistGui is just the app that is build from source (only Java built by gradle) and ARTist is only the binary compiler that is however built from AOSP. Then, at runtime, ArtistGui may download and use ARTist versions IF it downloads it from f-droid.org, right? B/c in this scenario, everything executed is built from source and from the f-droid. org repo.
I think the separate ArtistGui and the ARTistCompiler entries makes
sense. It could help with the transition to f-droid.org since the
ARTistCompiler binaries could be fetched from your repo while ArtistGui
is in f-droid.org. You’ll probably need to walk users through adding
the repo and installing the package. Both can be done with Intents.
Also, Major parts of AOSP are already built and included in Debian. So
what you need might already be there:
Great idea in general, but in this case the ARTist compiler is still provided as a binary from an “untrusted” source (from f-droid. org’s perspective). Or is an own f-droid based repo somehow considered more trustworthy? B/c if we just ship ArtistGui without the capability to auto-download ARTist compiler versions but still require the user to install them in some other way, isn’t that just resembling the idea of loading and executing “untrusted” code again? Sorry for re-iterating on this, I just want to make sure we are all on the same page about what the code is and does to avoid problems later.
Oh, great! I wasn’t aware… Guess we are looking into that in the near future, if most of the dependencies are already satisfied this might be a great alternative way to build ARTist without requiring a fully-built AOSP tree.
Great idea in general, but in this case the ARTist compiler is still provided as a binary from an “untrusted” source (from f-droid. org’s perspective). Or is an own f-droid based repo somehow considered more trustworthy? B/c if we just ship ArtistGui without the capability to auto-download ARTist compiler versions but still require the user to install them in some other way, isn’t that just resembling the idea of loading and executing “untrusted” code again? Sorry for re-iterating on this, I just want to make sure we are all on the same page about what the code is and does to avoid problems later.
Yes it is, but it could be an acceptable migration pattern.
Ok, I see. Thank you a lot for sharing your experience. We will probably start by investigating the dedicated repo idea for delivering ArtistGui and ARTist binaries independently and then eventually move towards a delivery model that is acceptable for f-droid.org as well.
Since this answers my question how to get started with the inclusion of ARTist in f-droid.org, I will mark one of your posts as the solution. Thanks to @Licaon_Kter and @hans for participating.
Unlocked rooted devices are a rarity let’s be honest (and the likes of Huawei removing a way to unlock the bootloader worses even more the situation), so either those users are XDA familiars and can install everything themselves (really you need the privileged extension and you are on your way) or non-rootable users that can disable Google stuff* and use F-Droid (press button to update app…is that so horrible?)
No need for rooted, only fastboot mode access to flash system.img; for Mediatek mid-range devices, Treble is real revolution : without modification to their (unpublished/GPL…) kernel and vendor, users have now up-to-date cve /system devices free of ads & analytics.
The misses are µG, unbloated/webview (Firebase), f-droid and others useful patches that allows non-rooted device/kernel to works fluently (conversely to missing Mediatek ota).
Also some floss addons like /bin/aapt, Exchange2 (or other specifics aosp tree) could be built in and then single-ly (?) added to f-droid repo (“~similar” to here).