[Help wanted] How to create a reproducible build (FairEmail)

M66B · February 27, 2020, 4:25pm

So, at least R8 is being used and can be removed from the list of potential problems.

I don’t see ‘ Transform artifact … ’ in my build.

You can find a complete transcript of a build session here:

https://gist.github.com/M66B/d15d2d2ec5d4fed51aad0bcad109833f

Licaon_Kter · February 27, 2020, 4:26pm

See my edited post above…

I think it’s because I can’t build assembleGithubRelease and can only build assembleFullRelease…which I did…that’s the APK…

So as expected…it’s very different…

M66B · February 27, 2020, 4:30pm

‘full’ is an incomplete specification. What needs to be built is build type ‘GitHub’, so only signature V1 will be used (which was what changed in version 1.992 besides specifying the exact NDK version) and product flavor ‘full’ to use the right build config (links, etc).

See here:

github.com

M66B/FairEmail/blob/master/app/build.gradle

apply plugin: 'com.android.application'
apply plugin: 'com.bugsnag.android.gradle'
apply plugin: 'kotlin-android'
apply plugin: 'de.undercouch.download'

def getVersionCode = { -> return 2113 }
def getRevision = { -> return "a" }
def getReleaseName = { -> return "Vallibonavenatrix" }
// https://en.wikipedia.org/wiki/List_of_dinosaur_genera

def keystoreProperties = new Properties()
keystoreProperties.load(new FileInputStream(rootProject.file("keystore.properties")))

Properties localProperties = new Properties()
if (rootProject.file("local.properties").exists())
    localProperties.load(new FileInputStream(rootProject.file("local.properties")))

android {
    //compileSdkExtension 4 // https://developer.android.com/guide/sdk-extensions
    namespace 'eu.faircode.email'

This file has been truncated. show original

Licaon_Kter · February 27, 2020, 4:30pm

Already looking, and I’ll try to bypass the gradle logic and build like that

M66B · February 27, 2020, 4:34pm

‘release’ has been split into ‘play’ and ‘github’ to be able to use different signing configs.

So, ‘release’ does not exist anymore.

Note that ‘github’ is meant to be the GitHub version and a reproducible F-Droid built.

Licaon_Kter · February 27, 2020, 4:52pm

If I put gradle: yes it builds both Full and Play, as releases, so Github can’t be build…no release.

If I omit gradle: if tries the old build system

DEBUG: > /home/strech/android/tools/android update lib-project -p .
Error: . is not a valid project (AndroidManifest.xml not found).
ERROR: Could not build app eu.faircode.email: Failed to update project at .

M66B · February 27, 2020, 5:04pm

Are you sure you are using the right version with the right gradle file?

‘release ’ does not exist anymore …

Also, ‘full’ is a build flavor and ‘play’ is a build type.

Licaon_Kter · February 27, 2020, 5:20pm

I’m not using files directly, nor commits, fdroid pulls the Tag, that means I can’t be sure, but I trust git.

It wasn’t plain play but play_release (as the flavour name) iirc, I was in a hurry.

When you don’t specify a flavour, it builds them all.

Again, the issue is that we (atm) can’t build a non-existing release.

M66B · February 27, 2020, 5:28pm

Do I understand correctly that we need to wait until F-Droid has built version 1.992?

Licaon_Kter · February 27, 2020, 5:41pm

Yes and no.

Yes, F-Droid builds to verify, and users will get an update, but you can have the inbuild autoupdate/your fdroidrepo update the app before we verify it.

The trick is that being signed twice users can update from multiple places.

hans · February 28, 2020, 8:05am

it is saying that these files are different:

AndroidManifest.xml
classes.dex
classes2.dex

So the next step is to find those differences, and what is causing them.

M66B · February 28, 2020, 8:19am

We know that these files are different, but why?
Do you have any suggestion about how to find out what is causing the difference?

Licaon_Kter · February 28, 2020, 8:27am

This is not the problem ^^^ ?

Licaon_Kter · February 28, 2020, 8:31am

@hans How do I bypass Gradle (see post 46 [Help wanted] How to create a reproducible build (FairEmail) - #46 by Licaon_Kter ) so I can build a non-release?

Licaon_Kter · February 28, 2020, 9:06am

@M66B In the mean time, reenabled autoupdates: FairEmail - update 1.993, enable AUM (!6402) · Merge requests · F-Droid / Data · GitLab

hans · February 28, 2020, 9:33am

check the F-Droid docs, like the Reproducible Builds page, for more
info. It will likely require you dig into things to figure out why.
reproducible-builds.org also has useful resources.

Licaon_Kter · February 28, 2020, 9:48am

I meant in F-Droid…

M66B · March 3, 2020, 7:13pm

@Licaon_Kter

`* What went wrong:
Execution failed for task ‘:app:stripFullReleaseDebugSymbols’.

Requested NDK version 21.0.6113669 did not match the version 12.1.2977051 requested by ndk.dir at /home/vagrant/android-ndk/r12b

Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.
Get more help at https://help.gradle.org

BUILD FAILED in 1m 34s`

https://f-droid.org/wiki/page/eu.faircode.email/lastbuild_993

Using NDK 12 while I use NDK 21 will likely not result in a reproducible build …

Also, NDK 12 is very old (June 2016) and should really not be used anymore.

M66B · March 3, 2020, 7:14pm

I have tried to figure this out on my own, but I have zero experience with reproducible builds, so I need help on this. What should I as developer do to make this happen?

hans · March 3, 2020, 7:29pm

first, find the diffs between builds. diffoscope is the tool for
that. You can just run the build twice on your own machine to start
with. Then take it from there.