[Help wanted] How to create a reproducible build (FairEmail)

1

I like to make the build of FairEmail reproducible (with developer’s signature, mine).

There is some documentation about reproducible builds here:

but it is not completely clear what should actually be done to get a reproducible build.

For example, it is unclear which NDK version F-Droid uses.

Preferably the documentation should be updated with steps to accomplish this.

@hans @Licaon_Kter

1 Like
2

Which one you want: Build Metadata Reference | F-Droid - Free and Open Source Android App Repository

ndk: <version>

Version of the NDK to use in this build. Defaults to the latest NDK release that included legacy toolchains (12b), so as to not break builds that require toolchains no longer included in current versions of the NDK.
The buildserver supports r9b with its legacy toolchains, r10e, r11c r12b them all, and the latest release as of writing this document, r21. You may add support for more versions by adding them to ’ndk_paths’ in your config file.

3

@Licaon_Kter can you please add

ndk: r21

This should also be added or linked from in the reproducible build documentation.

What else should be done?

4

Fixed now, yes: Update docs about ndk versions supported (!501) · Merge requests · F-Droid / Website · GitLab

Umm what exactly? The metadata syntax in general?

1 Like
5

Since the NDK version is important for a reproducible build, the documentation should tell that the NDK version should be specified and link to the available NDK versions.

6

Is important for all apps? Maybe, I dunno

That’s on a case by case imho (eg. no ndk needed here metadata/de.schildbach.oeffi.yml · master · F-Droid / Data · GitLab or here: metadata/rs.ltt.android.yml · master · F-Droid / Data · GitLab )

But that’s already settled when your app is added to F-Droid, in the metadata, no? Eg. metadata/eu.faircode.netguard.yml · master · F-Droid / Data · GitLab

Looking at email metadata/eu.faircode.email.yml · master · F-Droid / Data · GitLab are you saying that this builds with r12b on F-Droid ( since it’s the default see remove ndk default 12b (#717) · Issues · F-Droid / fdroidserver · GitLab ) but you build with r21 (or whatever AS uses as latest) ?

7

It is important to all apps wanting a reproducible build using native code.

I am using NDK r21, so the .yml file of FairEmail should be updated.

If I understand things correctly a ‘Binaries’ line needs to be added to the .yml file too.

8

Yes, after we build the unsigned APK is compared to yours (signed with v1 only for the moment) that you/Github hosts.

Also: Separate "dev signature retrieval method" from "wether F-Droid signs too or not" (#711) · Issues · F-Droid / fdroidserver · GitLab

@Bubu can add more info

9

The place to start is following the rebuilds on our rebuilder. You can
see some eu.faircode APKs have been reproducible, though not FailEmail:
https://verification.f-droid.org/verified.json
https://verification.f-droid.org/verified.html

You can see all the build/repro attempts here:
https://verification.f-droid.org/

Then try rebuilding the APK on your machine, using fdroid build and
then use diffoscope to compare it to the F-Droid APK. Then fix the
differences. Once the APK is reliably reproducible, then you can
consider also using the upstream developer signature, which requires
reproducible builds.

10

Then try rebuilding the APK on your machine

Or post here and I run test builds… so you don’t waste time setting up :slight_smile:

11

@hans can you please take a look at:

https://verification.f-droid.org/eu.faircode.email_914.apk.diffoscope.html

and maybe tell what is causing the differences?

12

that one looks like a diffoscope misconfiguration, since it is showing a
diff of binary data

13

Can this misconfiguration be fixed?

14

I have made the following changes:

Explicitly specify NDK:

Signature V1 for GitHub/F-Droid build:

@Licaon_Kter both changes likely require changes in the .yml file

15

@Licaon_Kter based on what I know so far, a reproducible build should be possible with version 1.992

16

I’m testing now… yes

17

I imagine so. Someone needs to dig into it to figure it out. That can
happen anywhere. My guess is that diffoscope is mislabeling one of the
APKs as somethung else, like a plain JAR or plain ZIP, then doing a
binary comparison. APKs are both valid JARs and valid ZIPs.

18

First try…

BUILD SUCCESSFUL in 37s
52 actionable tasks: 51 executed, 1 up-to-date
INFO: Successfully built version 1.992 of eu.faircode.email
DEBUG: Using androguard from "/usr/lib/python3/dist-packages/androguard/__init__.py"
DEBUG: Checking build/eu.faircode.email/app/build/outputs/apk/full/release/FairEmail-v1.992-full-release-unsigned.apk
INFO: ...retrieving https://github.com/M66B/FairEmail/releases/download/1.992/FairEmail-v1.992-full-github.apk
DEBUG: Starting new HTTPS connection (1): github.com
DEBUG: https://github.com:443 "GET /M66B/FairEmail/releases/download/1.992/FairEmail-v1.992-full-github.apk HTTP/1.1" 302 638
DEBUG: Starting new HTTPS connection (1): github-production-release-asset-2e65be.s3.amazonaws.com
DEBUG: https://github-production-release-asset-2e65be.s3.amazonaws.com:443 "GET /143298715/d8194c80-596f-11ea-803c-7e7046b0d821?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20200227%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20200227T143504Z&X-Amz-Expires=300&X-Amz-Signature=7d46a678f70be8fa38631003323cbf396d3babfa33fd0776fc927ffc49070582&X-Amz-SignedHeaders=host&actor_id=0&response-content-disposition=attachment%3B%20filename%3DFairEmail-v1.992-full-github.apk&response-content-type=application%2Fvnd.android.package-archive HTTP/1.1" 200 12174168
WARNING: Ignoring META-INF/MANIFEST.MF from unsigned/eu.faircode.email_992.apk
DOES NOT VERIFY
ERROR: SHA-256 digest of AndroidManifest.xml does not match the digest specified in META-INF/MANIFEST.MF. Expected: <oTOZfOh8OUkJZrrW3ByzKfy/zl1Rb2Ywg4l3PYpHrwQ=>, actual: <kSPZ+GnUbxRRZw0EvDerKqqEx9jebp9GJ8MVQ/jBjDA=>
ERROR: SHA-256 digest of classes.dex does not match the digest specified in META-INF/MANIFEST.MF. Expected: <IypiPzw/bBi5loZvI01Po2n++AYrLWfo5FSMz7knjWM=>, actual: <asXmVEQg4NzX6H/YLniWSTUgARHS5FRjGP3OrwUiX64=>
ERROR: SHA-256 digest of classes2.dex does not match the digest specified in META-INF/MANIFEST.MF. Expected: <VP6C33A0DM40xWvoMP/5VuN5l2xJuS9gnu//uhzzpQA=>, actual: <B/3V7v91IF5/q0+a+ighqdP8PIbx/U9EdwLgCJKN7Ts=>
ERROR:
/tmp/tmp_s1n3aci/sigcp_eu.faircode.email_992.apk:
INFO: ...NOT verified - /tmp/tmp_s1n3aci/sigcp_eu.faircode.email_992.apk
DEBUG: > diff -r /tmp/tmp_s1n3aci/unsigned_binaries_eu.faircode.email_992.binary /tmp/tmp_s1n3aci/_tmp_tmp_s1n3aci_sigcp_eu.faircode.email_992
DEBUG: removing unsigned/eu.faircode.email_992.apk
DEBUG: removing unsigned/binaries/eu.faircode.email_992.binary.apk
ERROR: Could not build app eu.faircode.email: compared built binary to supplied reference binary but failed
==== detail begin ====
Unexpected diff output:
Binary files /tmp/tmp_s1n3aci/unsigned_binaries_eu.faircode.email_992.binary/content/AndroidManifest.xml and /tmp/tmp_s1n3aci/_tmp_tmp_s1n3aci_sigcp_eu.faircode.email_992/content/AndroidManifest.xml differ
Binary files /tmp/tmp_s1n3aci/unsigned_binaries_eu.faircode.email_992.binary/content/classes2.dex and /tmp/tmp_s1n3aci/_tmp_tmp_s1n3aci_sigcp_eu.faircode.email_992/content/classes2.dex differ
Binary files /tmp/tmp_s1n3aci/unsigned_binaries_eu.faircode.email_992.binary/content/classes.dex and /tmp/tmp_s1n3aci/_tmp_tmp_s1n3aci_sigcp_eu.faircode.email_992/content/classes.dex differ
==== detail end ====
INFO: Finished
INFO: 1 build failed

I’m comparing with the Github published one, that’s what you want @M66B right?

Metadata now:

Categories:
  - Internet
License: GPL-3.0-only
AuthorName: Marcel Bokhorst (M66B)
WebSite: https://email.faircode.eu/
SourceCode: https://github.com/M66B/FairEmail
IssueTracker: https://forum.xda-developers.com/android/apps-games/source-email-t3824168
Translation: https://crowdin.com/project/open-source-email
Changelog: https://github.com/M66B/FairEmail/releases
Donate: https://email.faircode.eu/pro/
Bitcoin: 13nUbfsLUzK9Sr7ZJgDRHNR91BJMuDuJnf

AutoName: FairEmail

RepoType: git
Repo: https://github.com/M66B/FairEmail
Binaries: https://github.com/M66B/FairEmail/releases/download/%v/FairEmail-v%v-full-github.apk

Builds:
...
  - versionName: '1.992'
    versionCode: 992
    commit: '1.992'
    subdir: app
    submodules: true
    gradle:
      - full
    prebuild: sed -i -e '/keystoreProperties/d' build.gradle
    ndk: r21

AutoUpdateMode: Version %v
UpdateCheckMode: Tags
CurrentVersion: '1.992'
CurrentVersionCode: 992
19

Oh maybe disorderfs needs to be added…brb

20

Yes, that is what version 1.992 was released for.

Since the manifest and classes differ, I am wondering if the same source code is being used. In any case the are no native libraries different, which is good.