Syncthing-Fork reproducible build - work in progress

Hi,

now that we have a “green light” at Syncthing-Fork Reproducibility Status since v1.29.7.4, I’ve noticed the reproducibility of the build is currently only taking place within F-Droid’s build infrastructure.

Why are the build output APKs different? At first glance, the size (has always been over years and still) is different:

  • 56,4 MB (fdroid release build)
  • vs. 57,3 MB (non-fdroid release build)

Comparing the build processes, I see the go version is different between the builds.

The gradle and NDK as I understood are already taken from the git repo and thus are identical. The GO version is not specified in the build recipe which I think causes the differences in the build output.

Excerpt of the F-Droid build recipe, note the “golang-go”.

- versionName: 1.29.7.4
    versionCode: 1290704
    commit: 9ee947b33298886c5688c53abd1b8dd02ac8f083
    subdir: app
    submodules: true
    sudo:
      - apt-get update
      - apt-get install -y autogen automake autopoint bzip2 g++ libc-dev make gettext
        libtool pkg-config rename shtool
      - apt-get install -y -t bookworm-backports golang-go
    gradle:
      - yes
    rm:
      - syncthing/src/github.com/syncthing/syncthing/lib/model/testdata
    prebuild:
      - sed -i -e '/signingConfig/,+2d' build.gradle.kts
      - sed -i -e 's/java.net.URI/uri/' ../settings.gradle.kts
    build:
      - export ndkversion=$(grep "ndkVersionShared" ../build.gradle.kts | cut -d '"'
        -f 4)
      - sdkmanager "ndk;$ndkversion"
      - export ANDROID_NDK_HOME=$$SDK$$/ndk/$ndkversion

What is the easiest way to get the f-droid build container set-up using the version derived from here? syncthing-android/syncthing/build-syncthing.py at 4af19d0bc012787b20ddf9b750d4311debaff947 · Catfriend1/syncthing-android · GitHub

syncthing/build-syncthing.py

GO_VERSION = '1.24.1'

Or is it okay, if I open a merge request which removes the “golang-go” apt package pre-install completely. The “syncthing/build-syncthing.py” will then fall-back to install the appropriate go package if not found on PATH. See “install_go” for that (syncthing-android/syncthing/build-syncthing.py at 4af19d0bc012787b20ddf9b750d4311debaff947 · Catfriend1/syncthing-android · GitHub).

Kind regards,
Catfriend1

===

About CI reproducible builds, I’ve also opened a topic on the Syncthing Forum: Detect CI reproducible builds and behave accordingly? - Android - Syncthing Community Forum

I’ve also tried to omit “rm syncthing/src/github.com/syncthing/syncthing/lib/model/testdata” which was only applied to F-Droid builds in the past. But it only saved a few bytes by started marking “vcs.modified = true” (instead of false) in “libsyncthingnative.so/buildinfo” due to compression. That proves to me, that the “testdata” isn’t included in builds by syncthing(native’s) build.go script.

Size comparison:

ref: metadata/com.github.catfriend1.syncthingandroid.yml · master · F-Droid / Data · GitLab

    rm:
      - syncthing/src/github.com/syncthing/syncthing/lib/model/testdata

leads to:

*** Building for arm64
	 go version go1.23.5 linux/amd64
	 v1.29.7-dirty
	 Notice: Next generation GUI will not be built; see --with-next-gen-gui.

Suggestion: As this doesn’t save space in the built apk, can we leave “model/testdata” so the submodule “syncthing/syncthing” checkout will stay pristine?

MR openend at: Update com.github.catfriend1.syncthingandroid.yml: testdata is not compiled in (!24168) · Merge requests · F-Droid / Data · GitLab

Install golang from Debian

Add srclibs go@someversion

Extract exact version from your code

Checkout go at exact version

Build that exact version

Use that build binaries for the rest

1 Like
...
    sudo:
      - apt-get update
      - apt-get install -t bookworm-backports -y golang-go
      - apt-get install -y gcc libc-dev
...
    srclibs:
      - go@go1.23.0
    prebuild:
      - export goVersion=$(grep GO_VERSION syncthing/build-syncthing.py | cut -d " " -f2)
      - '[[ $goVersion ]] || exit 1'
      - git -C $$go$$ checkout -f go$goVersion
...
    build:
      - pushd $$go$$/src
      - ./make.bash
      - popd
      - export GOPATH=$$go$$
      - export PATH="$GOPATH/bin:$PATH"
...

just an example, did not test grep and etc :slight_smile:

we need a version in srclibs, that will stay like that forever, as we checkout the right one later on

1 Like

@Licaon_Kter Would that be also an acceptable way to grep “ENV GO_VERSION=1.24.1” from ( syncthing-android/docker/Dockerfile at main · Catfriend1/syncthing-android · GitHub ) and execute the same code when the fdroid container build is initialized?

# Install Go
RUN wget -nv https://dl.google.com/go/go${GO_VERSION}.linux-amd64.tar.gz && \
    tar -zxf go${GO_VERSION}.linux-amd64.tar.gz && \
    rm go${GO_VERSION}.linux-amd64.tar.gz
ENV GOROOT=/opt/go
ENV PATH=/opt/go/bin:${PATH}

If this doesn’t violate any policy, I’d prefer the shorter code which should run faster in one step.

don’t care, grep from wherever you need to :slight_smile: just get the exact version

1 Like

thanks, I think I can try this via pipeline/CI opening a MR… let’s see if fdroid can build this :-).

MR opened based on @Licaon_Kter’s suggestion and pseudocode:

@Licaon_Kter Ok, so far grep’ing and building the go 1.24.1 version works as I can see from the pipeline output.

It says…

*** You need to add /home/vagrant/build/srclib/go/bin to your PATH.

and then the path is not updated, so the gradle build + “go build.go” subprocess doesn’t get the exported var (we’ve had this a few days before regarding the BUILD_HOST var,… same problem).

It says:

[python] python3 -u ./build-syncthing.py
	 git_bin='/usr/bin/git'
	 go_bin='/usr/bin/go'
	 ANDROID_NDK_HOME='/opt/android-sdk/ndk/28.0.13004108'
	 Invoking git fetch ...
	 Invoking git describe ...
	 Building syncthing version v1.29.7
	 SOURCE_DATE_EPOCH=[1748505848]
	 
	 *** Building for arm
	 go version go1.23.5 linux/amd64
	 go: downloading sigs.k8s.io/yaml v1.4.0

What is the correct solution to this in order to get the fresh-built go 1.24.1 on the PATH of the builder? I think checking the hardcoded path “/home/vagrant/build/srclib/go/bin” is a bad idea. :man_shrugging:

Edit: good ideas on the MR :slight_smile: . Let’s use “gradle: no”.

Let’s discuss this in one place, there :slight_smile:

1 Like

Thank you all who were involved today to help me on improving the reproducible build :clap::slightly_smiling_face:!

I’ve downloaded the unsigned APK from the fdroid pipeline and compared it to my debian WSL instance and both builds are 100% “byte-wise” identical since we use the same Android SDK, GO 1.24.1, NDK r28.0.13004108, git checkout 6e3783867abfefa3c2629b877168618023ccaab4 and signingConfigs = null.

image

Is there now a chance, we could switch to developer signed APK as it is mentioned in the F-Droid docs? I know this would require users to reinstall, but we maybe could do it together with an AppID change to “com.syncthingfork”.

Reason behind this is, I’m currently working together with someone at grapheneOs forums and we try to publish “Syncthing-Fork” via “Accrescent app store”. Accrescent requires the validation of the authorship. Therefore, I’ve to change my appId according to their docs, and use a real domain (syncthingfork.com) which their system will automatically check for “permission” during the publishing process. I wouldn’t have so many build flavors and branches then if we could also get F-Droid updated to use the developer signature instead of its own APK signature. Accrescent also publishes including the developer signature instead of their own.

Sidenote: The build is reproducible as long as Linux-and-Linux is compared. Linux-and-Windows produces different bytecode, e.g. some elements within “libSyncthingnative.so” are different by CRLF vs. LF, caused by the NDK itself.

New app is…new app :slight_smile:

Users already jumped ship 6 months ago. We will announce in in weekly posts etc. Maybe you can even release a last version that popups and says: “switch now etc”"

Okay, I’ll note that procedure for the time when the appId change needs to be done.

Until then, I’m fine with the published app as it is now the same as the github CI release and kept up2date on fdroid as well :slightly_smiling_face:.

@Licaon_Kter Just fyi: Syncthing (core) now sets an empty buildId in their build script for release builds to ease reproducible builds for everyone.

F-Droid is so inspiring :grin:, I like it =).

ref: build: unset build ID in generated binaries by Catfriend1 · Pull Request #10203 · syncthing/syncthing · GitHub