How long for an app to be available for download once metadata are updated?

Hi,

I already noticed this minor inconvenience in the past, but once a build is successful, the new release sometimes takes several days to become available for download in the repository. For example, this build succeeded the 19th in the morning, the batch finished in the evening, but the build still isn’t available for download. If I recall correctly, this build took almost two weeks between the upstream release and the build being actually available for download.

How long does it usually take for the update to appear in the store once the build is done? Is there some caching involved? Can we do something to speed up the process? As a more general thought, I’m wondering if the “daily batch” model isn’t reaching some scaling issues. You have to wait 1 or 2 days for the new upstream versions to get caught by “checkupdates”, if a batch is currently running wait for it to finish which can sometimes take several days, wait for a new batch to complete which can also take several days to get started and again several days to finish, and then wait an unknown amount of time for the app being actually available for download. Yet you could theoretically start the build as soon as metadata are pushed and publish the app in the process. Are there any ongoing discussions about this issue? I guess that’s hard to solve as it looks like a major paradigm change.

Thanks.

3 Likes

There is a human involved. That’s why it sometimes takes time. The apps are built automatically but signing is performed on a computer that is not connected to the internet.

1 Like

Gotcha. Sounds highly suspicious, but that explains part of the issue.

However, I don’t think that explains everything: for instance Wikipedia 50343 was built twice (the 18th in the same batch as TousAntiCovid 152 and the 20th) and is available for download since yesterday (21st). In comparison, TousAntiCovid 152 was built on the 19th and still isn’t available for download. My guess is that the 19th batch was just dismissed, which already happened. Does the “super-human that nobody knows but everyone has to trust” only signs and uploads some apps and not others? How does they pick them? Is that documented somewhere?

1 Like

Sorry, I don’t know the details of how building works.

Some apps use reproducible builds. With that, you don’t have to trust the person. Other people can build and then compare the results.

1 Like

Thanks for the insights :+1:.

(Edit: just as a bottom-note, if I understand correctly, the offline computer is not about building apps but about securing signing keys. Reproducible builds don’t help in this regard. But that’s definitely out of topic.)

There is some discussion of the same topic at What triggers a build of new app versions?. See specifically my suggestion at What triggers a build of new app versions? - #22 by sorenstoutner.

In addition, this is a topic that keeps coming back up, because the current design will never meet the needs and expectations of users. See, for example, Is the F-Droid build process currently broken? from almost four years ago with over 10K views.

1 Like

Thanks for the pointers, that gives me a lot of insights.

Despite the fact that your proposal wouldn’t address all of the issues with the current build process, it seems that it would at very least solve the main bottleneck and wouldn’t require a lot of fundamental changes and rewrites. The security trade-off also looks very reasonable as it could basically be implemented by having an iptable rule that disallows any incoming traffic which greatly reduces the attack surface. On top of that, the host could easily be put in an private network that only the build server (or even better: some kind of scheduler) would have access to. Signing keys could be encrypted at rest using an external vault and access to the keys could be audited which would actually be a gigantic improvement compared to the current situation.

Unfortunately, I don’t feel like we can do much to move things forward as there doesn’t seem to be much traction for this from the maintainers.

I have the same issue with F-Droid (as a maintainer).
It took 14 days between the release of OpenTracks v3.15.0 and it’s availability on F-Droid.
Also, the new changelog was already available via F-Droid while the APK wasn’t.

Not really cool; only alternative I can think of is to create a custom F-Droid repo (also for nightly builds).

So: YES that is an issue affecting upstream maintainers.

2 Likes

If your users absolutely need your updates ASAP, then create your own repo. It is a trivial process, and has only been made easier over the years.

@SkewedZeppelin That is not an option (or not a good one).
Should now every app provide a custom F-Droid repo (like NewPipe) even for non-nightly builds?

With how many repos do I end up with in the end?
That would be a really ugly user experience.

And that also only works if the build is reproducible were implemented as otherwise the package names clash/signature clashes.

1 Like