Monthly Development Reports

9

April was a big month for us in terms of finishing up some big parts
that are directly visible to users, and easy to demonstrate. The
biggest is the final 0.103 release of the F-Droid app which includes the
complete overhaul of the user experience, which feels simple, friendly
and modern. This is one short step from a big 1.0 release, once we nail
down the last features and get some more testing completed.

We also launched the first alpha of the new F-Droid Repomaker, a simple
web tool for creating and managing collections of apps and media, and
delivering them to users via F-Droid repositories (aka “repos”). Try
the alpha demo! http://repomaker.grobox.de/

On top of those two launches, there are many other small accomplishments
from this biggest and final development sprint for Bazaar2.

##Objective 1 Simple multi-pronged distribution

###Make All Text Translatable

All texts within F-Droid and graphics associated with apps are now
translatable, including all the strings within the app itself, all app
names, summaries, descriptions, video links, recent changes, and
screenshots. With release of F-Droid client 0.103, it will use any
available language. For the F-Droid client app itself, many languages
are completely translated, and many more have reached the functional
level, thanks to the ongoing support from F-Droid community volunteers
and the Localization Lab:

  • 19 over 99%, including Belarusian, Brazilian, Persian, Russian,
    Spanish, Chinese, Turkish
  • 32 over 90%, including Arabic, French, Italian, Romanian, Shona, Ukrainian
  • 45 over 70%, including Burmese, Hungarian, Korean, Simplified Chinese,
    Thai, Vietnamese
  • see all and contribute here:
    F-Droid/F-Droid @ Hosted Weblate

We have not received any Tibetan translations yet. We will be hiring
translators to finish the Simplified Chinese and Tibetan translations.

For the per-app materials, we are now adding all the translated
materials for all the Guardian Project apps to the Guardian Project
F-Droid Repository, which users can enable with the flip of a switch in
F-Droid. We are also helping app developers to get their descriptive
materials integrated for automatic inclusion in f-droid.org.

###Reproducible Builds

For reproducible builds, we started out by doing mass rebuilds of all
apps in f-droid.org, as shown by https://verification.f-droid.org. This
let us fix the most common issues without getting stuck on a few hard
issues. Now that we have reproducibly built over 300 different apps,
we’re turning to focus on reproducibly building the most
security-sensitive apps. These tend to be the most difficult since they
frequently include “native” C code, which is much harder than Java to
build reproducibly.

###Handling Media

While the core tools for adding media files to F-Droid repositories were
created months ago, we turned to focus on one specific use case in order
to polish up the media file support: the F-Droid Privileged Extension
“Over-The-Air (OTA) update”. This is a ZIP file that users “flash” to
their device to install it with elevated privileges. This file is now
built, signed, and released using the full F-Droid stack, providing a
trusted download method for users of any Android ROM to flash to their
device:

That means the whole server-side deliver process is ready to handle any
file you can copy into a folder. The 1.0 release of the F-Droid client
app will fully handle installing common file types so that media
players, etc. will automatically find and play them. As part of the
Curation Tools section, RepoMaker already has some basic support for
handling media, which we are now working on completing and polishing.

##developer Support

In collaboration with Guardian Project’s Developer Square effort, we
held a workshop on the internet called GLOW2017:
https://devsq.net/glow2017 . The videos are archived and available for
anyone to learn from.

###Google Play Integration

When the Bazaar2 project was defined, there were not well known tools
for managing all of the localized files in Google Play. Now there are
two: Fastlane Supply and Triple-T Gradle Play Publisher. Both are free
open source software, so instead of reinventing the wheel, we instead
integrated with those existing tools. fdroidserver now automatically
detects the app store support materials in the app’s source repo if it
is already setup for Fastlane or Triple-T. So there is now one place to
put all of the app store materials (descriptions, graphics, etc) to
publish them to F-Droid and Google Play. Those descriptions can be
easily added to Weblate, Transifex, etc so that the translations can be
automatically synced when they are complete.

##Objective 2 Curation Tools for Organizations

RepoMaker has reached a functional level with the core features
implemented. It is currently being developed around the two basic setup
modes: as a hosted web app. Apps can be manually added or automatically
fetched from other F-Droid app repos. RepoMaker can publish the repos
in all the same ways that fdroidserver can, e.g rsync GitHub, Amazon S3,
etc. There is a alpha demo of the multi-user mode for anyone to try:
http://repomaker.grobox.de

You can see demos of a number of key features in Torsten’s RepoMaker
playlist:
https://www.youtube.com/playlist?list=PLts8E5OKFffNMtw0HG3MaDiyfig-sfczT

We also began to build the foundations of the localization support.
This current implementation strategy will also allow for standalone
installations like a desktop app following the web app model like Riot,
Signal, etc.

##Objective 3 Modern App Store with Built-in Circumvention

The new user experience is functionally complete and a full release,
v0.103, is now available via the normal release channels. We also
nailed down the full integrated experience using F-Droid Privileged
Extension, which allows for installs without enabling Unknown Sources
and automatic updates in background. It is now well tested and working
solidly on all Android versions. For the past month, we found and fixed
a number of issues specific to Android 7.x.

###User Tests

We ran two parallel user tests in Lubbock, Texas and Vienna, Austria of
the new user experience for the F-Droid client app. Overall, we are
happy to say that they confirmed the general approach of the new design,
and users overwhelmingly found it simple to use. There were two areas
where users had difficulty: nearby app swapping and adding new app
repositories. This was not a surprise since, first and foremost, those
are totally new concepts for most mobile users, who are used to getting
everything from one source: Google Play.

The full report is available at:

##website

The new website is ready for launch, once we complete the secure,
automated deployment procedure. The new website is generated using
Jekyll and consists entirely of flat files with no code running on the
server side. On client-side, Javascript is only required for the search
function. This makes the website work well with Tor Browser, and makes
it easy for anyone to deploy their own app store using simple cloud file
hosting services like Alibaba Cloud, GitHub Pages, Gitlab Pages, Amazon
S3, etc. as well as simple appliance devices like LibraryBox,
FreedomBox, etc. We also began the process of making the website fully
translatable. The staging server is publicly available here:

###Automated Circumvention

The fdroidserver tools for automated “collateral freedom” distribution
are in place. The current options for automatic publishing to mirrors
are: GitHub, Gitlab, Amazon S3, and SSH/rsync for webservers and Tor
Hidden Services. The F-Droid client app is already receiving the
metadata about those mirrors, but it does not yet automatically act on
it. Users can manually subscribe to individual mirrors now. The
Guardian Project app repo is currently setup for all of these types of
mirrors:

As for mirrors of f-droid.org, we launched a third mirror for the main
repo which is in the USA. This will better cover the Americas over the
two European mirrors.

##malware Tools

We added support for two sources of metadata about apps. Fdroidserver
can now automatically upload all new release to
https://androidobservatory.org and https://virustotal.com. These both
provide rich sources of metadata about apps and malware, viewable via
web pages or accessible via an API. They both are based on the SHA256
hash sum as a unique ID, so it is easy to link an APK on a device to the
data on those services. This data will be used to alert the user to
known malware in the new “Updates” tab of F-Droid client.

##Objective 4 Partner Deployments

We have two prototype libraries for ensuring that apps have a reliable,
trusted update channel no matter where they were downloaded from. There
are lots of custom versions of this, from Firefox to Signal. The
libraries that we are creating are standardized, free software
libraries. They also integrate with the whole F-Droid eco-system, using
the same tools to manage the server-side as are used for F-Droid
“repos”. This provides the flexibility for app developers to mix and
match the features they need, like direct app updates via a dedicated
app repo, updates via https://f-droid.org, confirmed reproducible builds
of releases, “collatoral freedom” mirrors, etc.

Our first test implementations for these new libraries will be Zom for
the direct updates, and Ripple and Location Privacy for the F-Droid
update channel.

##Objective 5 Usability Research on In-country Developers

The results of the survey have been compiled, and the public report is
nearing completion.
We ran user tests of the fdroidserver tools in a handful of locations.
We were unable to run the tests in Eastern Europe as we had hoped.

4 Likes