Monthly Development Reports

11

Bazaar2 Monthly Report - June 2017

June marks the end of the final big development sprint for the Bazaar2 project, and many parts of this whole project have been completed, with others just needing some final bits and pieces completed. For the remaining couple months of the project, a few of us will be working to close out all those remaining bits and pieces to deliver the last sections of this whole funding effort.

One big piece of news was that Boris Kraut aka krt retired from active work on F-Droid https://forum.f-droid.org/t/so-long-farewell-and-goodbye. He was one of the major contributors to F-Droid over the past few years, leading up the fdroiddata section where apps are added to f-droid.org. He will certainly be missed. He retired with grace, and indeed provided a shining example of how to retire from a free software project, since he drummed up a lot of new interest, as well as new contributors, with his announcement.

One key part of the Bazaar2 project was to make F-Droid a fully localizable app store ecosystem. We localized the Android app, the website, the developer tools, and the documentation. So now basically every string a user sees can be translated. Some of this work was just applying well known software, but we forged new ground on a number of aspects. The details are under “Objective 1: Make all text translatable” and “Objective 3: Website“.

Organizations running their own F-Droid “repos”

One key piece of this project was to polish up the F-Droid server tools so that it was easy for anyone to run their own F-Droid repository. This turns F-Droid into a decentralized distribution ecosystem, where anyone can choose which distribution sources they use, and anyone can become a distribution source themselves. Whether other organizations set up their own F-Droid distribution “repos” is an important measure of this project. The first example is Copperhead, which uses F-Droid as its only app store, and runs a number of custom app repos for clients. F-Droid allows Copperhead to deliver a tightly controlled mobile system that anyone can run without relying on the big gatekeeper organizations like Google or Apple. Security First, has setup their own repo for their apps, including Umbrella (https://secfirst.org/fdroid/repo/). There is a relatively new app repo known as IzzySoft (IzzyOnDroid F-Droid Repository) that is fulfilling an important role in the whole ecosytem. F-droid.org only includes apps that are 100% free software, built from source code. That excludes a lot of valuable software that includes proprietary libraries like Google GCM. IzzySoft includes lots of apps like these, serving as a stepping stone on the way to inclusion in f-droid.org.

For the last example, we learned about a whole new model in Cuba. Torsten went to Cuba in June, following up on contacts from the postponed Cubaconf event, and one contact who found us in the fdroid IRC chat room. We spent a day with a couple of Cubans who are running mobile shop. They have set up an F-Droid repo with 2000 apps, and anyone can come and use it by connecting to their WiFi access point. Read all about it in Torsten’s blog post: How F-Droid is Bringing Apps to Cuba | Torsten's Thoughtcrimes

We have also been working with Fairphone to get F-Droid integrated into their Fairphone Open Android system. They are working towards selling Fairphone Open devices directly on their website, so once that launched, then that will be the first hardware manufacturer shipping F-Droid that we know about.

And two last additional items we touched in June:

Full localization waiting on final deployment

We can now show all the key parts of F-Droid localized, this will all be shipped in the next release of the various components. Here’s examples in Chinese:

Android client app

image_0.png429×816 106 KB

Website

image_1.png754×666 81.1 KB

Repomaker

http://repomaker.grobox.de

image_2.png1248×534 41.6 KB

Weekly Meeting Logs

We have a weekly meeting on IRC mostly focused the developer facing sides of F-Droid. That happens every Thursday at 11.30 UTC on #fdroid-dev on FreeNode. The June 2017 logs can be found here:

Following Work Related to this Funding

All related work on F-Droid is tagged using the “bazaar” label:

All related blog posts are tagged with the “bazaar” tag:

Objective 1 Simple multi-pronged distribution

Make all text translatable

In June, we had a major push to get all the strings throughout the project, from app strings to documentation, in a format that works well for automated translation. Those are all up on Weblate now, open for contributions. At least 95% of the strings used in the F-Droid software is now translatable and upon Weblate for easy contributing. We have been getting a steady stream of translation contributions in a variety of languages. We also hired some translators to finish the community translations and review them for Farsi, Simplified Chinese, and Spanish. We did not receive any contributions in Tibetan, but have hired two people to translate and review all strings in the F-Droid app, Repomaker, 10 app descriptions, and much of the website material. The source, translations, and activity for all the F-Droid projects can be seen on the Weblate project page:

An essential part of the work we do is integrating with other free software projects, and helping those projects improve. In order to provide a complete, smooth translation workflow, we working through a issues in four separate projects that each form an essential piece of the puzzle.

For tracking the localization work in F-Droid, see the localization tag in the gitlab tracker:

Reproducible builds

We have be reproducing Android app builds for some months now on https://verification.f-droid.org, it has reproducibly built 372 APKs from 319 different apps. The whole F-Droid ecosystem can now support matching APKs with an arbitrary number of signers. This is the last key blocker to allowing f-droid.org to also add the developer’s signature for any app that is built. Previously, the F-Droid tools only supported a single signer, and that signer was f-droid.org. This is also an important feature for cases where people are working with collections of APKs like Repomaker users or the Cuban app store example.

We have collected a large number of APKs that include the original developer’s signature, and are working to retroactively add the developer’s signed APK to f-droid.org whenever the build can be reproduced. Here are the signatures we are currently working with:

Objective 2 Curation Tools for Organizations

In the beginning of June, our design lead did an user experience test where potential users of Repomaker tried out the software. From this test, we got lots of feedback to improve he workflow of Repomaker. Most of these improvements have already been implemented.

  • improved workflow for managing storage services
  • improved workflow for adding apps from remote repos
  • app details of remote apps
  • internationalization of JavaScript code
  • drag and drop to upload files
  • lots of other improvements after ux test
  • currently under review: endless scroll through apps

Objective 3 Modern App Store with Built-in Circumvention

Website

We have launched the new static site on https://f-droid.org, replacing the Wordpress site that has served us well for the past 6 or so years. This is the foundation for the fully localized website. We set a high standard for ourselves with this new localized website, in terms of the use cases we wanted to cover. On our staging server now is a version of the website that covers basically everything that we wanted to do:

  • fully localized without requiring Javascript or setting the language in the browser/system
  • automatic language selection based on browser preference
  • any supported language can be selected directly via a menu
  • static site of only files to greatly simplify the hosting and security maintenance
  • polish workflow with static site generation (Jekyll)
  • a static site is also more resistant to DoS attacks, especially when using a major CDN

The goal was to support both the most private setups as well as the most automatic. The site is designed to work well with both the bog standard Tor Browser or TAILS setup, as well as the standard Javascript-enabled browser with the language preference included in every web request. A high risk user can keep the default language, then only select their preferred language only when they require a translation for a given page, whether or not Javascript is enabled. Setting the language preference in the browser or the system can divulge a lot of information about a user, especially if it is a minority language. So we ensured that it was not a requirement for getting localized pages. We are happy to consult with other projects who have similar goals.

UX Overhaul

We polished up two parts of the new Android user experience:

Streamlining Circumvention

We sketched out how to implement the final missing piece of the work to automatically use “collateral freedom” mirrors. The F-Droid client will get the list of official mirrors from any repo that supports mirrors. It will then automatically retry failed downloads using the next available mirror. F-Droid repos can now automatically be hosted on Amazon S3, GitHub, Gitlab, and any webserver accessible via SSH. That webserver can then provide a Tor Onion Service. The Guardian Project F-Droid Repo is setup like this, here are the current mirrors (also visible at the top of the repo XML https://guardianproject.info/fdroid/repo/index.xml):

Add media handling to app store experience

With the release of 1.0-alpha0, the F-Droid client can finally support “installing” media files. For common file types like music, video, etc. the files are downloaded into the standard Android folders for storing those media types (e.g. Music, Movies, etc). Then any Android app that handles those files will find and use them automatically. We had to forge new ground for OTA (Over-The-Air) update ZIP files, since there is no other app store that ships those kinds of files. In this case, F-Droid puts them into a standard, protected folder that is only accessible by the Android “recovery” system that installs such updates (e.g. TWRP).

3 Likes