Lack of TLS 1.2 breaking apps in older Androids

Hi, the user Slinger on GitHub found an alternative for Google’s ProviderInstaller. Unfortunately, it increases the apk size noticeably and needs to be kept up-to-date. I built a little mock-up that allows to distribute the provider centrally in one app and then load the library from other apps. This is basically the same that Google does. If one central app (for example the F-Droid client) would provide the library, it would not be necessary that all apps bundle it on their own.

For detailed information and a pretty lengthy discussion, visit the Pull Request for AntennaPod: https://github.com/AntennaPod/AntennaPod/pull/4077

TLDR: AntennaPod’s F-Droid version will ship with a custom SSL stack (Conscrypt) until we find a better solution (for example using one that is provided centrally by F-Droid).

3 Likes

Do you want to write a blog post about those possibilities for publishing on f-droid.org? I think this is very much of general interest.

F-Droid also can’t to tls 1.2 on kitkat right now, though I think we’d want to do that. Have you looked on how that could be integrated into fdroid client yet?

  • reads the pr discussion now.
1 Like

Including it in an app by bundling Conscrypt is pretty easy:

implementation "org.conscrypt:conscrypt-android:2.4.0"
Security.insertProviderAt(Conscrypt.newProvider(), 1);

The problem is that every app is then forced to update the provider regularly. Also, it increases the apk size. So the idea would be to write a provider application that does nothing but bundling Conscrypt and providing a stable API that allows to use the ClassLoader. It can then be updated independently from the apps that include the provider (like AntennaPod, F-Droid). The provider app could do something like this:

public static void install() {
    Log.d(TAG, "Installing provider...");
    Security.insertProviderAt(Conscrypt.newProvider(), 1);
    Log.d(TAG, "Provider installed successfully.");
}

One could then maintain a library that does the security checks and ClassLoader handling:

Context targetContext = context.createPackageContext("com.bytehamster.providerinstaller",
        Context.CONTEXT_INCLUDE_CODE | Context.CONTEXT_IGNORE_SECURITY);
ClassLoader classLoader = targetContext.getClassLoader();
Class var2 = classLoader.loadClass("com.bytehamster.providerinstaller.ProviderInstallerImpl");
Method installMethod = var2.getMethod("install", new Class[]{ });
installMethod.invoke(null);
installed = true;

Code samples are also available (and a bit more detailed) on GitHub Gist.

Main problem: By including the provider, you execute foreign code in the context of your application. How do you verify that the provider does not do malicious things? I have some ideas how to deal with this:

  1. Let a trusted application be the provider. The F-Droid application itself could bundle the provider, keep it updated and provide the stable API. Apps could then hard-code the F-Droid package name and verify its signature. The problem with this approach is that users who build F-Droid themselves or use alternatives like G-Droid could not use the updated libraries.
  2. Send an intent that allows every app to be a provider. The library could query the Android system for compatible Conscrypt providers and include one of them. The problem with this approach is that any app could reply. Therefore, it would be possible to only include the provider if it is a system app. Then the F-Droid privileged extension could bundle and provide Conscrypt. If a malicious app managed to get system status, it is too late anyway, so we do not need to check a specific signature in this case. Many F-Droid users probably do not use the privileged extension though.
  3. Provide an open-source re-implementation of Google’s provider installer library (does this infringe their TOS?) and use ClassLoader to load Google’s provider if available. If it is not available (= no Google Play services and no microG), check for a system app like in the second idea. Then the problem would only occur to users who use a system without Google services (likely a custom ROM), so they know how to install the privileged extension as system app.

I am wondering if an app that uses the ClassLoader to run another (closed-source) app’s code can be considered open-source. Using ClassLoader for Add-Ons would be a pretty nice use-case but one can not really prevent executing the code of closed-source Add-Ons.

End note: I am pretty busy with maintaining AntennaPod. I am therefore not able to maintain the library and installer app. The proof-of-concept above (also on GitHub Gist) will probably be the only code I write for this :slight_smile:

What exactly do you think I should write there? Something like this post, together with the problem statement from the first post?

3 Likes

For the reasons you stated I don’t think it’s a good fit to include this into fdroidclient itself. BUt I think instead the simplest solution would be to make a standalone app, say org.fdroid.securityprovider and just hardcode the signatur of this app whereever it is used.

Does microg implement this already actually? :thinking: Could this be used/reused somehow?

Yep, pretty much! :slight_smile:

1 Like

Yes: https://github.com/microg/android_packages_apps_GmsCore/blob/master/play-services-core/src/main/java/com/google/android/gms/common/security/ProviderInstallerImpl.java

Not sure. Conscrypt seems to work just fine with one single line of code for initialization (in the provider app).

How would you encourage users to install that app, then? This sounds like additional work for developers who include the library (user confirmation, installing, etc).

1 Like

Well, you only need to install this, if you have connectivity problems, and you likely have that in all of your apps then. googling the problem should quickly point you in the direction of installing this app might fix it. Someone could of course also include a check for this in their app, test connectivity to a TLS v1.2 only server and if that fails with not supported then recommend installing this app.

2 Likes

Text proposal below. Any opinions? Did I forget something? If you are fine with it, I will create a PR for the website.

---
layout: post
title: "Android updates and TLS connections"
author: "ByteHamster"
authorWebsite: "https://github.com/ByteHamster"
---

Android device vendors are known for their short support periods. They only provide updates for a limited time or even ship devices with old Android versions. Developers then have to deal with a fragmented distribution of Android versions. Android 4.x is still used by around 2% of the active devices on Google Play. Old phones usually still have decent performance and updating them with Custom ROMs makes them well-suited even until today. Unfortunately, not all devices support Custom ROMs and not all users want to deal with the trouble of installing one.

AntennaPod is an open-source podcast manager for Android that I maintain. AntennaPod does not rely on a centralized server but fetches the podcast feeds directly using the device. This is pretty nice for user’s privacy because nobody gets a full list of your subscriptions. Unfortunately, more and more users with old Android versions experience problems with some podcast feeds. The reason is that servers are updated to more recent TLS versions that are not available on those devices. Upgrading the TLS version of servers is definitely a good thing because it makes the communication more secure. That’s why most browsers ship their own stack that they can update independently of the device vendors. Unfortunately, all other apps that are dependent on connections to many servers out of their control, like AntennaPod, still experience issues. Users then see download errors like the following: Failure in SSL Library, usually a protocol error. This is pretty bad because it prevents users from listening to podcasts. While this is basically the device vendor’s fault, it ultimately backfires on app developers.

TLSv1.1 and TLSv1.2 are supported starting with Android 4.1 and were enabled by default in Android 5.0. There also is a number of Cipher suites and certificates that are not supported in Android 4.x. While this problem currently only affects Android 4.x users, more will follow in the future as servers are upgraded to TLSv1.3. TLSv1.3 was first shipped with Android 10, which runs on about 8% of the active devices on Google Play. You can find more details about the SSL support in Android on the SSLSocket reference page. Developers now have to deal with the fact that vendors neglect software updates.

This is a known problem, so Google published a library called ProviderInstaller. It’s pretty easy to use (ProviderInstaller.installIfNeeded(context);) and fixes all those problems instantly. Because the provider is kept up-to-date independently of the app using Google Play Services, app developers do not need to worry about this anymore.

This sounds too good to be true, doesn’t it? Well, Google’s ProviderInstaller has a pretty big drawback. The library is closed-source, so it can not be used for apps published on F-Droid. AntennaPod currently builds with two different flavors: one that is 100% free software for F-Droid and one with the ProviderInstaller library for the Google Play Store. This means that the F-Droid users are left behind. They still experience connection problems to some servers and consider this a bug in AntennaPod.

An alternative that works with F-Droid is to include the open-source library Conscrypt. Bundling Conscrypt with an app is just as easy as using Google’s ProviderInstaller but it has two major disadvantages:

  • App developers are forced to update the provider regularly. This is additional work for the maintainers and might cause trouble with apps that have a slow development cycle.
  • Bundling Conscrypt increases the apk size. In case of AntennaPod, this means that the app gets around 4 MB bigger, which is a 40% increase just for this mostly invisible change. Now consider that every app would have to do that in order to get an updated TLS stack. If every app on F-Droid that deals with networking needs to bundle additional 4 MB of TLS libraries, this would increase storage usage noticeably.

Fortunately, there is a solution for this! The idea is to write an open-source provider application (just like Google’s provider) that does nothing but bundling Conscrypt and providing a stable API that allows other apps to load it. It can then be updated independently of apps like AntennaPod or F-Droid. The provider app could simply have a class with something like this:

public static void install() {
    Security.insertProviderAt(Conscrypt.newProvider(), 1);
    Log.d(TAG, "Provider installed.");
}

We could then develop an open-source library that uses the ClassLoader to include the Conscrypt library from the provider app.

Context targetContext = context.createPackageContext("com.bytehamster.providerinstaller",
        Context.CONTEXT_INCLUDE_CODE | Context.CONTEXT_IGNORE_SECURITY);
ClassLoader classLoader = targetContext.getClassLoader();
Class installClass = classLoader.loadClass("com.bytehamster.providerinstaller.ProviderInstallerImpl");
Method installMethod = installClass.getMethod("install", new Class[]{ });
installMethod.invoke(null);
installed = true;

Code samples are also available (and a bit more detailed) on GitHub Gist.

Obviously, just including code from another app is a security risk. By including the provider, you execute foreign code in the context of your application. How do you verify that the provider is not malicious? There are some ways to verify that the provider can be trusted:

  • Let a trusted application be the provider. The F-Droid application itself could bundle the provider, keep it updated and provide the stable API. Apps could then hard-code the F-Droid package name and verify its signature. The problem with this approach is that users who build F-Droid themselves or use alternatives like G-Droid could not use the updated libraries.
  • The library could query the Android system for compatible Conscrypt provider apps and include one of them. To make sure that this is secure, it would be possible to only include the provider if it is a system app. Then the F-Droid privileged extension could bundle and provide Conscrypt. Many F-Droid users probably do not use the privileged extension though, so we could still not reach all users.
  • To me, the best solution seems to have a standalone app (like org.fdroid.securityprovider) that provides Conscrypt. This was also discussed on the F-Droid Forum and originally suggested by @bubu. A trusted entity like F-Droid could build the provider and sign it with their keys. Those could then be verified by the calling apps that want to include Conscrypt.

I would love to see some movement in this direction. We should definitely explore the possibilities of a fully open-source security provider app. Unfortunately, I am pretty busy with maintaining AntennaPod. I am therefore not able to maintain the library and installer app. There is a proof-of-concept available on GitHub Gist but it misses security checks. Also, it would probably be good to add some simple ways to prompt users to install the provider on TLS failures. If you are interested in making an open-source provider possible, have a look at the F-Droid Forum.

Some notes:

  • This post was written from an app developer’s perspective. I try to make AntennaPod work best for all users, including those with Android 4.x. From a user’s perspective, you should think about whether you still want to use old Android versions like 4.x. Android 4.4 was released in 2014, which is 6 years ago. Many critical security issues have been fixed since then. Exposing such an old device to the Internet might be a pretty big security risk, independently of the TLS libraries that are included in apps.
  • Before looking into this, I did not know that you can use ClassLoader to execute other app’s code. This is pretty nice and can be used for things like plugins. I would love to see what other developers build with that.
4 Likes

@ByteHamster,

This write up looks great! I only have one comment/question:

If you hard code apps to check the signature of the trusted F-Droid “security provider app”, wouldn’t that also run back to the same problem if someone wanted to build the security app themselves?

Granted, I think this is the best solution for preventing every app developer from dropping old device support or having to bundle it themselves, just a thought about how this dependency would work for those who build them locally.

If this is the case, then bundling it with F-Droid would be simpler than making a separate app, wouldn’t it, because the end result is the same? People who build F-Droid themselves are likely to build the security app themselves also. E.g., the paranoid or enthusiast who builds it all themselves would still need to download the F-Droid security app because they can’t build it themselves or else their other apps will not work anyways.

Right? Or am I misunderstanding something (happens alot, I’m not very bright). :smiley:

2 Likes

Yes, they would run into the same problem.

The difference is that there are fewer reasons for building the provider app yourself (except trust). I build some apps myself because I (rarely) do pull requests for them. Doing lots pull requests for the provider app should not be necessary - the provider app does not have a UI and really limited functionality. Some users prefer G-Droid over F-Droid (UI, ratings, etc) – but having multiple provider apps is pretty pointless. So I think less users would build the provider app than the F-Droid client.

:joy:

1 Like

Sounds good! :slight_smile:

2 Likes

Great write-up @ByteHamster! Thanks for working on this important issue. I think your solution makes a lot of sense. I think that fdroidclient could easily include Conscrypt, since it would be nice to have there anyway. Then it could make Conscrypt available to apps via ProviderInstaller. A standlone org.fdroid.securityprovider also makes sense, there could even be both options, then have ProviderInstaller check org.fdroid.securityprovider, then org.fdroid.fdroid, then Google Play Services.

I don’t think it makes sense to include Conscrypt in F-Droid Privileged Extension. That is designed to be as minimal as possible and only handle install and uninstall commands. It has no networking at all.

4 Likes

Hi! First of all, I’d like to thank you @Bubu and @hans (and everyone else) for your work on F-Droid. It’s the only thing that makes android bearable for me. I’m really happy that this idea has been given so much positive attention. This has been high on my wishlist since I discovered conscrypt could be bundled with apps. This will benefit all devices, not just old ones (improved security, TLSv1.3, etc.)

I was originally planning to bring this up when I had a good idea for how to implement it, but since it’s already being discussed I’d like to add my thoughts on implementation and why using signatures could be a big problem. Most of this is just repeating what I’ve written here and here, but I’ve already written way to much there and I’ll try to keep this short and summarize my thoughts. [[[me from the future: turns out this became a giant wall of text… sorry everyone! I hope it’s still decipherable…]]]

If the solution relies on all apps checking a signature, no matter how it’s implemented, if that signature changes then all apps (and relevant versions of them) will have to be rebuilt against the new signature. Since most apps will benefit from the increased security and supported protocols (and cipher suites), even on relatively modern platforms, it’s possible that in the future the number of apps having to be rebuilt will become vary large, possibly without any clear way of telling which app will need to be rebuilt or not. And it possibly includes apps outside f-droid (which may or may not be easy to rebuild).

Compare this to the Privilege Extension: right now when building the F-Droid client with a different signature, it only requires rebuilding the Privilege Extension against the new build (Application ID, key fingerprint). This is just as easy as building the Client, and how to do this is even documented (very nicely) in the README. Rebuilding every single f-droid app requires setting up a build server and a lot more work (and time).

And there are many situations where the signature would change (=all apps rebuilt):

  • If a developer/contributor wants to improve f-droid (or conscrypt provider/installer depending on how it’s done) and/or the bundled conscrypt (like fix errors or update the conscrypt version), and then test it on his/her phone with apps before submitting the changes.
  • If someone wants to make a fork of the F-Droid client. In fact, there already is a (work in progress) fork that tries to restore the old/classic user interface. Should all apps then be rebuilt by that person? Or should F-Droid rebuild the apps against both signatures? Should this be done for every future fork?
  • If the signature leaks it can lead to extensive (and easy) attack vectors with code execution in every single app that relies on it. Completely fixing it will require rebuilding all apps, including old versions that people might have installed. And tracking down apps outside f-droid that might be affected… and finally get everyone to update…
  • If someone wants to make an alternative conscrypt provider/installer that is compatible with the f-droid apps… they would need to rebuild them and host an entire app repo.
  • If someone wants/needs to make a fork of the entire F-Droid project. Possibly because the project gets overtaken similar to CyanogenMod and needs to be rebranded. Admittedly this would be the easiest problem to fix (since new build servers would be set up anyway).

Also I feel that this would lead to a weird kind of Tivoization (“here are the sources for the f-droid client/conscryptprovider, but you can’t actually use them for real”). And possibly be a problem for Software Freedoms #1 and #3 (the freedom to change and to distribute changed versions). But if nothing else, it’ll be very impractical in the situations listed above. Imagine if manually building a new openssl package for debian also would require rebuilding every single package relying on it? :thinking:

The reason I originally suggested using the Security Extension (or something like it) is mainly because the way it’s installed (via a .zip through recovery) gives it special access to the OS. This could be a way to implement a safe/reliable API (that no normal app can mimic) for apps without requiring signature checks. And since it has a safe communication channel with the f-droid Client, the client could update the conscrypt version. Also it would not increase the normal f-droid client apk size. And no need to enable “untrusted sources” to install another apk. I see 3 approaches in this spirit (this is what I wanted to test and evaluate before starting this discussion, but maybe it’s for the best if everyone look at it):

  1. Add support for an API to the Privilege Extension which no normal app can mimic, through which normal apps can get conscrypt, and the f-droid Client can update conscrypt.
  2. Create a new “app” like the privilege extension (a .zip installed through recovery), but with the sole purpose of handling the conscrypt stuff. More modular, allows people to install only if needed (but even modern devices will benefit from an up-to-date conscrypt).
  3. The most interesting: create a .zip for installation through recovery that adds a new .jar file to the os files, which gets run at startup and does something similar to “Security.insertProviderAt(Conscrypt.newProvider(), 1)” but on a global level (like the security providers included in the os). The cool part about this is that it will replace the default security provider for the entire os, for default apps and installed apps (from f-droid or not) without requiring any app changes! Does anyone know how feasible this is?

Also it would avoid the catch-22 of the fdroid client needing conscrypt to download conscrypt (which will probably be necessary in the future since KitKat misses most/all sane cipher suites used for TLSv1.2). But of course that could also be solved by bundling conscrypt with the f-droid client, but it will increase the apk size. And as I’ve said above sharing the bundled conscrypt with other apps should not be done directly by having each app checking the signature of the f-droid client.

Ideally I’d like to see something like (3) above, combined with the ability of upgrading the provider through F-Droid. On the exact details, if either conscrypt is bundled with f-droid and provided to the conscrypt-/providerinstaller (with a signature check like Privilege Extension) which in turns provides it to other apps (inserted globally). Or if the F-Droid client updates conscrypt as a file which the conscryptsprovider/-installer blindly uses (like “libconscrypt_jni.so” in google’s SecurityProvider… I think)… well, I’m not sure. I’m leaning towards the later because then the conscrypt can be inserted at os-level before the f-droid client has even started or if it’s missing.

Also, this might be important to keep in mind: in order to make sure an app uses TLSv1.3 on all platforms and disables obsolete/insecure protocols (SSLv3, TLSv1.0, TLSv1.1), it’s not enough to just use an a modern conscrypt version. A little bit more work is necessary, see my example for bundling conscrypt. This could possibly be done automatically through the conscryptinstaller/API. Or at least it should be mentioned in some documentation.

So… I’m sorry for writing so much text! I just want to add, that I don’t believe that I know everything here, and in the end I trust you to make the right decision on this. I’m just worried that the wrong decision at this point can end up causing a lot of problem down the line. And it could become really difficult to undo once it’s in place and used by many apps. At some point in the future we might even have “historical” apps that can’t be rebuilt anymore, which could make the f-droid signature incredibly important (or dangerous…) for their compatibility.

There is one more thing I wanted to bring up: until this is turned into reality, would it be possible to put two versions of AntennaPod on F-Droid? I can probably change my pull request so that the Free flavor can be built both with and without bundling Conscrypt. Or even better, I could provide my changes as a .patch, which I think the f-droid build system can use through the build metadata(?). There has been concern about the increased apk size so giving people the option might be best.

1 Like

I do not think that we should publish a second app.

  • Users can not switch between versions (even with database export, file paths differ)
  • Confusion about the two versions (non-technical users should not be forced to decide between options they do not really understand)
  • More maintenance overhead (differences between Google Play and F-Droid version already led to support emails in the past that I need to answer)
  • Duplicate app can not easily be removed (when deleting one of the two versions, half of the users will keep the old version and think that it is no longer updated, even if we display a message. Most users don’t read)

In general, I am pretty sure that a duplicated app will lead to more work for me. I am busy enough already, so I’m not a big fan of doing that. By the way, help with replying to user requests (Google Groups or our new forum) is highly appreciated :slight_smile:

2 Likes

I’m really thrilled to see all the comments on this, by folks with much greater technical knowledge than me. As I read them all, one thought did occur to me. What if I’m correct in my suspicion that F-droid itself is affected (ie the failure to retrieve update notifications without toggling repos off and on)? In that case, it makes sense to install the solution by default along with the F-Droid client, despite the larger APK size.

The doesn’t answer any of the questions around how that bundled solution would be activated in the F-Droid client and other apps that need it, as well as in alternative UIs and forks, and how to prevent this creating security risks. But I note that any F-Droid fork or other app repo trying to support older devices will likely hit similar problems, and would also need to bundle a solution.

2 Likes

I agree that it can lead to complications, but I want to correct a few things (I’m not talking about duplicating/forking the app):

  • Unlike google’s app store, F-Droid supports multiple variants of an app. So the same app page can provide the same name and version of an app but in several flavours. The user can change flavour by just clicking on “install” on the version+flavour they want in the version list (the database will be kept).
  • By default F-Droid would install the flavour without conscrypt. And the description for the app would mention the flavour/version that bundles conscrypt and the point of it.
  • If a new version of antennapod appears that unifies the two flavours as one single flavour (either bundling conscrypt or using conscrypt from f-droid), then the F-Droid client will of course update both flavours of the old version to the single new version. This will require no user interaction, apart from choosing to update.

It was really just a suggestion if you’d want to avoid all users having to install a bigger apk (which you were concerned about). In the end, if the only version on f-droid would bundle conscrypt then I’d be happy with that (especially since I’d not have to make any major changes to my current patch for bundling conscrypt with antennapod). And if F-Droid can provide conscrypt to all apps (preferably without depending on a hardcoded signature check) then no more discussion on bundling conscrypt with antennapod or any other app would even be necessary.

1 Like

I thought we agreed on bundling conscrypt with the next update.

1 Like

I think we’re getting of topic, but as I said: I’m happy if conscrypt gets bundled with the next version. It’s what I originally wanted and what my patch/pull request does. :grinning:

But you were originally concerned by the increased apk size. You accepted bundling conscrypt when the only alternative was to not bundle it. I just wanted to point out that a third option exists: that f-droid (unlike google play) can offer two flavours of the same app (so installing antennapod with and without conscrypt would both be options for the user, with one of them being a default option).

I mentioned the idea in our discussion about my pull request for antennapod (bundling conscrypt) but I got no reply. And I mentioned it again here also to see if the f-droid developers had some opinion on the idea. I realize it’s mostly unnecessary and potentially could lead to more problems. And I’m happy with just one flavour (bundling conscrypt).

2 Likes

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.

1 Like