Skip to content

Home

Jump to bottom
Imran edited this page Sep 27, 2024 · 24 revisions

Obtainium's core goal is to automate the process of downloading and installing Android app updates directly from their "source" websites (sites where app files are available for direct download). This process needs automation since users may not be willing or able to rely on an app store to update a given app, but Android apps (unlike PC apps) usually assume they will be updated externally by an app store and therefore don't include a built-in self-update function.

While this concept is simple, the wide variety of supported sources, user preferences, and APK naming, versioning, and distribution methods make things more complicated. This Wiki explains the various app sources and settings available in Obtainium. It is not comprehensive and may not be fully up to date.

UI Overview

The tab bar at the bottom of the screen is the main way to navigate around Obtainium.

Apps Page

This is the main screen; it contains a list of apps being tracked by Obtainium, and provides the basic info for each app.

On this page, you can use the buttons at the bottom of the screen to perform various operations (like delete, update, tag, etc.) on multiple apps at once. You can do this after selecting one or more apps, either using a long-press to enter multi-select mode or using the "select all" button.

You can also filter for specific apps by pressing the filter button. This lets you view only the apps that fulfill specific criteria (like installed/not-installed, latest/outdated, source site, etc.).

Add App Page

This page lets you add an app by its source URL.

As you enter the source URL, Obtainium will automatically detect what source logic it should use, and will present the relevant options. If a URL does not correspond to any supported source, the "HTML" source will be selected as a default. This source is a general fallback that may work in some cases. If you think Obtainium has made the wrong choice, you can manually specify the source type to use.

For most sources, you don't need to be exact with the URL you enter; for example, any GitHub URL that contains the base repo URL would be accepted by the GitHub source (for example, https://github.com/ImranR98/Obtainium/releases/latest would automatically be trimmed down to https://github.com/ImranR98/Obtainium). However, your URL must be more precise in two situations:

  1. When using the HTML source
    • For example, the Tor Android APK is at https://www.torproject.org/download/ so entering only https://www.torproject.org/ would not work.
  2. When manually picking a source (overriding Obtainium's HTML fallback choice)

This page also lets you search for apps across sources that support this feature (the Import/Export page also provides a separate search tool). Note that just because an app shows up in the search results does not mean it will be added successfully - it still needs to fulfill all other criteria.

Finally, this page lists all supported sources, along with additional info such as whether a source is searchable.

Import/Export Page

This page lets you export your Obtainium data so that it can be imported later. You can also enable automatic exports that happen whenever any data changes.

This page also provides various other ways of importing large numbers of apps, including through search.

The search tool on this page also lets you search sources that require you to specify the source host/domain, such as third-party F-Droid repos.

Settings Page

This page provides various UI and behaviour settings, including the option to enable more functionality for specific sources by adding required credentials.

How Apps are Tracked

Basics

When you add an app URL to Obtainium, you must pick an app Source. Sources define how app info and APK files will be extracted from the URL you enter. Most of the time, Obtainium will automatically select the appropriate source to use - when this is not possible, an "Override Source" dropdown will be presented.

At minimum, an app source must provide the following data for its apps:

  • The app version (or a 'pseudo-version' - an identifier that changes for each new version of the app)
  • At least one APK download URL that corresponds to the version that was provided

App sources may also provide other info - these enable extra features or UI benefits. For example:

  • The app author's name
  • The app's package ID
  • The release date of the latest version
  • Info for previous versions or variants of the app

In an ideal world, each app source would provide all required info in a straightforward way - with a single app per given URL with all required info provided in a standard format. However this is often not the case - there are many different ways app releases are handled even by the same source, so it isn't possible to have a fixed set of steps to handle them all. For this reason, you are presented with various additional options when adding an app, and these can be used to modify the way app info will be extracted. While the defaults work for most apps, you may want to understand these options to deal with edge cases - more info in the App Sources section below.

Note: Many filter settings in Obtainium (including many source-specific optional filters) make use of regular expressions - you should be familiar with these.

Version Detection

When Obtainium is tracking an app that is currently installed, it grabs the version of the app from Android and compares it to the version string provided by the source. It then compares the two to decide whether an update is available or whether the install status of the app has changed. This comparison can only be made if the two versions follow the same format, which may not always be the case. For example, you could have any of these cases among others:

  1. Obtainium from GitHub:
    • Android-reported app version: 0.14.21
    • Source-reported version: v0.14.21-beta
  2. Cheogram from a SourceHut instance:
    • Android-reported app version: 2.12.8-2+free
    • Source-reported version: 2.12.8-2
  3. Tor from the Tor website:
    • Android-reported app version: 102.2.1-Release (12.5.6)
    • Source-reported version: none (no version string is provided by this HTML source so a URL hash is used instead as a 'pseudo-version')
  4. Quotable from GitHub:
    • Android-reported app version: 1
    • Source-reported version: v10

Obtainium stores a list of "standard" formats which it uses to make this comparison (like x.y.z or x.y). If both versions being compared conform to the same format, the comparison will be made. If not, version detection will be disabled for that app. In some cases, Obtainium will strip off extra parts from the source string if doing so would result in a standard version (like how v and -beta are removed from Obtainium's v0.14.21-beta), then it can make the comparison. We never try to strip parts off the "real" OS-provided version.

This piece of code defines how the various "standard" formats are generated: https://github.com/ImranR98/Obtainium/blob/main/lib/providers/apps_provider.dart#L64

It's always possible to expand that code to add support for more formats, but this requires careful consideration. For example if Android reports that an installed app's version is 1.2 but the source says the latest available version of that app is 1.2-4, should we strip off the -4 and say the two are the same (meaning there is no update available)? This may be fine in some contexts (where the -4 is not actually indicative of a change in the app itself) but not in other contexts. So it wouldn't be a good idea to support that specific case.

Version detection being turned off should not usually have a significant impact on day-to-day use. If version detection is disabled for an app, you may occasionally run into inconsistencies between the real version of the app installed on your system and the version shown in the Obtainium UI. This should only happen in two cases:

  1. If an app's version changes due to actions taken outside of Obtainium (for example if it gets updated by Google Play)
  2. If an attempt by Obtainium to silently update the app in the background fails

In such cases, Obtainium would not be able to detect that the app's real OS version has changed and so it would not update its internal records accordingly - you would need to manually correct the inconsistency.

See also: https://github.com/ImranR98/Obtainium/issues/946#issuecomment-1741745587

Background Updates

Obtainium checks for app updates in the background on a regular basis. You can control the frequency of these update tasks on the settings page.

After a background update checking task is completed, any available updates are divided into 2 categories:

  1. Updates that can be applied in the background
  2. Updates that cannot be applied in the background

For an update to be automatically installed in the background (AKA a silent update), certain criteria should be met:

  • The OS must be Android 12 or higher
  • The app being installed must target a recent Android API level
  • The currently installed version of the app must have been installed by Obtainium
  • You must have background updates enabled in Obtainium (both universally and for this app in particular - this is the default)
  • If there are multiple APKs available for the update, the additional options for that app must be configured such that Obtainium can filter these down to one APK

Each available update is downloaded and installed if possible, and the user is then notified either of the update's availability or that it was installed in the background.

Note that due to technical limitations, background updates can only be installed on an asynchronous, best-effort basis. So if a background update fails to install, you will not be notified of the error.

App Sources

The way that an app is added, checked for updates, and installs will vary depending on source and the settings configured for that app by the user.

The following options are available for all apps regardless of source:

  • Track-Only: Enabling this toggle allows you to receive update notifications for apps without attempting to actually download their APKs. This is useful to track updates for apps where no APKs are available, or where they cannot easily be extracted by Obtainium. Some sources are exclusively track-only - for example ApkMirror. Note that version detection will not work for any apps added as track-only.
  • Version Detection: See the section on version detection.
  • Filter APKs by Regular Expression: When more than one APK is available for an app release, the user is prompted to pick one manually. Aside from being inconvenient, this means the app will not be updated silently in the background even when doing so would have otherwise been possible. This option lets the user specify a regular expression that can be used to filter out APK that don't match it (ideally down to one option).
  • Attempt to filter APKs by CPU architecture if possible: This toggle will tell Obtainium to automatically filter out any APKs that don't appear to be compatible with the user's device. The logic used to do this is very simple and is based on the APK filename, so it is not always reliable and the user may still need to do their own filtering in many cases.
  • App Name: This allows the user to specify their own name for the app instead of using the one Obtainium extracts.
  • Exempt from background updates: This toggle will prevent the app from being updated silently in the background even if background updates are enabled and this app would otherwise have fulfilled all criteria to be updated in the background.
  • Skip update notifications: This toggle tells Obtainium to avoid notifying the user that this app has an update available or that it was updated in the background.

Aside from those, each app has additional source-specific options. Most of those are self-explanatory and may be updated frequently, so they are not covered in this Wiki. However, some sources do have more unique behaviours that need more explanation, and these are covered below.

GitHub

GitHub puts a cap on the number of API requests you can make in a given period of time. Since Obtainium uses the GitHub API to grab release info, you may run into a "rate limit" error if you have more than a few dozen GitHub apps. You can get around this by getting a Personal Access Token from GitHub and saving it in Obtainium's settings (the token should only need the minimal read access permission). Note: Some users have had issues with access token permissions - see here for a discussion and potential troubleshooting steps: https://github.com/ImranR98/Obtainium/issues/1830

GitHub also allows developers to host multiple releases of their app. This usually means older versions of the same app, but may also include pre-releases, variants, etc. - so Obtainium provides various filters that let you navigate this and grab the exact releases you are interested in.

GitLab

GitLab releases sometimes contain APKs that are attached in non-standard ways, such that Obtainium cannot get to them easily (see this comment for a more detailed explanation). The GitLab API provides a far more reliable way to extract APKs but it cannot be used without an API key. While this shouldn't be an issue for most GitLab repos, you can add your own Personal Access Token in Obtainium's settings for more reliable APK extraction in edge cases where it does turn out to be a problem (the minimal read_api scope should be enough).

Just like GitHub, GitLab also allows developers to host older releases of the same app, so additional options are provided as appropriate.

F-Droid Third Party Repo

Unlike with most other sources, F-Droid repos contain info about multiple apps all under the same URL. This means that, in addition to the repo URL, you must provide the name or ID of the app you want separately. If you're not sure what apps are available in a given repo, you can use the "Search Source" button on the Import/Export page to find out.

Note that Obtainium cannot automatically tell that a given URL points to a third-party F-Droid repo. This means that by default, adding a third-party F-Droid repo to Obtainium will result in the incorrect use of the HTML source (a fallback source Obtainium uses for any URL it does not recognize). You must manually select "F-Droid Third Party Repo" from the "Override Source" dropdown to get around this.

APKMirror

APKMirror is a "Track-Only" source. This means that while you can add an APKMirror URL to Obtainium for update notifications, you will not be able to actually download and install the app. This is because the maintainers of APKMirror do not allow it (see issue #44).

HTML

The 'HTML' source is a fallback option for any app sources that are not explicitly supported by Obtainium. Because of its flexibility, it is also a way of supporting niche, less popular sources without bloating the app.

The HTML Source works in this way:

  1. It sends a request to the source URL provided by the user, and parses the response as HTML. It then looks for links on the page.
  2. It filters out certain links. By default, this is any link that doesn't end in .apk but you can use "Custom APK Link Filter" to specify your own filter.
  3. It sorts the remaining links. This sorting is alphanumeric sorting on the whole link, but you can choose to sort by only the last segment of the link. This last segment is usually the filename but may not be if you used your own filter in step 2.
  4. It applies yet another optional user-defined filter on all remaining links. The difference between this filter and the one from step 2 is that this one is a more general filter that all Sources have - it is inherited from the parent AppSource class (the APK filter option described in App Sources). It's probably easier to use this in some situations rather than having a single more complicated regular expression in step 2 (see this comment for an example). It is also useful when used in combination with the intermediate link option.
  5. Of the remaining links, it picks the first one (or the last one if you enabled the reverse option).
  6. Now that we have the final APK link, we need a unique release ID to go with it so that when the ID changes, we know the app has an update available. For other Sources, the unique release ID is the app version, but for the HTML Source, it might not be possible to extract a version string. So by default, this is just a hash of the link.
    • However, links often have version strings embedded within them. Obtainium can't know how to extract these on its own (different websites would have different ways of doing it), so the user can choose to specify a regular expression that can be applied to the link in order to extract the version - this is what the "Version Extraction" field is for.
    • But often it can be difficult to come up with a regular expression that accurately matches the version while excluding extra characters. For this, we have the "Match Group" option that lets the user specify which group in the regular expression we should use as a version.
    • The version extraction feature isn't really necessary - using link hashes is easier and more reliable. Some users might just want it since having the real version looks nicer/more accurate and it allows Obtainium, in most cases, to use version detection.

As for the "Intermediate Link" filter, if this is used, the HTML Source works as follows (see issue #820 for a situation where this is useful):

  1. It looks for links on an HTML page.
  2. It filters out any links that don't match the intermediate link filter.
  3. It grabs the first remaining link (no reverse option here), and then feeds that link as the input for the normal HTML Source process described previously.

Notes for Various Other Sources

  • HTML: Note that the HTML source includes default request headers that should be appropriate for most sites. In some cases (for example SourceForge, you may need to delete them (and possibly specify your own).
  • Codeberg: This source is almost identical to GitHub in its additional options.
  • F-Droid: Any app from F-Droid is likely to be signed with a different key than releases of the same app from other sources. This means that trying to update from an F-Droid release of a given app to a release from another source (for example GitHub) is likely to fail.
  • Any Source that does not have a specific host associated with it (like third-party F-Droid repos, Jenkins instances, and SourceHut instances) will not be automatically recognized by Obtainium. You must manually pick the right source from the "Override Source" dropdown.
  • Some sources (like APKPure) may provide XAPK files instead of APK files. Obtainium's XAPK support is incomplete and may not work reliably.
Clone this wiki locally