A guide to back up and recover 2FA tokens from FreeOTP (Android)

freeotp_backup.md

Backing up and recovering 2FA tokens from FreeOTP

NOTE: THIS MAY NOT WORK ANYMORE - SEE COMMENTS

Backing up FreeOTP

Using adb, create a backup of the app using the following command:

adb backup -f freeotp-backup.ab -apk org.fedorahosted.freeotp

org.fedorahosted.freeotp is the app ID for FreeOTP.

This will ask, on the phone, for a password to encrypt the backup. Proceed with a password.

Manually extracting the backup

The backups are some form of encrypted tar file. Android Backup Extractor can decrypt them. It's available on the AUR as android-backup-extractor-git.

Use it like so (this command will ask you for the password you just set to decrypt it):

abe unpack freeotp-backup.ab freeotp-backup.tar

Then extract the generated tar file:

$ tar xvf freeotp-backup.tar
apps/org.fedorahosted.freeotp/_manifest
apps/org.fedorahosted.freeotp/sp/tokens.xml

We don't care about the manifest file, so let's look at apps/org.fedorahosted.freeotp/sp/tokens.xml.

Reading tokens.xml

The tokens.xml file is the preference file of FreeOTP. Each <string>...</string> is a token (except the one with the name tokenOrder).

The token is a JSON blob. Let's take a look at an example token (which is no longer valid!):

<?xml version='1.0' encoding='utf-8' standalone='yes' ?>
<map>
	<!-- ... -->
	<string name="Discord:[email protected]">{&quot;algo&quot;:&quot;SHA1&quot;,&quot;counter&quot;:0,&quot;digits&quot;:6,&quot;imageAlt&quot;:&quot;content://com.google.android.apps.photos.contentprovider/-1/1/content%3A%2F%2Fmedia%2Fexternal%2Ffile%2F3195/ORIGINAL/NONE/741876674&quot;,&quot;issuerExt&quot;:&quot;Discord&quot;,&quot;issuerInt&quot;:&quot;Discord&quot;,&quot;label&quot;:&quot;[email protected]&quot;,&quot;period&quot;:30,&quot;secret&quot;:[122,-15,11,51,-100,-109,21,89,-30,-35],&quot;type&quot;:&quot;TOTP&quot;}</string>
</map>

Let's open a python shell and get the inner text of the XML into a Python 3 shell. We'll need base64, json and html in a moment:

>>> import base64, json, html
>>> s = """{"algo":"SHA1","counter":0,"digits":6,"imageAlt":"content://com.google.android.apps.photos.contentprovider/-1/1/content%3A%2F%2Fmedia%2Fexternal%2Ffile%2F3195/ORIGINAL/NONE/741876674","issuerExt":"Discord","issuerInt":"Discord","label":"[email protected]","period":30,"secret":[122,-15,11,51,-100,-109,21,89,-30,-35],"type":"TOTP"}"""

We decode all those HTML entities from the XML encoding:

>>> s = html.unescape(s); print(s)
{"algo":"SHA1","counter":0,"digits":6,"imageAlt":"content://com.google.android.apps.photos.contentprovider/-1/1/content%3A%2F%2Fmedia%2Fexternal%2Ffile%2F3195/ORIGINAL/NONE/741876674","issuerExt":"Discord","issuerInt":"Discord","label":"[email protected]","period":30,"secret":[122,-15,11,51,-100,-109,21,89,-30,-35],"type":"TOTP"}

What we specifically need from this is the secret. It's a signed byte array from Java... Let's grab it:

>>> token = json.loads(s); print(token["secret"])
[122, -15, 11, 51, -100, -109, 21, 89, -30, -35]

Now we have to turn this into a Python bytestring. For that, these bytes need to be turned back into unsigned bytes. Let's go:

>>> secret = bytes((x + 256) & 255 for x in token["secret"]); print(secret)
b'z\xf1\x0b3\x9c\x93\x15Y\xe2\xdd'

Finally, the TOTP standard uses base32 strings for TOTP secrets, so we'll need to turn those bytes into a base32 string:

>>> code = base64.b32encode(secret); print(code.decode())
PLYQWM44SMKVTYW5

There we go. PLYQWM44SMKVTYW5 is our secret in a format we can manually input into FreeOTP or Keepass.

Thank you so much, guys, for figuring this one out!

Here is the full description of what I did

I had my app installed, so I didn't need to use ADB. I exported the backup file from the working FreeOTP using the "Backup" corner menu option. Then using this code I extracted the content that was almost JSON :)):

import java.io.*;
import java.util.HashMap;

public class DeserializeOTP {
    public static void main(String[] args) {
        try (ObjectInputStream ois = new ObjectInputStream(new FileInputStream("otp_backup.xml"))) { // here put the name of your backup file
            Object obj = ois.readObject();
            if (obj instanceof HashMap) {
                HashMap<?, ?> map = (HashMap<?, ?>) obj;
                for (Object key : map.keySet()) {
                    System.out.println(key + " => " + map.get(key));
                }
            }
        } catch (Exception e) {
            e.printStackTrace();
        }
    }
}

The above text save in the file DeserializeOTP.java and then run the following shell command:

javac DeserializeOTP.java && java DeserializeOTP

The resulting string I did half-assedly convert to json with these commands in nodejs (run node in the shell):

str = `<COPY THE OUTPUT OF THE PREVIOUS COMMAND HERE>`
replaced_arrows = str.replace(/(.+)\s=>/g, "\"$1\":")
replaced_newline = replaced_arrows.replace(/\n/g, ",\n")
jsonAttempt = `{${replaced_newline}}`
attempt = jsonAttempt.replace(/"\{"mCipher/g, "{\"mCipher")
fs = require("fs")
fs.writeFileSync("./backup.json", attempt)

Then I opened backup.json in my editor with syntax highliting and fixed a couple of issues that remained until the file parsed correctly.

Lastly I edited the scripts above to prepare all of my OTP keys at once for import into Aegis(ty @malcoriel for the suggestion, the app is really nice)

import json
import base64
from cryptography.hazmat.primitives.ciphers import Cipher, algorithms, modes
from cryptography.hazmat.primitives.kdf.pbkdf2 import PBKDF2HMAC
from cryptography.hazmat.primitives import hashes
from cryptography.hazmat.backends import default_backend

bytelist_to_bytes = lambda bytelist: bytes((x + 256) & 255 for x in bytelist)

def decrypt(cipher_text, key, parameters, aad):
    # paramerts is in asn.1 encoding with ivlen being optional and only included if != 12
    # => following code handles only 16-byte iv's
    iv = parameters[4:16]
    ivlen = ord(parameters[18:])
    cipher_text, tag = cipher_text[:-16], cipher_text[-16:]
    cipher = Cipher(
        algorithms.AES(key), modes.GCM(iv, tag=tag), backend=default_backend()
    )
    decryptor = cipher.decryptor()
    decryptor.authenticate_additional_data(aad)
    return decryptor.update(cipher_text) + decryptor.finalize()


password = b"<password>"

file = open("backup.json", "r")

json_data = json.loads("\n".join(file.readlines()))

master_key_data = json_data["masterKey"]
mEncryptedKey_cipherText = bytelist_to_bytes(
    master_key_data["mEncryptedKey"]["mCipherText"]
)
mEncryptedKey_parameters = bytelist_to_bytes(
    master_key_data["mEncryptedKey"]["mParameters"]
)
mIterations = master_key_data["mIterations"]
mSalt = bytelist_to_bytes(master_key_data["mSalt"])

kdf = PBKDF2HMAC(
    algorithm=hashes.SHA512(),
    length=32,
    salt=mSalt,
    iterations=mIterations,
    backend=default_backend(),
)
master_key = kdf.derive(password)

mToken = master_key_data["mEncryptedKey"]["mToken"].encode()

# Decrypt master_key
decrypted_master_key = decrypt(
    mEncryptedKey_cipherText,
    master_key,
    mEncryptedKey_parameters,
    mToken,
)

for key in json_data:
    if not key.endswith("-token") and key != "masterKey":
        totp_key_data = json_data[key]
        print("Metadata", json_data[key + "-token"])
        key_cipherText = bytelist_to_bytes(totp_key_data["key"]["mCipherText"])
        key_parameters = bytelist_to_bytes(totp_key_data["key"]["mParameters"])
        decrypted_totp_secret = decrypt(
            key_cipherText,
            decrypted_master_key,
            key_parameters,
            totp_key_data["key"]["mToken"].encode(),
        )

        print(
            "Decrypted TOTP secret:",
            base64.b32encode(decrypted_totp_secret).decode(),
        )