What is an Ethereum keystore file?

The barrier to entry to manage your Ethereum private keys is high, mostly because Ethereum clients hide a big part of the cryptographic complexity under straightforward command line or graphical interfaces.

For example, using geth:

$ geth account new
Your new account is locked with a password. Please give a password. Do not forget this password.
Passphrase:
Repeat passphrase: 
Address: {008aeeda4d805471df9b2a5b0f38a0c3bcba786b}

$ geth account list
Account #0: {8a1c4d573cc29a96547816522cfe0b266e88abac} keystore:~/.ethereum/keystore/UTC--<created_date_time>--  008aeeda4d805471df9b2a5b0f38a0c3bcba786b

I only had to type 3 words to create a new account. I was then asked to enter a passphrase twice and that’s it! My Ethereum keystore file was created.

Those very precious keystore files need to be backed up and stored in one or multiple secret locations so that you and only you can access these files to retrieve your funds.

From experience, when I don’t fully understand the subtleties of a new concept and rely too much on levels of abstraction and existing tools to make my life easier, I am more likely to forget about something, take an unnecessary shortcut and screw up. And when it happens, I may end up with my hard-earned Ether locked forever (thankfully it hasn’t happened yet!

Fortunately, there aren’t that many ways that you, an Ethereum user, can screw up:

1. You’ve lost your keystore file;
2. You’ve forgotten the password associated with the file;
3. Both.

In this article, we will go through how your Ethereum private key is computed from your Ethereum keystore file. We’ll talk about cryptographic functions (symmetric encryption, key derivation functions, SHA3 hashes) but we’ll keep the explanation as accessible and straightforward as possible.

What is a keystore file?

An Ethereum keystore file (stored in ~/.ethereum/keystore on Linux or C:\Users\<User>\Appdata/Roaming/Ethereum/keystore on Windows) is an encrypted version of your unique Ethereum private key that you will use to sign your transactions. If you lose this file, you lose access to your unique private key which means you lose the ability to sign your transactions — which means your funds are stuck in your account forever.

Of course, you could directly store your Ethereum private key in an encrypted file but it would make your key vulnerable to attackers that could simply read the file and use your private key to sign transactions to their account. Your Ether would be gone in less time that it’d take you to realise what had happened!

That’s why the Ethereum keystore files were created: they allow you to store your Ethereum private key encrypted under your passphrase. It’s the perfect compromise between security (an attacker would need the keystore file and your password to steal your funds) and usability (you only need the keystore file and your password to use your money).

To let you send some Ether, most Ethereum clients will ask you to type your passphrase (the same one you used when you created your account) in order to decrypt your Ethereum private key. Once decrypted, the private key is available to the client program to sign your transactions and let you move your fund.

What do keystore files look like?

If you were to open one of your account’s file, it would look like this (taken from here):

$ cat ~/.ethereum/keystore/UTC--<created_date_time>--  008aeeda4d805471df9b2a5b0f38a0c3bcba786b
{
    "crypto" : {
        "cipher" : "aes-128-ctr",
        "cipherparams" : {
            "iv" : "83dbcc02d8ccb40e466191a123791e0e"
        },
        "ciphertext" : "d172bf743a674da9cdad04534d56926ef8358534d458fffccd4e6ad2fbde479c",
        "kdf" : "scrypt",
        "kdfparams" : {
            "dklen" : 32,
            "n" : 262144,
            "r" : 1,
            "p" : 8,
            "salt" : "ab0c7876052600dd703518d6fc3fe8984592145b591fc8fb5c6d43190334ba19"
        },
        "mac" : "2103ac29920d71da29f15d75b4a16dbe95cfd7ff8faea1056c33131d846e3097"
    },
    "id" : "3198bc9c-6672-5ab3-d995-4942343ae5b6",
    "version" : 3
}

A chunky JSON file with many magic parameters that seem to be vaguely related to complicated cryptographic operations. Definitely not appealing.

Let’s dig a little deeper

If you look at the structure of that keystore file, you can see that most of its content is under “crypto”:

"crypto" : {
    "cipher" : "aes-128-ctr",
    "cipherparams" : {
     "iv" : "83dbcc02d8ccb40e466191a123791e0e"
    },
    "ciphertext" : "d172bf743a674da9cdad04534d56926ef8358534d458fffccd4e6ad2fbde479c",
    "kdf" : "scrypt",
    "kdfparams" : {
     "dklen" : 32,
     "n" : 262144,
     "r" : 1,
     "p" : 8,
     "salt" : "ab0c7876052600dd703518d6fc3fe8984592145b591fc8fb5c6d43190334ba19"
    },
    "mac" : "2103ac29920d71da29f15d75b4a16dbe95cfd7ff8faea1056c33131d846e3097"
   },

This contains:

• cipher: The name of a symmetric AES algorithm;
• cipherparams: The parameters required for the “cipher” algorithm above;
• ciphertext: Your Ethereum private key encrypted using the “cipher” algorithm above;
• kdf: A Key Derivation Function used to let you encrypt your keystore file with a password;
• kdfparams: The parameters required for the “kdf” algorithm above;
• mac: A code used to verify your password;

Let’s see how these work together to keep your protect your keystore file under your passphrase.

1. Encrypting your private key

As mentioned earlier, an Ethereum account is a private-public key pair used to cryptographically sign your transactions. To make sure that your private key is not stored in a file as plain (i.e. readable by anyone who has access to the file), it is crucial to encrypt it using a strong symmetric algorithm (cipher).

These symmetric algorithms use a key to encrypt some data. The resulting data is encrypted and can be decrypted using the same method and the same key — hence the name, “symmetric”. For this article, let’s call this symmetric key decryption-key as it will be used to decrypt our Ethereum private key.

And this is what cipher, cipherparams and ciphertext correspond to:

• cipher is the symmetric algorithm used to encrypt the Ethereum private key. Here, the value of cipher is aes-128-ctr.
• cipherparams are the parameters required for the aes-128-ctr algorithm. Here, the only parameter is iv, an “initialisation vector” consumed by the aes-128-ctr algorithm.
• ciphertext is the encrypted input of the aes-128-ctr function.

So here you go, you’ve got everything you need to compute your decrypted Ethereum private key… wait. You need to retrieve your decryption-key first.

2. Protecting it with your passphrase

To make sure unlocking your account is easy, you don’t need to remember your very long and non-user-friendly decryption-key that is used to decrypt ciphertext. Instead, the Ethereum developers have opted for a passphrase-based protection — that is, you just need to type your passphrase to retrieve your decryption-key.

To be able to achieve this, Ethereum uses a Key Derivation Function that computes the decryption-key given a passphrase and a list of parameters.

This is what kdf and kdfparams are used for:

• kdf is the Key Derivation Function used to compute (or “derive”) the decryption-key from your passphrase. Here, the value of kdf is scrypt.
• kdfparams are the parameters required for the scrypt function. Here, without going into too much detail, dklen, n, r, p and salt are the parameters of the kdf function. More information on the scrypt function can be found here.

Here you go, tweaking the scrypt function with the kdfparams parameters and feeding it our passphrase, you get our decryption-key as the output of the Key Derivation Function.

3. Make sure your passphrase is right

We’ve described everything we need to determine our Ethereum private key from our keystore file and our passphrase. However, what happens if the passphrase supplied to unlock the account is wrong?

From what we’ve seen so far, all the operations (derivation and decryption) will succeed, but the Ethereum private key computed at the end won’t be right — which defies the point of using the keyfile in the first place!

We need to guarantee that the passphrase typed to unlock the account is right, that it is the same one as the one entered when the keystore file was originally created (remember geth account new?).

This is where the mac value in the keystore file comes into play. Just after the Key Derivation Function is executed, its result (decryption-key) and ciphertext are processed (*) and compared to mac (which acts like a seal of approval). If the result is the same as mac, then the passphrase was right and the decryption can start.

(*): A little bit of a shortcut here. The decryption-key (second leftmost 16 bytes only) and ciphertext are concatenated and hashed (SHA3–256) before being compared to mac. More info here.

Putting everything together

Phew! If you’ve made it this far, then congratulations!