Using encrypted private keys with Golang HTTPS server

Coming from a Java world (I am embarrassed enough), this hit me like a wall. Default golang does not have a way to accept private keys that are protected by a passphrase. If you look up the documentation for , it needs 2 parameters, the certFile and keyFile strings, that represent the location of PEM encoded format for certificate and private key. There is no option to supply the password !!

Another weird thing I saw is there is field in the struct, it has a parameter that too has a field to configure the object and that is not just for Certificates but for PrivateKey as well. I thought, why does it need the parameters in server.ListenAndServerTLS when it already has it. I got into the messy part because I can’t read documentation.

func (srv *Server) ServeTLS(l net.Listener, certFile, keyFile string) error { // Setup HTTP/2 before srv.Serve, to initialize srv.TLSConfig // before we clone it and create the TLS Listener. if err := srv.setupHTTP2_ServeTLS(); err != nil {  return err }

config := cloneTLSConfig(srv.TLSConfig) if !strSliceContains(config.NextProtos, "http/1.1") {  config.NextProtos = append(config.NextProtos, "http/1.1") }

configHasCert := len(config.Certificates) > 0 || config.GetCertificate != nil if !configHasCert || certFile != "" || keyFile != "" {  var err error  config.Certificates = make([]tls.Certificate, 1)  config.Certificates[0], err = tls.LoadX509KeyPair(certFile, keyFile)  if err != nil {   return err  } }

 tlsListener := tls.NewListener(l, config) return srv.Serve(tlsListener)}

It is first checking the TLSConfig that we might have supplied, if not then it would use the supplied cert and key file names.

It occurred to me, why not we try to build the with the encrypted key file that we have.

Let’s get to some action

How do you load a plain cert and key files to build a Certificate Object

The first one takes file names and second one takes PEM encoded blocks. First is no use, as it is the same as the Server interface. The second one looks interesting. Let’s try to get the private key as a PEM block.

But first, What is PEM encoding. There is a very interesting and intimidating thing called ASN.1 that can be used to serialise structures and cryptographic implementation uses the DER encoding rules to store the ASN.1 structures.

A basic guide for layman can be found here: ASN.1 and surprisingly here: DER encoding for ASN.1

For now just think of those as serialization techniques and know that DER is a binary encoding scheme or it produces binary output. Which is ok to store on disk. But when you have to transfer it over emails, for which PEM (Privacy enhanced mails) was originally built, which is plain text, you need to make it a little simpler.

Here comes the base64 cannon. Another encoding scheme on top of an encoding scheme (I know, right !!). It can represent any binary data as printable characters. eg:

$ echo hello | xxd00000000: 6865 6c6c 6f0a                           hello.

$ echo -ne "\x68\x65\x6c\x6c\x6f\x0a" | base64aGVsbG8K

$ echo "aGVsbG8K" | base64 -Dhello

Yay ! So now you can transmit binary data in a printable format.

PEM is the fancy name for this base64 encoding of DER format. Try this for fun, it’s pure pleasure :D

$ wget https://secure.globalsign.net/cacert/Root-R1.crt$ cat Root-R1.crt | base64MIIDdTCCAl2gAwIBAgILBAAAAAABFUtaw5QwDQYJKoZIhvcNAQEFBQAwVzELMAkGA1UEBhMCQkUxGTAXBgNVBAoTEEdsb2JhbFNpZ24gbnYtc2ExEDAOBgNVBAsTB1Jvb3QgQ0ExGzAZBgNVBAMTEkdsb2JhbFNpZ24gUm9vdCBDQTAeFw05ODA5MDExMjAwMDBaFw0yODAxMjgxMjAwMDBaMFcxCzAJBgNVBAYTAkJFMRkwFwYDVQQKExBHbG9iYWxTaWduIG52LXNhMRAwDgYDVQQLEwdSb290IENBMRswGQYDVQQDExJHbG9iYWxTaWduIFJvb3QgQ0EwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDaDuaZjc6j40+Kfvvxi4Mla+pIH/EqsLmVEQS98GPR4mdmzxzdzxtIK+6NiY6arymAZavpxy0Sy6scTHAHoT0KMM0VjU/43dSMUBUc71DuxC73/OlS8pF94G3VNTCOXkNz8kHp1Wrjsok6Vjk4bwY8iGlbKk3Fp1S4bInMm/k8yuX9ifUSPJJ4ltbcdG6TRGHRjcdGsnUOhugZitVtbNV4FpWi6cgKOOvyJBNPc1STE4U6G7weNLWLBYy5d4ux2x8gkasJU26Qzns3dLlwR5EiUWMWea6xrkEmCMgZK9FGqkjWZCrXgzT/LCrBbBlDSgeF59N89iFo7+ryUp9/k5DPAgMBAAGjQjBAMA4GA1UdDwEB/wQEAwIBBjAPBgNVHRMBAf8EBTADAQH/MB0GA1UdDgQWBBRge2YaRQ2XyolQL30EzTSo//z9SzANBgkqhkiG9w0BAQUFAAOCAQEA1nPnfE920I2/7LqivjTFKDK1fPxsnCwrvQmeU79rXqoRSLblCKOzyj1hTdNGCbM+w6DjY1Ub8rrvrTnhQ7k4o+YviiY776BQVvnGCv04zcQLcFGUl5gE38NflNUVyRRBnMRddWQVDf9VMOyGj/8N7yy5Y0b2qvzfvGn9LhJIZJrglfCm7ymPAbEVtQwdpf5pLGkkeB6zpxxxYu7KyJesF12KwvhHhm4qxFYxldBniYUr+WymXUadDKqC5JlR3XC321Y9YeRq4VzW9v493kHMB65jUr9TU/Qr6cf9tveCX4XSQRjbgbMEHMUfpIBvFSDJ3gyICh3WZlXi/EjJKSZp4A==

$ openssl x509 -in Root-R1.crt -inform DER -outform PEM-----BEGIN CERTIFICATE-----MIIDdTCCAl2gAwIBAgILBAAAAAABFUtaw5QwDQYJKoZIhvcNAQEFBQAwVzELMAkGA1UEBhMCQkUxGTAXBgNVBAoTEEdsb2JhbFNpZ24gbnYtc2ExEDAOBgNVBAsTB1Jvb3QgQ0ExGzAZBgNVBAMTEkdsb2JhbFNpZ24gUm9vdCBDQTAeFw05ODA5MDExMjAwMDBaFw0yODAxMjgxMjAwMDBaMFcxCzAJBgNVBAYTAkJFMRkwFwYDVQQKExBHbG9iYWxTaWduIG52LXNhMRAwDgYDVQQLEwdSb290IENBMRswGQYDVQQDExJHbG9iYWxTaWduIFJvb3QgQ0EwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDaDuaZjc6j40+Kfvvxi4Mla+pIH/EqsLmVEQS98GPR4mdmzxzdzxtIK+6NiY6arymAZavpxy0Sy6scTHAHoT0KMM0VjU/43dSMUBUc71DuxC73/OlS8pF94G3VNTCOXkNz8kHp1Wrjsok6Vjk4bwY8iGlbKk3Fp1S4bInMm/k8yuX9ifUSPJJ4ltbcdG6TRGHRjcdGsnUOhugZitVtbNV4FpWi6cgKOOvyJBNPc1STE4U6G7weNLWLBYy5d4ux2x8gkasJU26Qzns3dLlwR5EiUWMWea6xrkEmCMgZK9FGqkjWZCrXgzT/LCrBbBlDSgeF59N89iFo7+ryUp9/k5DPAgMBAAGjQjBAMA4GA1UdDwEB/wQEAwIBBjAPBgNVHRMBAf8EBTADAQH/MB0GA1UdDgQWBBRge2YaRQ2XyolQL30EzTSo//z9SzANBgkqhkiG9w0BAQUFAAOCAQEA1nPnfE920I2/7LqivjTFKDK1fPxsnCwrvQmeU79rXqoRSLblCKOzyj1hTdNGCbM+w6DjY1Ub8rrvrTnhQ7k4o+YviiY776BQVvnGCv04zcQLcFGUl5gE38NflNUVyRRBnMRddWQVDf9VMOyGj/8N7yy5Y0b2qvzfvGn9LhJIZJrglfCm7ymPAbEVtQwdpf5pLGkkeB6zpxxxYu7KyJesF12KwvhHhm4qxFYxldBniYUr+WymXUadDKqC5JlR3XC321Y9YeRq4VzW9v493kHMB65jUr9TU/Qr6cf9tveCX4XSQRjbgbMEHMUfpIBvFSDJ3gyICh3WZlXi/EjJKSZp4A==-----END CERTIFICATE-----

Look at both of the outputs, They are the same, with a better formatting though. And the

-----BEGIN CERTIFICATE-----...-----END CERTIFICATE-----

part is according to the RFC which tells about the type of the PEM block.

Coming back to our problem,

If we can get golang a PEM block that has the private key unencrypted, bingo. First let us get the PEM block in golang Native structs.

has a useful function: . This function goes through the bytes and tries to find the marker of the PEM block and returns the first block as a pointer to

-----BEGIN Type-----Headersbase64-encoded Bytes-----END Type-----

Let’s try to see how to write this thing.

Not bad, but we still haven’t reached the magic yet.

Another amazing function, a pair actually, that golang’s crypto library provides is

Check if a PEM block is encrypted, if yes, decrypt it. Yes, it’s that awesome.

All you need now is check if the detected Private key block is encrypted, If yes, decrypt it.

And, that was it.

If you would have seen the signature of the tls.X509KeyPair(certPEMBlock, keyPEMBlock []byte) all they need is a PEM block as bytes. And already has a Block.Byte Value. What’s this for ?

This is the DER encoding of the ASN.1 structure and we just don’t need bytes in the function, but a well formed PEM block with all the markers and headers (yes, that took me a couple of hours and some facepalms to find out, as I read documentation as carefully as people think about their passwords.)

Now, how do you protect the password to the encrypted private key, is another problem altogether and my face already has enough finger marks.