HTTPS, TLS, and SSL “trust” explained

TLS is a fundamental part of networking that all software engineers should understand at a high level. I think the key detail to understand is where does “trust” come from and what’s the basis for that trust?

In HTTPS, the “S” means “secure”, i.e. the secure version of HTTP. And you’ll see the same with a lot of other non-HTTP protocols, just suffixed by “s”. In all of these cases, secure means encrypted by Transport Layer Security (TLS), formerly know as “SSL”. I’m assuming the reader is familiar already with TCP, in short it’s a way to talk between a client and server over an unreliable network. TLS is TCP plus encryption, and so this is not an HTTP-specific post, it’s generic to any TLS connection.

The basis for TLS is public / private key cryptography, aka asymmetric cryptography. To quickly explain this concept, if you can lock (i.e. encrypt) a file with a password, then unlock it with the same password, this is symmetric. If you can lock a file with one password, but you need a different password to unlock it, that’s asymmetric cryptography. In this analogy, the password you use to lock files is the public key, and the password you use to unlock is the private key. The private key and the corresponding public key are known as a key pair. Also, an important detail is that in some asymmetric encryption it works both ways, so you can encrypt a message with either the public or private key, and only the other key can unlock it.

This property of asymmetric crypto is very useful. For example, this is the basis of “digital signatures”. Let’s say I want to send a message and prove I’m the one who sent it, to someone who already has my public key. I can write a message, then encrypt it with my private key. The recipient can then decrypt the message with my public key, which proves that the encrypted message was generated by someone who has the private key (which should be me - this is why the key is private). To avoid encrypting the entire message, which in practice could be large and possibly computationally expensive to encrypt or decrypt, a digital signature is often done by using a secure hash function on the message body (hashing out of scope for this post), then encrypting the hash with the private key. This allows the message recipient to decrypt the hash with the public key, then hash the message body and check it matches the signature. For example, this is how a JSON Web Token works. If you decode the token, you find the body of the token, with digital signature appended.

In the above explanation, there is a key detail that was ignored - how can the message recipient receive my message securely if they don’t already have my public key? Well, the answer to this is key to understand in HTTPS and it’s because the public key is literally already installed on your computer when it comes from the factory, or it’s physically present in your installation media when you are installing an operating system. This system is an industry standard around TLS and it’s called Public Key Infrastructure (PKI). There are a set of Certificate Authorities (CAs) that everyone’s computers “trust” because everyone already has the public key of these CAs installed on their system. These CAs are trusted organizations like Microsoft, Amazon, Let’s Encrypt and so on. The vast majority of websites are not publicly trusted CAs, instead they are trusted by clients’ machines because a CA digitally signed, i.e. approved, their public key (aka certificate). This is like when you bring your friend to a party, the host trusts your friend because they are with you, and you can say who they are with conviction. On the web, this works like as a domain owner you send your public key with some metadata on it (like your domain names) and the CA will sign it to indicate it’s valid and should be trusted. The CA will require the domain owner to prove they own the website domain. Generally it’s done with a “challenge”, and is covered by the “ACME” protocol. The CA will challenge the domain owner like “if you own the domain, then host this random string with HTTP” or “if you own this domain, then add this random string to your DNS settings”. The CA checks the challenge matches, which proves you own the domain name, then they return you your signed certificate. As a web user, if you connect to an HTTPS service you may sometimes see a message about a trust issue like “site not trusted”. That means this site is presenting a certificate that is not signed by a public key you have in your computer already, and it means you should not trust that the server you connect to is actually corresponding to the domain name in your browser. When you click “trust anyways”, it will download the certificate and add it to the list of certificates that your computer trusts by default. This happens in local development if you “self sign” a certificate, which means the digital signature is from yourself. If you are troubleshooting a problem where this shouldn’t be happening, it’s probably because your certificate (public key plus metadata) does not have the right domain name in the metadata matching the domain name in the browser search bar, or the certificate is expired.

So, when you connect to a website with HTTPS, this is what happens. You perform a DNS lookup of the domain name, which gives you an IP address. You connect to the server using this IP address, then a TCP handshake is performed, followed by a TLS handshake. You can find info about those online, but in the TLS handshake, one of the things that happens is that the server gives the client its certificate, which is a bunch of metadata, a public key, and a digital signature from a CA. This allows the client to trust the server. Then, the client will send messages to the server by encrypting them with the public key of the server. There are some more details here, like session keys, client keys, mutual TLS and so on, but from a high level, that’s how it works.