How HTTPS actually works, from a typed name to an encrypted connection

You type a hostname into a browser, the little lock icon (the padlock) appears in the address bar, and a page loads. It feels like one thing. It is really a short, ordered sequence of independent systems, each of which has to be correct, and each of which fails in its own recognizable way.

This post explains HTTPS at two levels, easy first. The first half is the operational view: the three systems that have to line up for an endpoint to serve a request, and the distinct failure signature of each. That is the practical, 2am-debugging model, and for a lot of people it is all they need. The second half goes under the hood into what actually happens on the wire: recursive DNS, the TLS handshake, and how a certificate is really validated. If you only want the debugging model, stop after Part 1. If you want the protocol, skip ahead to Part 2.

I spend a lot of my time at exactly this layer (domain resolution, certificate issuance, and TLS termination on a large multi-region platform), and the whole thing is far less mysterious than it looks once you watch it run in order. No networking background assumed: if you have ever typed a URL, you know enough.

Part 1: the operational view

For https://app.example.com to actually serve a request, three independent things must all be true at once:

  1. DNS resolves. The name app.example.com has to translate into an address your client can connect to.
  2. A certificate covers the host. Whatever terminates the encrypted connection has to present a TLS certificate whose list of valid names includes app.example.com.
  3. A route matches the host. Once the connection lands on a gateway, something has to look at which hostname was requested and send it to the right backend.
flowchart LR
    C["Client"] -->|"1. resolve name"| D[("DNS")]
    D -->|"address"| C
    C -->|"2. TLS handshake"| G["Gateway"]
    G -.->|"cert SAN must<br/>cover the host"| G
    G -->|"3. match Host header"| B[("Backend service")]

Three systems, usually owned by three different teams, often in three different tools. The reason production debugging is hard is that the symptom (“I cannot reach the site”) is identical no matter which of the three broke. The skill is reading the distinct failure signature of each.

Ingredient 1: DNS that resolves

DNS is the phone book that turns a name into an address. In production it is rarely a single lookup. A name like app.example.com is often a CNAME (an alias) that points at another name, which points at another, until you finally reach a terminal A or AAAA record holding an actual IP address. Each hop has a TTL, a cache lifetime, which is why DNS changes are not instant: the old answer lives in resolvers around the world until its TTL expires.

The failure signature is specific and recognizable: the connection never even starts. You get “server not found” or a hang before any handshake. Crucially, load balancers and traffic directors constantly run health checks: small probes that mark each target healthy (it responds correctly) or unhealthy (it is down, erroring, or unreachable), so traffic is only sent to the healthy ones. If one of those checks reaches a target by name and that name has no DNS record yet, the target looks unhealthy for a reason that has nothing to do with the service itself. It is simply unresolvable. I have watched a perfectly healthy fleet of endpoints report as degraded purely because their names had not been published in DNS yet. The service was fine. The phone book did not have its number.

A detail that bites people: the same name can resolve to different addresses depending on where you ask, a setup called split-horizon DNS, where an internal resolver and a public resolver give different answers for the same hostname. So “it resolves for me” is not the same as “it resolves,” and the first question in a DNS incident is always “resolves from where?”

Ingredient 2: a certificate whose names cover the host

When the encrypted connection is established, the server presents a TLS certificate, and the client checks one thing above all: does this certificate actually claim to be valid for the hostname I asked for? That claim lives in a field called the SAN (Subject Alternative Name), a list of hostnames the certificate is good for. If app.example.com is not in the SAN list, the browser refuses the connection and shows a scary warning, no matter how healthy everything behind it is.

The failure signature is the opposite of the DNS one: the connection starts, then is rejected at the handshake with a certificate error. That alone tells you DNS worked (you reached something) but the certificate is wrong or missing for this host.

One detail here is responsible for an enormous share of real-world certificate incidents. Wildcards cover exactly one level. A wildcard certificate for *.example.com covers app.example.com and api.example.com, but it does not cover the apex example.com itself, and it does not cover app.internal.example.com (that is two levels deep). The wildcard is one label to the left of the part you wrote, no more. Naming schemes have to be designed around this limit, not discovered to violate it in production. (There is a second, deeper certificate trap, “internal does not mean private,” that I save for Part 2 because it is really about how the trust chain works.)

Ingredient 3: a gateway route that matches the host

Now the connection has landed on a gateway with a valid certificate. The gateway is typically serving many hostnames from the same IP address, so it has to decide where this particular request goes. It uses two pieces of the request to do that. During the TLS handshake, SNI (Server Name Indication) tells the gateway which hostname the client is asking for, so the gateway can select the right certificate before encryption is even established. Then, inside the request, the Host header says the same name again, and the gateway matches it to a route pointing at a backend.

The failure signature is different again: DNS resolved, the certificate was accepted, the connection is fully established, and then you get an error from the gateway itself, a 404 or a 502 or a default landing page, because no route matched this host. This is the “everything works except it goes nowhere” failure, and it is the one people misdiagnose most, because the padlock is green and the connection is real. The problem is purely that the last hop has no instruction for this name.

The diagnostic shortcut

Once you internalize the three signatures, you can localize most “site is down” reports in seconds, before opening a single dashboard:

What you observe Which ingredient failed
Name does not resolve; connection never starts DNS
Connection starts, then a certificate warning Certificate / SAN
Connection completes, padlock is fine, but you get a 404/502 from the edge Gateway route

Three systems, three owners, three tools, three signatures. The reason this framing is worth carrying around is that it converts a vague, stressful “it is broken” into a precise, almost mechanical question: which of the three, and the evidence is already in the error.

Why global scale makes this sharper, not fuzzier

In a multi-region platform, the same hostname is served from many places, and a DNS-level traffic director (think a global load balancer like Azure Traffic Manager or latency-based routing in AWS Route 53) decides which region a given client is sent to. The important and frequently-missed detail is that this director works at the DNS layer. It answers the name lookup with the address of a healthy nearby region, and then it steps out of the way. It is not in the request path. The split is clean: the traffic director picks which region and runs only once, at name-lookup time; the regional gateway then picks which backend and runs on every single request.

That single fact has a big operational consequence: you cannot read per-request logs at the traffic director, because requests never flow through it. It emits health-probe and traffic-distribution signals, but the actual request logs live at the regional gateway, keyed by hostname. People burn hours looking for request data at the global layer that was never there to find. Knowing exactly where each signal lives, resolution at DNS, request logs at the gateway, certificate state at the binding, is the difference between a five-minute diagnosis and a five-hour one.

This connects directly to something I have written about before: a system is only as debuggable as the telemetry it is willing and able to tell you about itself. The three-ingredient model is useful precisely because each ingredient emits its own kind of evidence, and good infrastructure makes that evidence easy to read at the layer where it actually exists.

Part 2: under the hood

The operational view treats each system as a box. This half opens the boxes and walks the actual protocol, in the order it runs, in the second or so between pressing Enter and seeing the padlock.

The whole sequence in one breath

When you load https://app.example.com, four things happen in strict order:

  1. Resolve the name to an IP address (DNS).
  2. Open a transport connection to that address (TCP).
  3. Negotiate encryption and verify identity (the TLS handshake).
  4. Send the actual request over the now-encrypted channel (HTTP).

Steps 1 and 3 are where almost all of the interesting machinery lives.

Step 1: resolving the name

Your machine does not know where app.example.com lives, and it does not ask one server that magically knows. It asks a recursive resolver (usually run by your ISP or a public one like 1.1.1.1), and that resolver walks a hierarchy on your behalf:

sequenceDiagram
    participant C as Your machine
    participant R as Recursive resolver
    participant Root as Root servers
    participant TLD as .com servers
    participant Auth as example.com authoritative
    C->>R: where is app.example.com?
    R->>Root: who handles .com?
    Root-->>R: ask the .com servers
    R->>TLD: who handles example.com?
    TLD-->>R: ask example.com authoritative
    R->>Auth: where is app.example.com?
    Auth-->>R: CNAME or A record
    R-->>C: final IP address

This is the same DNS from Part 1, seen from the inside. The CNAME chain, the TTL caching, and split-horizon resolution all live in this walk. The global traffic director from Part 1 plugs in right here: it is the authoritative answer at the bottom, handing back the address of a healthy nearby region and then stepping out of the request path entirely. If this step fails, the connection never even starts, because there is nothing to connect to yet.

Step 2: opening the connection

Now your machine has an IP address, but a bare address is just a destination, not a live conversation. Before any HTTPS can happen, the two machines have to confirm they are both present and ready to exchange a reliable, ordered stream of bytes. That job belongs to TCP (Transmission Control Protocol), the layer that turns a raw IP address into a dependable two-way pipe where bytes arrive in order and nothing is silently dropped.

Opening that pipe is the three-way handshake: three short messages that get both sides in sync before any real data flows.

sequenceDiagram
    participant C as Client
    participant S as Server (port 443)
    C->>S: SYN (I want to talk, here is my starting sequence number)
    S->>C: SYN-ACK (I am here, and I acknowledge yours)
    C->>S: ACK (acknowledged, we are connected)
    Note over C,S: reliable byte pipe is now open (still unencrypted)

For HTTPS this connection is made to port 443, the standard port reserved for it (plain, unencrypted HTTP uses port 80). A port is just a numbered door on the server: one IP address can run many services at once, and the port number says which one you are knocking on.

When the handshake completes you have a reliable byte pipe to some server, but two things are still not true. Nothing is encrypted yet, and you have no proof that the machine on the other end is really app.example.com rather than an impostor that happens to sit at that address. Those are exactly the problems the TLS handshake solves next.

This step has its own recognizable failure signature, and it is distinct from a DNS failure. The name already resolved, so you have an address; it is the connection itself that fails. You see a connection timeout (the SYN went out and nothing came back, usually a firewall, security group, or network rule silently dropping the packet) or connection refused (something answered, but nothing is listening on port 443, often the wrong port or a service that is down). The “resolves but will not connect” symptom points here, one layer below the certificate and routing problems.

Step 3: the TLS handshake

This is the heart of HTTPS, and the part people find most mysterious. It does two jobs at once: it proves the server’s identity and it agrees on a secret key that only the two of you know. Here is the modern (TLS 1.3) version, simplified to the parts worth understanding:

sequenceDiagram
    participant C as Client
    participant S as Server
    C->>S: ClientHello (SNI: app.example.com,<br/>supported ciphers, key share)
    S->>C: ServerHello (chosen cipher, key share)
    S->>C: Certificate (server's cert + chain)
    S->>C: Finished (signed, proves it holds the key)
    Note over C,S: both sides derive the same<br/>symmetric session key
    C->>S: Finished (encrypted)
    Note over C,S: application data now flows encrypted

Two things in the very first message do a lot of work:

  • SNI (Server Name Indication) is the client announcing, in the clear, which hostname it wants. This is the same SNI from Part 1’s ingredient 3: a single IP and gateway typically serve many hostnames, so the server needs this before it can even pick which certificate to present. SNI is how it chooses.
  • The key share is the client’s half of an ephemeral key exchange (Diffie-Hellman). The server sends back its half. Through a small piece of math, each side independently arrives at the same shared secret without that secret ever crossing the wire. An eavesdropper who recorded every byte still cannot reconstruct it. This is also why capturing the traffic later, even with the server’s private key, does not retroactively decrypt it: the session key was ephemeral and never transmitted.

Notice the elegant split. The expensive public-key cryptography (the certificate, the signatures) is used only to bootstrap: prove identity and safely establish a shared secret. Once that is done, all the actual data uses fast symmetric encryption with that shared key. Asymmetric to start, symmetric to run.

Step 4: validating the certificate, the chain of trust

The server handed over a certificate during the handshake. Why should the client believe it? Because the certificate is signed by someone the client already trusts, and that trust forms a chain.

flowchart TD
    Root["Root CA<br/>(in the OS/browser trust store)"] -->|signs| Inter["Intermediate CA"]
    Inter -->|signs| Leaf["Leaf certificate<br/>for app.example.com"]
    Leaf -.->|presented during handshake| Client["Client validates upward"]

The client validates the leaf certificate by walking up this chain:

  1. Does a trusted root sign it (transitively)? The leaf was signed by an intermediate CA, which was signed by a root CA. The client checks each signature in turn until it reaches a root that is already in its trust store, the set of root certificates shipped with the OS or browser. If the chain does not terminate at a trusted root, validation fails.
  2. Does the name match? The hostname you asked for (app.example.com) must appear in the certificate’s SAN list, the same SAN field from Part 1. A certificate for the wrong name is rejected even if it is perfectly valid otherwise.
  3. Is it currently valid? Not expired, not yet revoked.

This chain is the source of the deeper certificate trap I deferred from Part 1: internal does not mean private. People assume an internal-only endpoint can present a privately-issued certificate. It cannot, not if a normal client validates it, because the client’s trust store contains only public roots. A private CA has exactly one correct job: backend service-to-service authentication where both ends are deliberately configured to trust that private CA. The instant a browser or a standard client is on one end, you need a publicly-trusted issuer, internal hostname or not. Picking the wrong issuer for an internal gateway is a classic way to ship something that works in testing and fails the moment a real client connects.

It also pays to keep three PKI steps separate in your head, because conflating them produces baffling failures: authorizing a domain (allow-listing who may issue certificates for it), issuing the certificate, and binding it at the gateway. Under-scope the authorization and issuance silently fails upstream of everything else, and nothing downstream explains why. This is the machinery behind Part 1’s “certificate / SAN” failure signature.

One last thing worth being precise about, because it trips up almost everyone: what the padlock proves and what it does not. A valid certificate proves two things, that the connection is encrypted and that the server genuinely controls the exact hostname in the address bar. It does not prove that hostname is the one you meant to visit. Certificate authorities only check domain control, so an attacker who registers a lookalike like paypa1.com (digit one) or a Unicode near-twin legitimately owns that domain, can get a perfectly valid certificate for it, and will show a real green padlock. The browser’s built-in check confirms “this certificate is valid for this name,” not “this name is who you think it is.” Defenses against that are a different layer: browsers display suspicious Unicode as punycode and flag known-malicious sites through reputation services like Safe Browsing, but a brand-new typosquat with a valid certificate still gets its padlock. Encryption and identity-of-the-name are solved; is-this-the-right-name is on you.

Step 5: the request finally goes out

Only now, after the name resolved, the connection opened, the handshake completed, and the certificate validated, does your actual GET request travel down the encrypted channel, where Part 1’s third ingredient (the gateway matching the Host header to a route) finally runs. Everything that felt like a single instant was really this ordered pipeline. The padlock is just shorthand for “all of the above succeeded.”

The takeaway

An HTTPS endpoint looks like one thing and is really a short, ordered sequence: a name that resolves, a connection that opens, a handshake that proves identity and agrees on a key, a certificate that chains to a trusted root, and a route that knows where the name goes. They are independent, they are usually owned by different teams, and they fail in distinct, recognizable ways. Most production “it is down” incidents stop being mysterious the moment you ask the only question that matters first: which step broke, and the answer is almost always sitting in the exact shape of the error.

That is the same theme that runs through what I write about telemetry, infrastructure reliability, and data and AI infrastructure: systems get debuggable when you understand exactly where each responsibility and each piece of evidence actually lives. HTTPS looks like one opaque padlock, but it is a sequence of clean trust boundaries, and once you can see them separately, a whole category of scary outages turns into a short, calm checklist.

Read more like this