SMTP, STARTTLS and MTA-STS

What the SMTP conversation looks like

SMTP (RFC 5321) starts with the server speaking first: a 220 greeting. The client replies with EHLO and its own name, and the server answers with a list of extensions. Only after that do the commands that actually move a message begin (MAIL FROM, RCPT TO, DATA). The SMTP test stops at the second step: it reads the greeting and the extension list and then sends QUIT. It does not send mail, attempt authentication, or test whether a recipient exists.

The extension list is where STARTTLS appears. If the server advertises it, a client may send the STARTTLS command and both sides then perform a TLS handshake on the same connection (RFC 3207). Everything after that is encrypted, and the client repeats EHLO.

Opportunistic TLS and why it is not enforcement

On port 25, the port servers use to deliver mail to each other, TLS is opportunistic by default. A sending server uses TLS if the receiver offers it, usually without checking whether the certificate is valid for the destination name, because failing delivery to hosts with imperfect certificates was considered worse than encrypting loosely. If TLS is not offered, the message is sent in plaintext.

Two weaknesses follow. A party who can alter traffic between the servers can delete the STARTTLS line from the reply, forcing plaintext, which is a downgrade attack. And a party who can redirect traffic can present any certificate, because it is not checked. Opportunistic TLS defends against passive eavesdropping only.

MTA-STS and DANE add enforcement

MTA-STS (RFC 8461) lets the receiving domain publish a policy that sender servers retrieve over HTTPS and cache. In enforce mode, a compliant sender must use TLS with a certificate that is valid for one of the MX names listed in the policy, or hold the message. The DNS record _mta-sts carries an id that changes when the policy changes. The MTA-STS checker fetches the policy and compares its patterns with the real MX set.

DANE (RFC 7672) reaches the same goal differently. The domain publishes TLSA records that identify the expected certificate or key for each MX host, and these records must be protected with DNSSEC. It does not rely on the public certificate authorities, but it needs DNSSEC on the zones involved. Neither mechanism is part of STARTTLS itself, and neither protects against senders that do not implement it. The STARTTLS checker in this site does not evaluate DANE.

TLS-RPT: knowing what happened

TLS-RPT (RFC 8460) is a reporting record at _smtp._tls that asks senders for daily summaries of TLS successes and failures to your domain. It is the only practical way to see whether other organisations succeed in verifying your certificates before you ask them to require it. Reports are sent by senders that support the standard; their absence does not prove there are no failures.

A safe order of work

1. Run the SMTP test and the STARTTLS checker to confirm each MX host offers STARTTLS with a modern TLS version and a certificate valid for its own name.
2. Fix certificate names, chains and expiry on every MX host, including backup hosts that receive little traffic.
3. Publish TLS-RPT with a mailbox you read, and collect reports.
4. Publish MTA-STS with mode: testing, a short max_age, and an id. Check that the policy host's certificate renews automatically.
5. Move to mode: enforce when reports show no failures, and raise max_age gradually.
6. Update the policy before changing MX hosts, and change the id each time.

Risks and rollback

The dangerous step is enforcement. If an MX host presents an expired or mismatched certificate, compliant senders hold the mail and eventually bounce it, and they continue to hold the cached policy for up to max_age. To back out, publish a policy with mode: none and a new id; senders pick this up only when they next refresh. For that reason keep max_age short until you trust the setup, and never enforce with certificates you cannot renew automatically.

What none of this proves

These checks inspect servers and DNS from one network position. A good result does not show that a particular message was encrypted, since the sender decides. Port 465 and 587 for user submission are different services with their own authentication rules, and certificate trust here is judged with the scanner's own root store.

Frequently asked questions

Written by DNS Tools editorial · Last updated 2026-10-09