Setting Up a Mail Server That Doesn't End Up in Spam

"We set up Postfix and all our emails are going straight to spam" is a sentence I hear constantly. And the fix is almost never found in Postfix's configuration files.

Mail servers—Postfix, Exim, whatever you choose—are the incredibly easy part. They have been solved problems for literally decades. What actually determines whether an email lands safely in an inbox or gets aggressively dumped into a spam folder is a combination of DNS records that mathematically prove you are allowed to send mail for your domain, and a sending reputation that carefully builds up over time.

SPF: Who is allowed to send as you

SPF (Sender Policy Framework) is a DNS TXT record strictly listing which specific IP addresses are allowed to send mail claiming to be from your domain:

example.com.  TXT  "v=spf1 ip4:203.0.113.10 -all"

This effectively says: mail from example.com should absolutely only come from 203.0.113.10, and -all tells receiving servers that anything else should be aggressively rejected outright.

The most common SPF mistake is actually not a missing record. It is an SPF record that is way too permissive (~all everywhere, or it includes random third-party services you no longer use). Or, just as bad, it is a record so incredibly strict that it breaks legitimate mail from a forwarding service someone forgot to list.

DKIM: Proving the message was not altered

DKIM (DomainKeys Identified Mail) cryptographically signs outgoing messages with a private key. It publishes the matching public key directly in your DNS so receiving servers can instantly verify the signature was not tampered with in transit:

mail._domainkey.example.com.  TXT  "v=DKIM1; k=rsa; p=MIGfMA0GCSq..."

Most modern mail server software (like OpenDKIM running alongside Postfix) automatically generates this key pair for you. The actual work is publishing the public half correctly and verifying that outgoing mail is actually getting signed. That is something absolutely worth checking directly rather than just blindly assuming it works:

echo "test" | mail -s "DKIM test" [email protected]

Then, explicitly look at the received message's raw headers for dkim=pass.

DMARC: Telling receivers what to do if SPF or DKIM fail

DMARC ties SPF and DKIM together. It explicitly tells receiving servers what to do with mail that completely fails both checks, and exactly where to send daily reports about it:

_dmarc.example.com.  TXT  "v=DMARC1; p=quarantine; rua=mailto:[email protected]"

Starting with p=quarantine (send failures to the spam folder rather than rejecting them outright) instead of p=reject is a much safer rollout strategy, especially in the first few weeks. It gives you immediate visibility into anything you accidentally misconfigured without that mail simply vanishing into the void. The rua reports are genuinely useful here; they show exactly which sources are attempting to send mail as your domain and whether they are successfully passing your strict checks.

PTR records: The one DNS record you don't control

A PTR record (reverse DNS) strictly maps your server's IP address back to its hostname. A massive amount of receiving mail servers actively check that this perfectly matches the hostname your server presents in its initial SMTP greeting.

Unlike the other three records, this one is entirely controlled by your hosting provider, not in your own DNS zone. Most VPS providers have a simple control panel option or a support ticket category specifically for setting reverse DNS on an IP. If your server is proudly sending mail as mail.example.com but its IP's PTR record blindly resolves back to something ugly like vps-1234.providername.com, that mismatch alone is more than enough to get you instantly flagged by aggressive spam filters.

The part DNS records cannot fix: Reputation

Even with SPF, DKIM, DMARC, and PTR absolutely flawless, a brand-new IP address sending mail has literally zero history. Many massive receiving servers (like Gmail or Microsoft) treat unfamiliar senders highly cautiously, regardless of how technically perfect their DNS is configured.

This is where "warming up" your IP heavily matters. Start with a very low volume to addresses you are absolutely confident will actively engage with the email (open it, reply to it, not just ignore it). Increase that volume gradually rather than suddenly blasting thousands of messages from a server that sent exactly zero yesterday.

A brand new mail server with perfect DNS and zero sending history still desperately needs a slow start. Reputation is painfully built over days and weeks; it is not simply configured in a text file.

Get those four specific records exactly right. Verify that mail is actually being signed and perfectly aligning using a tool like mail-tester.com. Then, give a brand-new sending IP the actual time it needs to build a solid track record before aggressively relying on it for anything time-sensitive. That combination reliably handles the vast majority of "why is our mail going to spam" cases. Almost none of them turn out to be the mail server software itself.