Postfix, Google Apps, and You - Bigdinosaur Blog

All the cool kids have web servers, but all the REALLY cool kids have web servers with the ability to send e-mail. In days of yore, when dragons roamed the Internet and a web page with a graphical background was considered a novelty, sending e-mail from your home was as easy as setting up sendmail (or your preferred MTA) and letting ‘er rip. Things are different these days, though, as spammers long ago ruined it for everyone and made doing your own e-mail a lot more difficult.

If you’re doing your web hosting through an actual hosting provider, then chances are you already have some avenue available for your web server or its applications to send e-mail; if you’re hosting out of your basement or closet, though, like I am, it can be a little more complicated. The first obstacle is that most ISPs will block the standard SMTP TCP ports; even if you get past that (by having an unblocked business-class connection, for example), damn near every operating e-mail system in the world blacklists the IP address ranges used by ISPs for customer connections, in order to stop infected home PCs from drowning them with viagra spam.

The solution is mail relaying, where you send e-mail to a trusted e-mail server, which then sends it on to your recipient. Here again spammers have peed in the pool, because no mail server on the Internet is set up to just happily relay messages—in fact, any mail server that is configured as a so-called “open relay” will quickly find itself blacklisted by every other mail server.

But there is an easy way that your web server and its appliations can send e-mail outside of your LAN, and that way is to relay your mail through Google. Specifically, through a Google Apps Standard e-mail address.

This is a tutorial that’s been done elsewhere in lots of different ways—googling around for “postfil gmail relay” or “relay postgix through google apps” will net you scads and scads of results. I’m going to add to the pile by going through the method that I settled on. I was interested in setting up e-mail because I wanted to be notified of changes to the Bigdinosaur.org Minecraft wiki, and DokuWiki has a nice customizable notifiaction system.

You’ll Need an Account

I’m going to assume that if you’re interested in this enough to click through and read up to here, you’ve got Google Apps (standard or pro) set up already, and so you’ll need to create a mailbox for your local Postfix server to authenticate against. I end up sending most of my alerts to that account as well, so that they’re all collected in a single place without cluttering up one of my other mailboxes. So, pop over to your domain’s Google Apps control panel an make an account. Whatever name you create will be displayed in the “From” field in any e-mail relayed through the account, so you’ll want to pick something distinct but not app- or service-specific. I used “postasaurus@bigdinosaur.org,” because it fits with the domain theme and could be used by different applications as a general notification address.

You have the option of using multiple accounts, of course, but I wanted to keep things simple with my configuration, and so I’m using the single address for everything.

You’ll Need an MTA

I elected to use Postfix for my MTA, because it’s become the Internet gold standard for GNU/Linux mailers. As with most things, installing it on Ubuntu is easy:

1 $ sudo aptitude install postfix

Yes, I’m using aptitude instead of apt-get, and so should you. That’s the subject of another blog post, though.

Postfix installs itself and is immediately ready to begin slinging e-mail, but we need to do some surgery on it first—we have to tell it what it’s allowed to do, where it’s allowed to do it, and for whom. We also need some encrypted credentials for Postfix to use.

Configuring Postfix 1: Credentials

Easy part first: we’re going to add the Google Apps/Gmail server names and the mailbox you created to Postfix’s configuration and get that out of the way. Create a file named passwd in /etc/postfix/sasl (and create the directory, too, if it isn’t already there) and make the file’s contents look like this:

/etc/postfix/sasl/passwd1 2 gmail-smtp.l.google.com youraccount@yourdomain.com:password smtp.gmail.com youraccount@yourdomain.com:password

Substitute in your information, obviously, using the mailbox address and password you created in the Google Apps control panel. Once the file is saved, we’ll need to transform it into a hashed table, both so that Postfix can properly read it and also so that your mailbox name and password aren’t sitting around in a plaintext file:

1 $ sudo postmap /etc/postfix/sasl/passwd

This will create a file named passwd.db in the same directory; after that, you can delete the plaintext passwd file. Alternately, if you’d feel better keeping the file around for whatever reason, you can run chmod 600 passwd and make it only readable by its owner, which should be root.

Configuring Postfix 2: main.cf

The rest of the changes we need to make are to /etc/postfix/main.cf. Below is my main.cf, with hostnames and networks genericized. Modify the default config file to like this one, substituting in your information where required:

/etc/postfix/main.cf1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 smtpd_banner = $myhostname ESMTP $mail_name (Ubuntu) biff = no append_dot_mydomain = no readme_directory = no myhostname = yourhost.yourdomain.com alias_maps = hash:/etc/aliases alias_database = hash:/etc/aliases myorigin = /etc/mailname ## Next line tells Postfix the destinations where it will send mail directly ## instead of relaying, so that you can use Postfix to send mail to and ## from your local root account, for example. mydestination = localhost, yourhost.yourdomain.com, localhost.yourdomain.org ## Next line tells Postfix what networks it can trust. Any address or ## block of addresses listed here will be allowed to relay e-mail through ## this Postfix server and on to Google Apps. Handy if you've got a printer ## or other device that can send e-mail! mynetworks = 127.0.0.0/8 10.10.10.0/24 [::ffff:127.0.0.0]/104 [::1]/128 ## Tells Postfix which host to use when relaying mail. Google's SMTP server ## name goes here, along with the correct SMTP port. relayhost = [smtp.gmail.com]:587 relay_transport = relay relay_destination_concurrency_limit = 1 # TLS client parameters ## The certificate authority file below is in the default location for Ubuntu, ## and is used by Postfix to validate the CA used by Google's mail servers ## for when Postfix talks to them to send mail smtp_tls_CAfile = /etc/ssl/certs/ca-certificates.crt smtp_use_tls = yes smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache smtp_tls_note_starttls_offer = yes tls_random_source = dev:/dev/urandom # TLS server parameters ## This section is necessary if you're going to be using your Postfix install ## to relay from other LAN hosts smtpd_use_tls = yes smtpd_tls_cert_file = /path/to/your/certificate.crt smtpd_tls_key_file = /path/to/your/private.key smtpd_tls_CAfile = /path/to/your/ca-file-or-bundle.pem smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache smtpd_tls_loglevel = 1 smtpd_tls_received_header = yes smtpd_tls_session_cache_timeout = 3600s smtpd_tls_req_ccert = no smtpd_tls_ask_ccert = yes ## These settings tell Postfix where to find the username and password to use ## when relaying e-mail through the server set in the relayhost directive above smtp_sasl_auth_enable = yes smtp_sasl_password_maps = hash:/etc/postfix/sasl/passwd smtp_sasl_security_options = noanonymous smtp_sasl_tls_security_options = noanonymous smtp_connection_cache_destinations = smtp.gmail.com mailbox_size_limit = 0 recipient_delimiter = + inet_interfaces = all default_transport = smtp default_destination_concurrency_limit = 5

The TLS client parameters section is particularly important. The Google mail servers require you to connect with encryption, and Postfix doesn’t come with any certificate store of its own. Consequently, right out of the box, postfix won’t trust the certificate authority with which Google’s certificates are signed. The way to correct this is to tell Postfix the location of the system’s certifice authority bundle, which on Ubuntu is /etc/ssl/certs/ca-certificate.crt.

The TLS server parameters portion may be optional, depending on what you want to do. Postfix prefers encrypted connections from mail clients, and if you’ve got other devices on your LAN which might want to relay e-mail through Postfix and out using the Google Apps mailbox we’ve created—like a printer that wants to e-mail you toner notifications, or a NAS device that wants to e-mail you about failing disks—you’ll need to add SSL/TLS certificate info in here.

There are tutorials out there for generating your own self-signed certs, but in this day and age there’s no excuse for self-signed certs. There are plenty of certificate authorities who will issue you free SSL/TLS certificates that you can use, and which won’t trigger big ugly UNTRUSTED HOST warnings in browsers. I recommend using StartSSL because they’re friendly and respond extremely quickly to support questions (like, usually within minutes). Once you’ve got a certificate and key pair, stash them somewhere appropriately secured on your web server and add them to the main.cf file above in the correct spots.

And that should do it for the configuration! Restart Postfix:

1 $ sudo /etc/init.d/postfix restart

Depending on how Postfix was installed, you might get an error here that looks like this:

1 2 3 4 5 6 * Stopping Postfix Mail Transport Agent postfix postmulti: fatal: /etc/mailname: cannot open file: No such file or directory ...done. * Starting Postfix Mail Transport Agent postfix postmulti: fatal: /etc/mailname: cannot open file: No such file or directory ...done.

If so, simply create a text file called mailname under /etc/ and put in it your server’s FQDN:

/etc/hostname1 yourserver.yourdomain.com

Then restart Postfix again.

Checking to See if it All Works

To verify that Postfix can correctly send e-mail, first start watching the Postfix log, like this:

1 $ tail -f /var/log/mail.log

This will open up the Postfix log file and as new entries at added to it, it will automatically update itself in your terminal window. Open up a new window, log into the server you’ve been working on (if necessary), and try to send an e-mail with the sendmail command:

1 2 3 4 5 6 $ sendmail you@somewhere.com To: you@somewhere.com From: youraccount@yourdomain.com Subject: Testing! Hi there! Checking to see if Postfix works! .

Use your personal e-mail address with the sendmail command, since that’s where you’ll be sending the e-mail. You’ll have to type the next five lines manually. Enter the correct To: and From: addresses (and remember that the From address is going to be the mailbox you created back at the beginning), give the e-mail a Subject: and then write a brief message. End the message with a single period on a new line. Once that’s done, Postfix will attempt to deliver it, and you’ll see a bunch of activity in the log window:

1 2 3 4 5 Dec 17 20:55:28 wwwserver postfix/pickup[596]: C868A1839D3: uid=1000 from=<lee> Dec 17 20:55:28 wwwserver postfix/cleanup[637]: C868A1839D3: message-id=<20111218025528.C868A1839D3@www.bigdinosaur.org> Dec 17 20:55:28 wwwserver postfix/qmgr[597]: C868A1839D3: from=<lee@www.bigdinosaur.org>, size=292, nrcpt=1 (queue active) Dec 17 20:55:30 wwwserver postfix/smtp[639]: C868A1839D3: to=<lee@bigdinosaur.org>, relay=smtp.gmail.com[74.125.45.109]:587, delay=12, delays=11/0.05/0.71/0.68, dsn=2.0.0, status=sent (250 2.0.0 OK 1324176930 5sm34029427anz.14) Dec 17 20:55:30 wwwserver postfix/qmgr[597]: C868A1839D3: removed

If you’ve gotten everything set right, you’ll first see a notice from the Postfix pickup process, then a cleanup notice assigning it a message ID, then a note from the queue manager saying that the message has been queued for sending, then a note from the smtp process saying that it was successfully relayed through smtp.gmail.com (look for status=sent, along with the 250 2.0.0 OK part), and then finally a note from the queue manager saying that the e-mail has been removed from the mail queue.

Of course, the real confirmation will be when your e-mail shows up in your inbox!

Postfix is now set to relay e-mails through your Google Apps account, and if you have any applications on your web server that need an e-mail server address so that they can send e-mail, you can now simply give those apps the address of your web server running Postfix. Same thing for printers or other devices on your LAN—if they can send e-mail alerts, you can give them the server’s address and they’ll relay through it, and your alerts will reach you.

Obviously, this post hasn’t covered anything about receiving e-mail with Postfix. I use my Google Apps domain accounts for receiving e-mail, so I have no need for Postfix to be able to fetch any mail for me, but there are tons of other tutorials out on the Internet if you need Postfix to receive mail.


Discussion topic for the original blog entry at: http://blog.bigdinosaur.org//postfix-gmail-and-you
1 Like

This is all well and good but, why relay mail through your server to google’s servers?
Why not just have any and all outbound mail sent directly from your personal server to its connecting destination (including your domain, which would get the connection route by your MX provider and then picked up by your google apps domain)?

Read the second and third paragraphs again. That’s why you can’t just set up a mail server and have it work.

1 Like

You can just set up a mail server and have it work.

You are incorrect that all personal servers are “open” or blocked by default. There are plenty of personal consumer connections, or furthermore VPS servers out there that do not have blocked ports for their IP connections to the internet that are not getting blacklisted by other IPs. Simply, they aren’t “open” to anyone on the internet rebounding email off of them with spam. They send secure, authenticated and legitimate email through their own ports directly from the system outbound to the internet.

Yes, if you’re sending from a VPS you’re probably ok. The article however explicitly refers to hosting “out of your basement or closet”. That means a home broadband connection, which means the IP space will be on various blacklists (as the article says) because, unsurprisingly they’re a massive source of spam.

There may be a handful of ISPs left that have dynamic IP space that isn’t on a blacklist, but they’re pretty rare.

Most residential customers will have to relay through a trusted server to ensure their mail gets delivered, period. For example, it’s relatively common practice for mail servers to reject SMTP connections from hosts that don’t have correct reverse PTR records (I do this with my mail server)—and since ISPs don’t let customers change their reverse lookup hostnames without charging them for static IP addresses, mail from just about anyone running an e-mail server out of their closet at home on a residential ISP connection will be rejected. Also, almost all residential IP blocks are one one blacklist or another, so most folks wanting to send e-mail from a home server will need to relay to get around that.

A VPS, on the other hand, will almost certainly not be in an RBL’d netblock, unless you’re using a shady hosting company that’s already been sullied by spammers. VPS customers also almost certainly can correctly configure DNS and get reverse-lookups working for their e-mail server. If you’re using a VPS then you can almost certainly send mail directly without needing to relay. In that case, as long as your server is properly configured, e-mail isn’t an issue.