MonkeyBrains! The Official MailChimp Blog about email marketing and monkeys

Monkey Brains   WWW

« Showcase: Emmaus | Main | How AOL Designs Their HTML Emails »

Monitoring Deliverability

Dm_stats Spam filters are making it really hard to get emails delivered. In this latest email deliverability study from ReturnPath, 19.2% of legitimate emails they monitored never made it to the inbox. This study over at Pivotal Veracity says about 15% of emails get blocked. Yikes.

Do you have any idea how many of your emails are making it into your recipients' inboxes? The only way to tell is to setup a TON of email accounts with all the major ISPs, send your campaign, then check each account one by one. Poking your eye with a hot needle would be more fun.

Luckily, you can just sign up for a deliverability monitoring service. They'll do all the mundane work for you. I just recently signed up with one, and tested a MailChimp campaign...

There are quite a few services to choose from. A quick Google search turns up:

We went with, because they were the easiest to jump right in and get started. I hear SenderScore and Pivotal Veracity are extremely thorough, but no free trials are readily available. DeliveryWatch looked very promising, but they don't take American Express yet.

Here's how these services typically work:

  1. They create a "seed list" of email addresses that you add to your subscriber list.
  2. The seed list is composed of email accounts they've setup with all the major ISPs
  3. For every campaign you send, you add a little tracking code somewhere in your content. It can be embedded so that recipients don't see it.
  4. After you send your campaign, you can check back in a couple hours to see how you did (screenshot below).

You'll see which ISPs let your emails through to the inbox, and which ones sent you to the spam folder (and which ones just lost your email outright). They'll also check major blacklisting services and tell you if you were blocked by any of them.

When we tested the service, we sent a campaign to our own MailChimp customers, and we used the same range of IP addresses as all our "new" user accounts (none of our  "reserved" or "trusted-customer" IPs were used). Below is a screenshot of the graph that DeliveryMonitor gave us.

Some tips:
I've spoken with a couple people who run these services, and they gave a couple tips for people who are just getting into delivery audits:

  • Results are regional, meaning if you send lots of emails to customers in Europe, go with a provider who has a seed list with European ISPs (DeliveryWatch is one).
  • Don't try to use these services on small/test campaigns. They need you to send to your actual list, because ISPs behave differently when you're sending a small test vs. a real campaign to thousands of recipients on their network. That means you don't send, tweak, send, tweak. You slowly gather data over several "real" campaigns and make changes accordingly.
  • Most of them charge a 'per event' fee. You pay each time you run a test. Sounds like it can get expensive, but think about what a royal pain it would be to actually setup a bazillion test accounts and then run reports on each one. DeliveryMonitor charges a monthly fee, with a monthly limit on tests.

Here's the report we got from our campaign (click to zoom in). You can see that our biggest problem was with AOL. Seems 20% of our emails to their AOL recipients got spam filtered. This isn't enough to alarm me, but if it were significantly higher, I'd make adjustments to my email content. If that didn't work, I'd contact the AOL postmaster for help, or look into some accreditation services, like Goodmail or SenderScore.

If you've ever hit your send button and wondered, "how many of my emails are actually making it through?" you should try one of these services. It can be a huge time saver, and the information you get back can be extremely valuable (especially if you do a lot of e-commerce via email).


October 3, 2006 in Tips, Tricks, Best Practices | Permalink


TrackBack URL for this entry:

Listed below are links to weblogs that reference Monitoring Deliverability:


I enjoyed your post and particularly the good guidance you are providing. I agree that services such as ours and some of our competitors may be too expensive for small businesses and while I hate to see business referred elsewhere;->, I do believe delivery monitoring is crucial for all size businesses and am very supportive of your recommendation. I wanted to share a couple tips based on our experience and that of our clients that may be helpful to your readers..

First, you note that the seedlist should always be mailed with the live campaign. While it is true that you will not be able to gauge the impact of ISPs' volume filters if you just mail the seedlist, doing an initial test to just the seedlist does provide extremely valuable information. Mailing to just the seedlist prior to your live campaign allows you to "forecast" results and, most imporantly, fix problems prior to your live campaign. For example, it can tell you whether the ISP is blocking all mail from your IP (e.g. you are on a public or private blocklist) or if there is something about your content that is triggering their spam filters. You can then "fix" these problems prior to mailing your live campaign. With your live campaign, you can also mail the seeds (ideally parsed throughout the mailing or, if not possible, placed at the end of the mailing) to gauge volume filters and the ultimate inbox/spam folder disposition. Thus we recommend using the seedlist as both a pre-campaign forecasting tool as well a live campaign tracking tool. If forced to choose only 1 use of the seeds most our clients elect to use it as a forecasting mechanism as it permits them to fix many problems before they affect their real campaigns.

A second tip for your readers with no budget - there is a free way to check whether your IP is on any public blacklists. This can be done at (top middle column). One note, most blacklists are unimportant as they are not used by the major ISPs. Some of the ones to pay attention to are Spamhaus, SpamCop, RFC Ignorant, SORBS.

Third, its important for all mailers to pursue whitelisting and feedback loop setup (feedback loops are how the ISPs return spam complaints to you). Mailers can ask their Deliverability Service Provider or their ESP (Email Service Provider such as Silverpop, Responsys, Exact Target, etc) for assistance on this.

Posted by: Deirdre Baird | Nov 7, 2006 10:22:53 AM

The comments to this entry are closed.

MonkeyBrains! is the official MailChimp Blog. MailChimp is a product of The Rocket Science Group.