A/B testing your email messages

I have about 30,000 friends who follow my blog on email, and the open rate is about 8% (which according to Mailchimp is very low) on each blog post. Besides this I have another 9,500 friends and acquaintances that I send an email update to once a month on key blog posts I have written during that month. That has an open rate of 17%. Roughly about 5000 of my friends read a blog post, which gives me a fairly good sample size to do some serious A/B testing on my blog post titles.

My email friends are about 61% from India, and 25% from the US (largely the valley). This was inverse just 5 years ago in terms of %, but the number was about 3200. So a nearly 10 times growth in 5 years. 

Since wordpress does not allow me to segment the email by friends, everyone gets the same blog post and title as soon as it is published. But the email acquaintances is where I have a lot fun.

Here’s what I found last week in 3 sets of 2000+ subscribers.

I had the same body of the message but changed the Subject of the email.

First subject (Sent on a Thursday at 10:30 IST): Where is analytics headed in 2020? An insight gathered from 25 top #startups – 14% open rate

Second subject (Send on a Friday at 10:30 IST): The future of analytics is in offerings based on derived insights – 11% open rate

Third subject (Sent on a Saturdaya at 10:30 IST): How analytics companies are making 92% margins compared to software companies – 21% open rate

I am shocked that the Saturday message had best open rates, since weekends have been consistently low on my open rates.

The message though was one of margin – which I think appeals to Indian entrepreneurs a lot.

What do you think is the reason that the first subject got a better open rate than the second? Is it more specific?

5 thoughts on “A/B testing your email messages”

  1. The final one is the most provocative about a specific topic. It suggests something compelling is going on that is outside the intuitive grasp, which you will shed light on. The first two suggest the article talks about trends, which people may feel like they know, or that the article will be more general in nature.

  2. I think the first title trigger some curiosity about the future and so more people are interested to read through it. In the second title, the outcome of the anlysis is available and as the final verdict is already known, less people are interested in reading how you reach that outcome.

Comments are closed.