Testing Newsletter Subject Lines: What We Learned From 6 Months of A/B Tests

How much of a difference can changing a single word in your email subject line REALLY make?You might be surprised!(It still surprises us, sometimes.)Every week, we send our newsletter to 125,000 + subscribers - and every week, we send it with two different newsletter subject lines. (Before we keep going, here's the link where you can get on that list.)One reason we do this is to maximize our open rate for each individual newsletter - a process you can read more about in this post - but the OTHER reason is so we can learn more about email subject line testing and writing better subject lines.Changing just one or two words in your email subject line can make a HUGE impact on your open rates - and reveal a lot about your readers!Want to see how?A while ago, we took a hard look at six months of testing subject lines for our newsletter and found some pretty distinct patterns in which kinds of subject lines score more opens - and which ones are easy for our readers to ignore.

What is an email subject line?

Your email subject line is your chance to make a first impression. It’s the line of text your email subscribers see in their inbox before opening your email. The subject line could be the deciding factor as to whether someone opens your email or not. You should always check the subject line before clicking send on your email!

What have we learned from 6 months of testing newsletter subject lines?

These trends aren't necessarily true for everyone, but they DO demonstrate the importance of testing your own subject lines - you could learn something that dramatically improves your open rates over time!

The more specific, the better

A long, long time ago, when the Earth was young and Edgar hadn't yet been created, our founder and CEO Laura Roeder sent out a weekly newsletter to her own audience of social media marketers.Here are a few actual newsletter subject lines from those emails:

• my "no-no" face
• two magic words
• here's what to do next
• the #1 most important factor
• nicely done!

We will mail you ten thousand dollars if you can guess what ANY of those emails was about.(Note: we will not mail you ten thousand dollars.)Laura's newsletter consistently performed pretty well, which might make you think that the less information you give in the subject line, the better - right? A little curiosity can go a long way!But not necessarily.We've put that logic to the test with our own newsletter email subject lines, and when it comes to our audience, we've found that specificity is more useful than ambiguity.There's a good reason for that, and we're going to talk about it in a second - but first, here are some examples of what we're talking about.

These email newsletter subject lines aren't that different - but one big distinction is their degree of specificity.In Combo 1, "Twitter's new rules" could mean a lot of different things.(Maybe they have new guidelines about the appropriate use of reaction GIFs taken from TV's The Bachelor, which would be highly relevant to our own interests.)

Combo 2, however, is more specific - Twitter's new rules have to do with social media scheduling. A reader can see that email's subject line and understand immediately just how relevant it is to them, and that makes the importance of opening it feel much more urgent.(Here's where you can read that post, by the way.)Let's check out another:

1. [The MeetEdgar Newsletter] Facebook’s THREE new algorithm updates ????‍????
2. [The MeetEdgar Newsletter] Facebook's latest algorithm changes explained ????‍????

Again, these subject lines are barely different - but Combination 1 took the lead by specifying that Facebook has implemented three algorithm updates, speaking to how comprehensive this post is going to be.Here are a few more quick examples - see if you can guess the winner for each:

1. [The MeetEdgar Newsletter] the social media news you might’ve missed ????
2. [The MeetEdgar Newsletter] everything you might’ve missed ????

If you guessed Combination 1, you guessed right! "Everything" isn't compellingly specific - it could mean pretty much anything. The other version is much clearer about what it offers.Let's try one more:

1. [The MeetEdgar Newsletter] this new Facebook Live feature ????
2. [The MeetEdgar Newsletter] Facebook Live’s new screensharing feature ????

In this case, Combination 2 was the clear winner - and again, it was by offering a specific detail. An unspecified new feature may or may not make someone curious enough to open this email, but specifying the type of update makes a big difference!Why do we find that specificity works better for us than ambiguity?Especially when that newsletter from years ago, for example, found so much success being vague?

One answer may simply be that these are two different audiences, who respond differently to a single format.An equally likely culprit, though, is the fact that vague headlines in general just don't perform the way they used to.Being vague and tease-y has fallen out of favor with both audiences and social networks like Facebook. These days, clickbait-style ambiguity is more likely to get you ignored or hidden from news feeds, while giving readers a clear idea of what to expect is more popular than ever.(Which speaks to the importance of monitoring your success over time - what works for you right now might not be as successful in a year or two!)Okay, enough specifics about specificity.What ELSE have we learned from our email subject tests?

Use language with muscle

"Use strong language" is one of those lessons that sounds kind of insultingly obvious - like "don't season your porterhouse steak with glitter."In practice, though, it takes different forms that aren't always so clear!And while specificity like we just talked about can make your language stronger, sometimes colloquialisms can give a little extra oomph, too!Here's an example:

1. [The MeetEdgar Newsletter] our most popular sales page tips ????
2. [The MeetEdgar Newsletter] our hottest sales page tips ????

Technically, Combination 1 is more specific, but Combination 2 is more exciting. (Especially because "hottest" can indicate both that something is popular and that something is useful.)Here's another example of that exact principle in practice:

1. [The MeetEdgar Newsletter] 10 can’t-miss Facebook strategy guides ????
2. [The MeetEdgar Newsletter] roundup: our fave Facebook strategy guides ????

Combination 1 was the winner here - not only does it give a specific number of guides the reader can expect, but it describes them as "can't-miss" instead of "our fave." The former ascribes an intrinsic value to them, which is much stronger than suggesting their worth is dependent on our opinions.(We won't take that too personally.)Here's one more example of how valuable strong language can be:

1. [The MeetEdgar Newsletter] Facebook’s “engagement bait” page demotions
2. [The MeetEdgar Newsletter] Facebook is demoting pages and posts

In this case, Combination 2 was the winner, despite being less specific about the reason Facebook is demoting Pages and posts.Why was it the winner, then?Because it tells a story - it describes an action. Combination 1, by comparison, describes a thing.Active language can be a lot more compelling than static or passive language - and that's something we're going to explore a little more in the next section!Which is...

Questions and answers

Which is more useful: asking a question, or giving an answer?

On the one hand, asking a question can make someone curious enough to want to find out the answer - but on the other hand, that's asking them to do a certain kind of work they might prefer to avoid.Here's an example:

1. [The MeetEdgar Newsletter] our remote workstations revealed ????
2. [The MeetEdgar Newsletter] how does your home office compare? ????

Between these two subject lines, Combination 1 was more popular - it promises an answer instead of a question.(Our audience was MUCH more interested in seeing our home offices than they were in thinking about their own!)Here's another, which is a lot more subtle:

1. [The MeetEdgar Newsletter] new data - how do you compare? ????
2. [The MeetEdgar Newsletter] new data - see how you compare ????

These email newsletter subject lines are virtually indistinguishable, save for one small detail: the winner, Combination 2, offers the reader information, while Combination 1 asks them to perform analysis.We've found over time that our newsletter subscribers consistently prefer when we offer information instead of giving them tasks, so this isn't much of a surprise!Here's one more example of that - see if you can guess which was the winner:

1. [The MeetEdgar Newsletter] upgrade your blog in literally 10 minutes ⏱
2. [The MeetEdgar Newsletter] the 10-minute blog upgrade we swear by ⏱

If you guessed that Combination 2 won, you're right!Even though it seems like it breaks that pattern of action words being more compelling than static language, it goes to show how much patterns depend on context.A subject line like "Facebook is demoting pages and posts" performs well for us because it tells a story about what someone else is already doing, but one like "upgrade your blog in literally 10 minutes" doesn't, because it asks the reader to do work. In a case like this, a subject line teasing a static thing can be more valuable than one teasing an active verb - it all depends on who's performing the action!All of this illustrates how important it is to measure the success of your subject lines both in the context of your own readers AND the context of your subject lines over time.You might find that certain rules only apply under certain circumstances - and that the things that work consistently for other marketers don't necessarily work for you!

What have you learned from you own email subject line testing?

What are some of your subject line rules?Do your readers find questions hard to resist?Do they love emoji?How do they compare to our own?Let us know some of the subject line rules you try to follow in the comments below!