Like many of you who read this blog, I’ve received plenty of unsolicited marketing emails from vendors promoting “qualified sales leads” that are available for purchase. Typically, my reply sounds like this one from the other day:
“No thanks – we’re an email service provider that preaches the benefits of opt-in email marketing. Looks to me like you’re giving people a way to send unsolicited Spam.”
Ultimately, I will mark such unsolicited messages as Spam, and therein lies the point. Sending unsolicited messages is a surefire way to receive Spam complaints, because by definition that’s exactly what you’re sending. I remember talking to a prospect who once said “…but once they realize how valuable the content is, they’ll be glad we sent it…” which sounds like it could be a soundbite from the old Saturday Night Live “Bad Idea Jeans” sketch:
Assuming that recipients will see the same value in your content that you do—especially those who don’t have an existing relationship with your brand—is a bad idea. Your reputation as an email sender is not unlike your reputation as a person. Sending to a list of purchased email addresses is a bit like being “that guy” at the office Christmas party — even if it’s an isolated incident, such a misstep can inflict long-lasting damage on your reputation.
We’ve all heard that if you’re in marketing, in particular email marketing, you should constantly be testing to maximize results. The most common test mentioned is the ubiquitous “A/B” split test, meaning a 50/50 list split to test one variable against another (graphics, copy, offer, layout, list, time of day, day of week, etc.).
But is an A/B test all you can or should do? If you have only a few thousand or fewer emails to work with, an A/B test may be all you can do to ensure statistically reliable results. However, if your list is too small, an A/B test might not make any sense. For example, if you only have a few hundred email addresses, splitting and conducting one test will literally tell you nothing (statistically) other than directionally relevant information. Instead you may need to try to replicate the test over time, to aggregate the results and to analyze your collective data over a longer period.
The first consideration is to quantify how many email addresses you need to test to ensure you have a representative sample and more importantly, to ensure the results are reliable. There is a lot of math and science behind this topic, and fortunately a lot of math/science/statistics sites have free online tools such as this one.
You must set up the test(s) correctly (with sufficient sample sizes and assumed response rates) on the front end to ensure that results on the back end are reliable, meaning with a confidence level that you’re comfortable with (we recommend a 95% confidence level if it’s possible). Again, there are resources online to assist such as this one. The key is to avoid the common mistake of merely looking at results and assuming winners/losers based on seemingly different response rates.
Before testing, you have to identify the goal or the question you’re trying to answer. We recommend that you actually write these down and then, as briefly and concisely as possible, describe the various yardsticks you will use to determine your winner. As form follows function, the goals/objectives of the test coupled with the means to measure results should help drive copy, graphics, and/or layout to ensure the messages are properly structured and focused on whatever question you’re trying to answer..
Let’s say your goal is a higher click rate and after an A/B test you find “A” has a 2.7% CTR and “B” has 2.85%. It is a common mistake to use subtraction and declare that “B” was the winner or that “B” was only 0.15% higher and that could lead you down the path of thinking it wasn’t a significant result (i.e. a virtual “tie”). Or maybe you routinely just pick the higher percentage as the winner and run with that. Using proper percent increase/decrease calculations, we find that this is actually a 5.56% increase from “A” to “B.”
That however may or may not be statistically significant, but as you can see it’s a much larger increase than originally assumed. In order to determine if the results are statistically significant, use one of the calculators, plug in each version’s list size and the click percentage (or open percentage, or conversion rate, etc. depending on the key metric you’re analyzing) and it will instantly tell you whether this difference is enough to be reliable (with a 95% confidence level).
In this example, let’s pretend I sent “A” and “B” to a random 2,000 people each. The calculations indicate that this would not be enough of a difference to be statistically reliable. In fact, the “B” cell’s click rate would have to have been at least 3.81% in order for the difference to be reliably significant. However, if you didn’t analyze the results properly you wouldn’t know this.
The other way to ensure you’re maximizing your results is to avoid doing a full scale A/B test. If your database for an email marketing campaign is large enough (again calculate minimum sample size), you can do a different kind of split test. First, split your list 10%/90% (ensuring it’s random). Then split the 10% group in half so you have two small splits and the remaining 90%.
Deploy your test to the 10% splits, give as much time as possible for activity to occur (twenty-four hours if possible), analyze the results and then deploy the winner to the remaining 90%. That way you’ve done your best to maximize the campaign’s results without going “all in” on a typical full file A/B split.
As with gambling, learn the rules, do the math, analyze the data and place your bets. Do it right, and the odds will swing in your favor.
In parts one and two of How Long is a Piece of String?, we focused on two of the most commonly asked questions regarding email marketing campaigns. The third and final question of what we call The Email Big Three is “What’s the best time of day/day of the week to send our emails?” Just like the answer to the first two questions, the answer to this question is also, “It depends.”
The best time of day or day of the week to send an email depends on a variety of factors such as when your recipients are opening their email, when they’re interacting/and not interacting with messages, when they’re inundated with all kinds of messages (not just yours), when they’re grooming their inbox/deleting messages, when they’re most intensely interested in your messages and when they’re not. The best way to answer this question and many others is to let the data drive the answer.
Earlier this week we went over the ins and outs of Gmail’s new Priority Inbox. The biggest challenge the new inbox will pose will be continuing to get your message seen by your email subscribers. Many of the tried and true email marketing best practices that you should currently be using will increase the likelihood of your message being seen by your audience.
Consistency - Be consistent with how often you’re sending as well as the “feel” of your content. Everyone likes something they are familiar with and can rely on.
Relevancy - How many email messages do you get daily that you consistently check? Probably not too many, but there’s a reason you check the ones you do, they interest you. Once again put yourself in the recipient’s shoes and make sure each mailing is not only consistent with relevant content, but also widespread enough to catch the eye of people interested in different areas of your services or products.
The Basics – Don’t forget where we started. Your reputation, delivery and use of best practices need to stay intact while focusing on the points above. You need to make it to the inbox before worrying how to save your heated seat there.
Overall, Gmail Priority Inbox is another flavor of categorizing your email messages. It gives the user the ability to separate what they feel is important and leave the rest to either be deleted or read at some other time. The mailbox being separated into different sections may steer early interests away – people are for the most part still used to the traditional list style of their inbox. Whether the new Priority Inbox becomes widely used or not, it is still wise to make sure your email campaigns are as effective as possible.
So how is Gmail’s new Priority Inbox going to affect you? Well, the concept is not new, we’ve been hearing about engagement and relevancy for a while now. The new Hotmail, general smart filter implementations and now the new Priority Inbox at Gmail all reinforce the need for relevant and engaging content in your campaigns.
If the release of Priority Inbox tells us anything, it’s that the challenge to get your email messages seen by subscribers will become increasingly difficult. Over time, most email clients will give the ability to categorize mail or mark some as relevant and leave the rest to decay. That cold and lonely decay box doesn’t sound very appealing. You want the heated seats while you wait for your clicks and opens, and the way to get there is by supplying relevant content.
The new inbox consists of 3 tiers, an ‘Important’ section, followed by a ‘Starred’ section and lastly an ‘Everything Else’ section. Gmail uses your recent mailbox behaviors to decide what gets flagged important. Your email priority can be manually raised by the user, allowing emails that Gmail missed to be placed in the important folder. The priority function works on a simple ‘+ or -’ system, allowing the user to raise or lower the importance of an item. Items rated with high importance will display with a small yellow tab next to the subject line and moved to the ‘Important’ folder after refreshing.
The second tier down is your ‘Starred’ section. Only emails that you manually mark with a star will show up here. Moving an email to the ‘Starred’ section does not place it in the ‘Important’ category.
Lastly is the ‘Everything Else’ section, which will consist mostly of email messages that are not interacted with on a regular basis.
The main topic that stems from all of this is, of course, how to keep yourself out of the “Everything Else” category. Check back later this week for part two, to learn about the best practices that you can utilize to keep your messages important in the eyes of the recipient.
In parts 
So how is Gmail’s new 
