Proving You’re Wonderful!
Posted by Bill Leming on September 14th, 2009
A question came up last week regarding how to best measure the impact of a specific commercial email we recently deployed for a client and which, on the surface, appears to have produced nearly unbelievable positive results. While this endeavor is far more preferable than proving that you’re not guilty of lying, cheating or stealing, it’s nonetheless a challenge.
Anyone and everyone responsible for promoting the offer through the website, or via banner ads, social media, print and email wants to claim responsibility for the parentage of this genius idea and the attendant sales figures associated with it. So how do we “prove’ our numbers?
One suggestion that was offered was the idea of matching the email addresses of actual buyers against our email list to see how high or how low our match rate actually was. The hypothesis here was that a high match rate would indicate that there was a strong correlation between our list and those who bought and that either our email efforts must have been successful or that we had the ability to predict before deployment who was likely to buy.
Conversely, a low match rate would signify that our email efforts were less than successful. Without going into the various scenarios that might explain why this approach is less than logical and therefore less than irrefutable, suffice it to say that statisticians and most non-statistically oriented people will have problems with this methodology. 
