*****************************************************************
Message delivered directly to members of the group:
internet_marketing_articleblaster@yahoogroups.com
*****************************************************************
Please consider this free-reprint article written by:
James Brausch
==================
IMPORTANT - Publication/Reprint Terms
- You have permission to publish this article electronically in
free-only publications such as a website or an ezine as long as
the bylines are included.
- You are not allowed to use this article for commercial
purposes. The article should only be reprinted in a publicly
accessible website and not in a members-only commercial site.
- You are not allowed to post/reprint this article in any
sites/publications that contains or supports hate, violence,
porn and warez or any indecent and illegal sites/publications.
- You are not allowed to use this article in UCE (Unsolicited
Commercial Email) or SPAM. This article MUST be distributed in
an opt-in email list only.
- If you distribute this article in an ezine or newsletter, we
ask that you send a copy of the newsletter or ezine that
contains the article to support[at]glyphius.com (replace [at]
with @)
- If you post this article in a website/forum/blog, ALL links
MUST be set to hyperlinks and we ask that you send a copy of
the URL where the article is posted to support[at]glyphius.com
(replace [at] with @)
- We request that you ask permission from the author if you
want to publish this article in print.
The role of iSnare.com is only to distribute this article as
part of its Article Distribution feature (
http://www.isnare.com/distribution.php ). iSnare.com does NOT
own this article, please respect the author's copyright and
this publication/reprint terms. If you do not agree to any of
these terms, please do not reprint or publish this article.
==================
Article Title: Split Testing Sucks And Other Heresies
Author: James Brausch
Word Count: 2053
Article URL: http://www.isnare.com/?aid=52351&ca=Marketing
Format: 64cpl
Author's Email Address: support[at]glyphius.com (replace [at]
with @)
Easy Publish Tool: http://www.isnare.com/html.php?aid=52351
================== ARTICLE START ==================
Split testing is what marketing is all about; right? Wrong!
There is a much better way to get the answer to 95% of the
kinds of questions you might consider split testing. Let�s step
through it.
First of all, you must decide your desired outcome. Is it
profitability? Is it search engine ranking? Is it more traffic?
Is it more inbound links?
Once you have decided your desired outcome, you must find a way
that OTHER PEOPLE�S sites can be measured for that outcome.
That�s right; I�m not advocating split testing because there is
a much faster and even more accurate way to get the same results
by simply looking at the results of OTHER PEOPLE!
It�s called statistical analysis. It�s what most scientific
advancements have been based on! When you go to the doctor,
does he split test antibiotics and exercise on you? Of course
not! You would fire him immediately. Instead, he looks at the
results of studies where other people had the same desired
outcome you want (getting rid of that fever, cough, whatever)
and prescribes the drug or treatment that was proven safe and
effective for a statistically significant number of OTHER
PEOPLE!
Will it always work for you because it worked on a
statistically significant number of OTHER PEOPLE? No; of course
not. But it will work a majority of the time. The fact is that
split testing has the same problems. When you find an answer
from split testing and choose the �a� version over the �b�
version based on 20 actions� There is a percentage chance that
you actually chose the wrong version. Increase the number of
actions and you increase your chances of picking the right one.
The same is true when looking at OTHER PEOPLE�S results instead
of your own.
OK; so how can we measure some of these things. Search Engine
Ranking is easy. You can compare sites at the #1 ranking with
sites at the #100 ranking�. or sites in the top 10 with sites
in the 101-110 range in ranking. No problem.
More traffic? You can find open logs for a statistically
significant number of sites. A less perfect measurement might
be Alexa or Jupiter ratings. When dealing with less accurate
measurements like that, simply increase your sample size or
compare more distant extremes (Alexa rankings in the top 100 vs
Alexa ranking in the 99,900-100,000 traffic rankings).
More inbound links? Once again you can use open logs and look
at referral entries� or you can trust the link: command at MSN�
or something less reliable like the link: command at Google
(once again, just increase your sample size and/or compare more
distant extremes in number of inbound links).
How about profitability? This is my favorite. This is why most
of us are here. I used this measurement for my Glyphius
software and all of the copywriting statistical studies posted
on my blog and in the Statistical Copywriting Online Home Study
Course.
How can you get the profitability figures for other web-sites?
Are they just going to turn them over? Actually, public
companies do just that. You could use something like that
public data. Also, many affiliate networks give indirect
profitability figures (the marketplace at Clickbank, the EPC
rating on CJ are both less accurate profitability numbers).
Or you could do what marketers have quietly done for decades
before ecommerce and the Internet even existed. This is how
Glyphius and the Statistical Copywriting course were created.
It is really quite simple. Ask yourself these two questions:
1. If I was paying for advertising to a site that was
profitable� would I continue to pay for that advertising month
after month as long as it was profitable?
2. If I was paying for advertising to a site that wasn�t
profitable� would I continue to pay for that advertising month
after month?
The answer to #1 is clearly a �yes� for a vast majority of
people. The answer to #2 is clearly a �no� for a vast majority
of people. There are exceptions. Some large companies don�t
even track the profitability of their ads� so they will add
some incorrect data if we use this measurement.
The same thing happens in science all the time. Patients in
medical studies lie or exagerate about their results (in both
directions). People aren�t completely predictable. They do
things against their own self-interest sometimes.
The solution? Same as with the other imperfect measurement
techniques� increase the sample size and/or compare more
distant extremes. If people were 100% predictable, then we
wouldn�t need a very large sample size to look at these
web-sites.
So, let�s take an example to see how this works. Let�s say some
idiot �guru� marketer is telling you that using the digit
�7″ in your pricing will increase your conversion ratio
and your profitability. OK; fine. You have three choices:
1. Believe him/her. Congratulations; you have just joined the
religious society of idiot �guru� marketers. Instead of
embracing science for those things that can be easily
researched, you have decided to apply a religious belief to
such a topic instead. You can feel a little better about
yourself because 98% of all people are in the same boat.
Unfortunately 98% of all people will retire at 65 years old
broke.
2. Split test it. Arrgh! You know why you haven�t already done
this� it takes forever to get statistically significant
results. Then you know that there are still hundreds or
thousands of things you still need to split test after that.
3. Statistically analyze OTHER PEOPLE�S pricing and their
profitability.
Look; some things need to be split tested. Still; wouldn�t it
be nice if you could come very close to the real result before
you even start your split tests? Do you really need to split
test a bright purple background on your site when it�s obvious
that a vast majority of profitable sites use a white
background? Of course not! Test only those things that can�t be
easily found by doing a statistical analysis of OTHER PEOPLE�S
sites!
OK; so how do we get the above answer?
First we make us a list of profitable and unprofitable sites.
We already know how to do that; right? No? OK; let�s break it
down.
First we search for a statistically significant number of
keywords on Google or any other search engine that shows paid
advertising (or we go get some magazines that advertise
web-sites� or we go get a nickle ad sheet and find some
classifieds that include URLs� it doesn�t matter� just go to a
source of paid advertising that shows URLs).
Then we make a list showing the exact ad and the URL. Now we
wait. How long do we wait? We wait long enough to have some
folks with unprofitable ads change them or remove them. If you
used magazines, wait 6 months. Some magazines have a policy of
only allowing you to run an ad for three months at a time. So,
it takes more than 3 months before you see unprofitable ads
disappear. On Google, you can probably get away with waiting
just a couple of weeks. The longer you wait, the fewer sites
you will have to compare (the old comparing more distant
extremes rule).
Now you do it again. Go make a list of ads and their URLs that
are still paying for advertising.
Now compare your two lists. Wherever you find an ad that is
exactly the same months later, you have an ad that is very
likely to be profitable. Put that URL on your �profitable�
list. Put the other URLs that were on your first list, but are
missing on your second list on your �unprofitable� list.
Cool; now you have a list of profitable and unprofitable sites.
You can now run all kinds of quick statistical analysis on these
sites. Since you can review a hundred sites much, much faster
than you can usually get 100 results in a split test� you are
way ahead now.
Now, simply go through your �proftiable� list and figure out
the percentage of prices that have a �7″ in them. Now go
through your unprofitable list and figure out the percentage of
prices that have a �7″ in them. Compare the two numbers.
Are they very close to each other? Then using a �7″
probably doesn�t matter very much in your pricing when it comes
to profitability. Are they very distant? Did you find that 83%
of unprofitable sites used a �7″ in their price, but only
21% of profitable sites did? Woo hoo! You have found your answer
in just an hour instead of waiting for days for a split test.
So how do you know if you had a statistically significant
number of sites to test? Yikes; with something this complex,
that would take a couple of years of statistical training to
figure that out. Let me give you a real world way of finding
out that will save you from the hell of majoring in statistics
in college (although if you are a single male, I highly
recommend doing the major� for some reason statistics chicks
are much better looking than chicks taking other majors�
statistically speaking).
Predictability. Turn around your study and do a split test.
What percentage of the time did the results of the split test
match the predicted outcome of the statistical analysis. The
closer you get to 100%, the more you can rely on the dataset of
profitable and unprofitable sites you built. Of course, 0% isn�t
the bottom of the scale here� 50% is. So if you are only a bit
over 50%, then you realy need to look at the assumptions of
your study� and the sample size� but if you are well over 75%,
then you have a decent dataset that you can rely on for future
studies.
You might be thinking� wait! I still had to do a split test!
And worse� building that list of profitable and unprofitable
sites took me just as long as it would take to do a split test�
so what gives?
You only have to build the list of profitable and unprofitable
sites and validate it once! Then you can do an unlimited number
of studies using the same data!
The same is true for any measurement. My ranking data asks
several thousand questions using just one set of data of high
ranking and low ranking sites (with a 91% confidence rating)!
I�ve answered dozens of copywriting questions using my one
dataset of profitable and unprofitable sites (with a 96%
confidence rating in that case).
Yes; it�s a lot of work just to get the answer to the �7″
question�. but then it is very little work to then ask �what
about the digit �3′�? Or �Are red headlines more
profitable or blue headlines?� or �Is long copy really more
profitable or not?�
Instead of wasting your time doing a split test for days for
each answer (or if you are a newbie still struggling to get
traffic� maybe months to get an answer) and during that time
not making as much money as you could be making� you can just
spend an hour tabulating the results from your profitable and
unprofitable lists of sites! Then you can have an answer that
you can count on 96% of the time.
Guess what? With 20 actions in a split test (20 sales in this
case for both �a� and �b�), you only have a 95% confidence in
your results. How long will it take you to get at least 20
sales for both your �a� and your �b� version in a split test?
So there you go. I�m a heretic. I hate split testing my own
sites for answers I can easily get by analyzing OTHER PEOPLE�S
sites. It isn�t the first time I disagree with 98% of marketing
�gurus� and it probably won�t be the last.
About The Author: James D. Brausch is the creator of several
software and information products about copywriting, traffic
generation and and product creation. His blog is here:
http://www.JamesBrausch.com
Please use the HTML version of this article at:
http://www.isnare.com/html.php?aid=52351
================== ARTICLE END ==================
For more free-reprint articles by James Brausch please visit:
http://www.isnare.com/?s=author&a=James+Brausch
------------------------ Yahoo! Groups Sponsor --------------------~-->
Everything you need is one�click away.� Make Yahoo! your home page�now.
http://us.click.yahoo.com/AHchtC/4FxNAA/yQLSAA/7VpplB/TM
--------------------------------------------------------------------~->
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/internet_marketing_articleblaster/
<*> To unsubscribe from this group, send an email to:
internet_marketing_articleblaster-unsubscribe@yahoogroups.com
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
No comments:
Post a Comment