This Is Why You Shouldn’t Use Alexa
By Collin Maessen on comment
That Alexa isn’t good at giving reliable statistics is well known in IT. Any website like Alexa that tries to estimate traffic to a website in a similar way via indirect measurements will encounter the same issues it has. Those that work in IT often know what those issues are and know what the consequences are for the data that’s gathered via those methods.
The main issue that Alexa has is that it gathers the data it uses via users that installed the Alexa toolbar (or a toolbar that passes information to Alexa). This has as a result that demographic, used browsers, and even the country visitors are from influence the statistics that Alexa gathers about a website. This can introduce serious artefacts and biases into the collected data and basically makes Alexa data worthless. At best it can give you an idea about how well a website is doing, but that doesn’t mean that what you’re seeing matches reality. I’ve already written a far more detailed blog post about how Alexa works and why you never should rely on the data it provides; it’s just too unreliable.
The blog post that I wrote about Alexa were all sparked by Anthony Watts using Alexa data to claim he’s doing better than his competition. His website is certainly big and it could very well be the case that he is outperforming his competitors. But Alexa is not the tool that you can use for determining if you are doing better than your competitor.
One good example is how my site is doing compared to the website HotWhopper. This website is run by Sou and debunks claims made by climate science deniers, it often touches on similar topics as I do. We share some of the same target audience and in that sense this website is one of my competitors.
If you compare my Alexa ranking with that of HotWhopper you’ll notice that I’ve passed HotWhopper. According to Alexa I now have a ranking of 858,248, but HotWhopper has dropped below me with a ranking of 897,008 (lower number is better). What you also clearly can see is that HotWhopper is dropping in the Alexa ranking, at its height it had a ranking of 382,166. So clearly HotWhopper readership is on the decline while I’m at least keeping my readership. From this data you would also expect that I’m in the same ballpark with the number of visitors and pageviews that I get. Right? Wrong.
If we take a look at the Google Analytics data for both websites a very different story emerges:
The first graph is the number of pageviews my website gets on a monthly basis, the second one is for HotWhopper. Last month Real Sceptic had 4,146 pageviews and HotWhopper had 58,901 pageviews. HotWhopper had fourteen times the number of pageviews Real Sceptic received. I’m not even in the same league as HotWhopper, let alone in the same ballpark.
You see the same when you look at the number of visitors for each website (as far as I can tell Alexa weighs this more heavily in their ranking calculations):
Last month Real Sceptic had 3,015 visitors while HotWhopper had 28,320 visitors. HotWhopper had nine times more visitors than Real Sceptic.
What makes it even worse for the Alexa data is that readership for HotWhopper isn’t declining, it’s increasing. Readership did reach a plateau for a while, but the decline you see in the Alexa data isn’t there. That’s just an artefact that’s present in the Alexa data. This is why you never should use Alexa statistics to do any serious comparisons. The data Alexa uses doesn’t have to match reality at all.
I work for a software development company where we develop and maintain complex web retail software. We also link these retail websites to online campaign and tracking software for our clients. It has made me well aware of the limitations of certain technologies and products. So it’s no surprise for me that Alexa got the rankings for HotWhopper and Real Sceptic wrong. What did surprise me is how wrong Alexa was, I never thought it would deviate this much.
But I think I know why Alexa got it so wrong. It’s probably because I’ve written about Alexa a couple of times and I have a high ranking for my Alexa content in Google searches. My website is often displayed on the first page in Google, I can see that with the statistics I have in Google Webmaster Tools. What you need to realize about this is that people who search for information about Alexa rankings tend to be Alexa users, which means that there’s a good chance that they also have the Alexa toolbar installed.
Alexa gives the hint this is happening with the keywords it lists for Realsceptic (on the left) and HotWhopper (on the right):
This keyword top 5 doesn’t match any of the lists that I have for the most used search terms for my website. The list provided by Alexa also doesn’t match with the amount of traffic pages get on Real Sceptic. This is probably the reason Alexa thinks my website is doing better than HotWhopper. It also shows how easy it is to introduce a bias into the data Alexa gathers and how unreliable this makes the statistics Alexa generates.
These inaccuracies in Alexa statistics is why my colleagues laugh when I tell them that some use Alexa data for comparing websites. I hope you now understand why they don’t take Alexa serious and why you shouldn’t use Alexa statistics.
Note:
I wouldn’t have been able to write this if Sou hadn’t been so generous to share her Google Analytics data with me. As a thank you please do not forget to visit her site and check it out.
Update 2014-02-08 @ 18:07:
Sou also wrote about the discrepancy between her Alexa ranking and the actual traffic she’s getting. It’s a good supplement to what I wrote as it gives more detail into what her traffic statistics were doing compared to her Alexa ranking.
Collin Maessen is the founder and editor of Real Skeptic and a proponent of scientific skepticism. For his content he uses the most up to date and best research as possible. Where necessary consulting or collaborating with scientists.
Thus next to there being an Alexa WUWT toolbar, the statistics of WUWT could also be inflated because Anthony Watts has written about Alexa very often.
http://lmgtfy.com/?q=alexa+site%3Ahttp%3A%2F%2Fwattsupwiththat.com
It could influence his statistics, but that hinges on him getting views via those searches. But I don’t see him rank high for these type of searches and it’s probably just noise compared to the traffic that he gets.
Thanks for doing this Collin. I was foolishly pleased when the Alexa rating showed an increase in popularity, even though you and others had warned me about it. When the Alexa rating got worse all of a sudden, I almost began to doubt the other stats that showed a steady rise in readership.
Your article has brought me back to earth. I believe that Google Analytics is among the best – would that be right, Collin?
Best of all is blogger stats, which I know are counted on actual hits and don’t rely on cookies. But unlike Google Analytics I think blogger stats include the referrer spam (it’s hard to separate them out of real visits but I can estimate from the data). Anyway, thankfully, the worst offending referrer spam doesn’t show up in blogger any more.
For anyone without knowledge on how these types of statistics are generated/gathered it’s not easy to figure out what it can or cannot tell you. Explaining this can help, but that’s still a lot of information that isn’t intuitive. That’s why I’m so grateful for you sharing your Google Analytics data, there’s nothing better for understanding the issues with Alexa statistics when you can compare it the actual data.
Google Analytics is one of the best statistics programs out there. Unless you need some very specific type of data or a specific method of gathering data there’s not really a reason to use anything else (especially as it’s free).
I’m not familiar with how Blogger collects its statistics. But most statistics that are gathered by direct measurement are reliable. You just need to be aware of what can and cannot track (how you’ve configured it can also matter).
Don’t get me started on the shortcoming of Google Analytics… You absolutely do not want to get in the habit of using google analytics for your websites. The data ownership and privacy issues alone are epic. Do *I* want the largest internet advertising company to own my website’s data and tax my industry by knowing more and more about it. NO THANK YOU! When people praise google analytics, I can tell that they have not tried very many alternatives out there.
Technically this comment is off-topic according to the commenting policy, but I’ll address what you raise (the blog post is about Alexa reliability).
I wasn’t praising Google Analytics with this post, my focus was on addressing the inaccuracies in the Alexa data. The reason Google Analytics was used was because both websites use it. If I had used two different statistics tools I would had have to deal with the differences between then and would have made this post unnecessarily convoluted.
However, I did say this about Google Analytics in a comment:
Indeed Google Analytics is one of the best out there, and most of the time I don’t see a reason to use something else. But this doesn’t mean there aren’t other tools out there that give you the same quality, or better for your particular usage.
There are valid criticism you can raise about Google Analytics and Google’s stance on data usage. It’s no secret that they have as goal to make all the data in the world searchable. For this very reason I have turned off certain features in Google Analytics as I don’t see them as beneficiary to me or my visitors (for example the sharing of traffic statistics with advertisement networks).
Also data ownership questions are always an issue as soon as a second party is involved for statistics gathering. But so far Google hasn’t been problematic.
However, for you this does matter. And there’s no reason if this is a concern for you to use something else (for example a statistics tool running on your own server). If it meets the requirements you have and addresses concerns you have I also see no reason not to use it.
It’s why I have 3 different tools running for this website. One gives me a good view of what is happening on the file level, another gives me a quick overview of traffic and what is being viewed, the third one is Google Analytics which gives me a lot of information about visitors and how they are using my website.
What matters is that the tool can be used for what you’re using it for and meets your requirements.
Everything Alexa says about traffic is absolutely wrong. My website is having 1600 unique visits per day and 85% of the traffic is coming from search engines. But according to alexa my rank is 900000+ and search engine traffic is only 37%.
Totally Agree that Alexa is useless. My site’s rank has yoyo’d between 12 million & 2 million and back all the while steadily growing as per Google Analytics. I tracked my Alexa number until I realized I was tracking random noise.
lol When I entered in the web world 1 year ago, with my modest site, I was around 2 milion place in Alexa. Since then, every 3 months I drop with 500-600 k ranks, when the site should actually creep a little up. Now I know it is unreliable information from there, thanks.