It all started with a tweet. I wanted to see if my theory that the number of bots exploring links posted on Twitter was on the increase, was correct.
If it was true, it would mean that it would correlate with an increase in Twitter ’spam’ and automatic tweets filled with affiliate links; two highly irritating types of tweet that in my opinion pollute Twitter.
This triggered me to do some investigating. So I fired up Excel and started playing with my stats from the last 5 months from the links I’ve posted to Twitter (via cli.gs). This is what I found:
Percentage of hits from bots in Twitter links over the last 5 months
[Blue = percentage of bots, red = trendline]
It appears as though the proportion of hits from bots is decreasing over time; disproving my theory. So, what does this tell us?
- Are there fewer bots? I doubt this would be due to a decrease in new bots emerging, but more likely that Twitter is wising up and blocking bots. Lo and behold, here’s a recent example of this: Twitter calls time on sports stats tweeting [5th May 2009, digitalmedia.strategyeye.com]
- Or, are fewer bots being detected? Perhaps bot programmers are finding ways around being detected by Twitter and therefore cli.gs too?
It could be either. Actually, as I realised later upon further investigation, it could be that new bots are actually going back and exploring older links, thus causing older links to continue to accrue more hits. Cli.gs is limited in that it only shows a visitor graph up to the last 30 days, but I did notice literally one or two visits cropping up occasionally:
The lesson? Investigate more! And demand more features from URL shorteners such as cli.gs to enable us to come up with more conclusive results.
Once we have this reliable data, I’d also like to see a breakdown of which links triggered the highest percentage of bot hits – were there keywords in the tweet that attracted bots? (I’d expect so) Therefore, are my tweets latterly lacking those keywords and contributing to the trend above? How does that correspond to new followers? Which brings me to a topic I’ve been curious about for a while: SEO for tweets.
Naturally it’d be great to identify the bots too.
So in conclusion, a great starting point for a lot more investigation! Watch this space.
