Toby Shaw

How to get data out of Twitter

The modern SEO professional needs to be able to do a wide variety of things quickly and easily. Linkdex helps you do a lot of these things extremely well, but one thing we don’t offer (yet) is easy social media integration. To that end, today I’m going to give you a simple way of getting Twitter data yourself as a placeholder for the top-notch social tools we have planned.

Get Data Out of Twitter

The number that shows up on the ‘Tweet!’ button on pretty much every website in the world is actually fairly carefully cloaked by Twitter. Trying to crack your way into the Javascript is quite complicated. There is an API call that you can do to get this data in bulk, however, and I’ve written it up into this Google Doc.

All you need to do is copy the document into your own account, input the URLs, and let it chug through them. It’ll then give you an output you can use for whatever you want. It’s easy to customise. If you want to look at Facebook ‘Likes’ rather than tweets, you can simply swap the twitter API URL in column B for the Facebook API URL (“http://graph.facebook.com/?ids=”) and do some jiggery-pokery with the substitutions in column C.

There’s one problem – Google only lets you do 50 API calls on any one spreadsheet. That means, if you have thousands of posts to process, you’re going to need a better way of doing things. That’s where we have to move into actual programming.

This is a Python script. It’s incredibly easy to run – just install Python 2.7 and put my script file together with a CSV file of the URLs you want to check into the ‘scripts’ folder. Then go into the Command Prompt of your computer (run ‘cmd’ from the Start menu) and do something like this:

Command Line

It’ll save a CSV file of the output, which you can then easily get the data out of. Simple – but not as simple as just using the Google Doc, admittedly.

Data like this can be used for a variety of things. I used it for some interesting analysis on what kind of posts generate the most tweets and links (that you can find in an earlier post on this blog), which would obviously be useful to any website trying to optimise their content output for the social media world.

Running analysis like this on your own site can help you learn what people enjoy reading about on your blog, and can help you tailor your content towards what will get you noticed in the social arena. Alternatively, you could run the analysis on link prospects – if you want to know which sites it is best to do a guest blog post on, for example, you could find out easily which would be able to get you the most mentions. You can take your link prospect data out of Linkdex and feed it into the Google Doc, and it will give you a great idea of who to try to contact to maximise your post’s impact on the social world (and therefore Google rankings, hopefully!)

  • Most popular posts this month
  • Submit Your SEO Stories to Win $250

  • The Psychology of Online Sharing

  • My Name is Zeus

  • Linkdex Competition Winners Announced

  • Why some SEO’s build better links than others