For this sort of user demographics your pricing appears to be off by an order of magnitude. Just sayin' :)

i know, but as said, it started as a funny experiment, became a toy and then I felt bad charging more for things i can not influence. It'll definitely be able to charge 19.99 a month up to 99.00 a month for corporations, there's a lot of features I can add - but right now, it'd put me under even more pressure to charge that much. it's a messed up weird situation.

Considering increasing the highest price level. There are people who build businesses around Twitter. They will pay more than $1200/yr for something that helps them make more money than that.

yeah well, i instantly would and I have the features/service to back it up with quality data, but as long as I can't deliver any sort of service to larger accounts (1M+ followers), it doesn't make sense to charge like that. long term, no problem. right now, twitter limits me too much.

why don't you discuss this with some of the social data mining firms. they may have encountered similar problems

yeah, good point for sure. problem is that I am living in Europe right now (I miss the US), and it used to be a lot easier to just meet up with someone knowledgable from any industry back in the US. Here, it's hard to do so. And cold-emailing companies in that field might come off as me trying to drain their competitive advantage? No? I'd be curious who might be able to help out. But calling up Klout would likely not get me a call back at this stage.

and just to add: I think my particular problem is, and that's sort of the selling point, my analysis and reports is not just growth data (which is easy), but my calculations require all of your follower details to work correctly (to provide correct results). Twitter is sending me random follower details, so having a partial set means little reliability (for up to 60 days).

I don't know where in Europe you're currently based, but Datasift are in the UK and seem to be one of the hottest "social media analytics" companies out there. Their staff seem fairly active on twitter itself, maybe give them a try [1]

1 - http://datasift.com/whoweare/team

Yeah, I was about to suggest Datasift as well. IIRC I had a chat to some of them at the last SiliconMilkRoundabout & they seemed like they knew their stuff.

It does occur to me that the OP is effectively trying to replicate large chunks of the twitter datastore & that's going to be very difficult to manage! It's not like twitter themselves were particularly reliably to start with after all.

yeah, their data records are sometimes faulty and I need to scale down (even if it's not a time out) to 1 record per request to find the faulty in one hundred accounts. so that sucks.

and regarding replicating, originally that's what I did. I had up to 20M of Twitter's 140M records cached almost - but that probably wasn't cool with them on the long run and i was unable to maintain a database with one table having multiple gigabytes of data.

checking them out right now, thanks for the hint!

Also Peerindex.

there is also an excellent start-up based in Portland, OR called luckysort. Good people, only the time difference from Europe sucks ;-)

I'm happy to answer questions in regards to Klout as far as I can. We utilize GNIP and Datasift both for different situations, but we're working on a different side of data than you are, it sounds. Feel free to email me at api@klout.com.

Also: I've helped Twitter fix two other bugs that I reported, but on this one, it's a grey area (they don't consider it a bug, but a safety measure in terms of traffic time out, but still charge me API call credits). so yeah.

Well, they are right : if someone manage to find a request which regularly fail with timeout, he could use it to spam Twitter's servers indefinitely.

Which version of the API are you using?

i've switched over to the new version now, except for a few calls.