> It's all fun and joy until you come back to it 1 or 1.5 years later, and resources are now throttled because they're so over used because of the explosion in AI.
No yesterday, I just tried to spin up an old project using common crawl, which is hosted on AWS open data, seems like it's gotten quite popular since I lasted worked on it, and they're experiencing throttling from over usage, maybe even ddos (or just a really bad developer).
But seems to be a pretty common thing with stuff related to data and crawling. Everyone is cracking down on crawlers, adding paywalls, removing previously free APIs, so coming back to old projects in that domain seems to be impossible.
What does this mean?