Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

That argument has been tried in a variety of ways and been shot down in court repeatedly. (there are parallels to tenants not agreeing to the terms of their internet connection where the landlord provided it).

LinkedIn has two things that they do which protect them; First, they specify they disallow access in their robots.txt file. While not a binding agreement per se it is the default mechanism that is accepted by the community for apriori identifying whether or not automated access is possible. Second, when they detect an access pattern that violates their terms of service they actively block the access proactively notify the source of the violation.

The sad truth is that web scraping has been around since the very beginnings of the Web back in 1993 and this question has been litigated in every way that you might choose to argue it, the body of case law is enough to fill at least two volumes in the reference section of the library.

There is no legal or ethical basis for scraping the web without permission. And if it isn't explicitly allowed by a site the presumption is that it is disallowed (no 'open door' exception).



Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: