Everyone needs to put the following in their robots.txt to give Linkedin a taste... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		altdataseller on July 31, 2022 \| parent \| context \| favorite \| on: Tell HN: LinkedIn is automatically importing job p... Everyone needs to put the following in their robots.txt to give Linkedin a taste of their own medicine. User-agent: LinkedinBot Disallow: / # Notice: If you would like to crawl us, # please email whitelist-crawl@domain.com to apply # for white listing. *taken from linkedin.com/robots.txt

smt88 on July 31, 2022 | [–]

HN messes up whitespace, so you'll have to indent this to make if preserve the newlines.

eyegor on July 31, 2022 | [–]

Ah yes, surely they will always respect the holy robots.txt

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact