"Benchmarks" for LLMs are a total hoax, since you can train them on the benchmar... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		andrepd 4 days ago \| parent \| context \| favorite \| on: GPT-5.2 "Benchmarks" for LLMs are a total hoax, since you can train them on the benchmarks themselves.

XCSme 3 days ago [–]

I would assume a good benchmark has hidden tests, or something randomly generated that is harder to game

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact