Can anyone share what a training dataset would look like for something like this... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		ideashower on Jan 30, 2025 \| parent \| context \| favorite \| on: SmolGPT: A minimal PyTorch implementation for trai... Can anyone share what a training dataset would look like for something like this? What are some use cases?

qrios on Jan 30, 2025 | [–]

It is downloading a dataset (TinyStories) from huggingface[1]. Here you can drill down deeper into the structure and content of the source data.

[1] https://huggingface.co/datasets/roneneldan/TinyStories

efm on Jan 30, 2025 | [–]

Karpathy's nanoGPT has a full training pipeline using Shakespeare. [1]

The use case for this is learning in simple example.

[1] https://github.com/karpathy/nanoGPT

Consider applying for YC's Summer 2026 batch! Applications are open till May 4
Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact