Great post (5mo ago) "chinchilla's wild implications" giving context to LLM goldrush shifting from model size to dataset size following Chinchilla https://t.co/aDdUAPYCI8 Subtle important detail: analysis assumes 1 epoch. Recent work (e.g. Galactica) gives hope for 1+ regime.
— Andrej Karpathy (@karpathy) Jan 4, 2023
from Twitter https://twitter.com/karpathy
January 04, 2023 at 10:18AM
via IFTTT
No comments:
Post a Comment