The next token is taken by sampling the logits in the final column after unembed... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

throwawayk7h on April 1, 2024 | parent | context | favorite | on: But what is a GPT? Visual intro to Transformers [...

The next token is taken by sampling the logits in the final column after unembedding. But isn't that just the last token again? Or is the matrix resized to N+1 at some step?

HarHarVeryFunny on April 1, 2024 [–]

There is an end-of-sequence token appended to the input sequence, and this is what is transformed into the predicted next token.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact