Tokenization — AI Dictionary
A subword unit — the smallest piece of text a model reads or generates.
Continue to AI University →