How can I load "tokenizer"?

#5
by NeuronJun - opened

I want to check the token_id of the output

Sign up or log in to comment