tokencategorical
TokenCategorical
Bases: Distribution
Source code in hfppl/distributions/tokencategorical.py
__init__(lm, logits)
Create a Categorical distribution whose values are Tokens, not integers.
Given a language model lm
and an array of unnormalized log probabilities (of length len(lm.vocab)
),
uses softmax to normalize them and samples a Token from the resulting categorical.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
lm
|
CachedCausalLM
|
the language model whose vocabulary is to be generated from. |
required |
logits
|
array
|
a numpy array of unnormalized log probabilities. |
required |