Compute perplexity from per-token log probabilities.
perplexity = exp(-1/N * Σ log P(token_i))
Expects log probabilities (≤ 0). Higher perplexity = more surprised.
Compute perplexity from per-token log probabilities.
perplexity = exp(-1/N * Σ log P(token_i))
Expects log probabilities (≤ 0). Higher perplexity = more surprised.