beginner
Text is cut into sub-word pieces; you pay for, and are limited by, tokens.
Text is cut into word piecescut into word pieces Why?, like Lego bricks; you pay per brick.
Here: What AI is (and isn't) · Next: How LLMs work
The scenario
After launch your LLM bill triples while traffic only doubles.
The question
How do you bring cost down without hurting quality?