Home / Glossary / autoregressive generation

autoregressive generation

How a model writes: one token at a time, each conditioned on everything before it, prompt and output alike, with no draft held in reserve. The property behind both the power (each step can react to the last) and the bills (see prompt caching) (Chapter 2).