Home / Figures / Chapter 19

Figure 19.4: Streaming changes the wait you feel, not the wait itself

Streaming changes the wait you feel, not the wait itself.
Figure 19.4 Streaming changes the wait you feel, not the wait itself. Both runs take the same total time, but the buffered run is a long silent wait ended by the whole answer at once, while the streamed run puts the first token (in accent) on the screen almost immediately and then fills in steadily. It front-loads feedback without making the model any faster.