Home / Figures / Chapter 19

Figure 19.3: Prompt caching

Prompt caching.
Figure 19.3 Prompt caching. Because a byte-identical stable prefix can be reused, the provider saves the work done on the front of the first call and the next call pays full price only for its changing tail—the accented segments are the cache hit, the reused prefix. The rule follows directly: put stable content first and variable content last.