techniques

Prompt Caching

Saving the processed form of a long, repeated opening of a prompt so the model does not redo that work on every request. It only helps when the start of the prompt is identical each time, and saved entries expire after a short while. Like keeping a bookmark in a long book instead of rereading from page one.

Want to learn more about AI?

Peter Saddington has trained 17,000+ people on agile and AI. Let’s talk.

Work with Peter