techniques
Prompt Caching
Saving the processed form of a long, repeated opening of a prompt so the model does not redo that work on every request. It only helps when the start of the prompt is identical each time, and saved entries expire after a short while. Like keeping a bookmark in a long book instead of rereading from page one.
Want to learn more about AI?
Peter Saddington has trained 17,000+ people on agile and AI. Let’s talk.
Work with Peter