Building software on top of Large Language Models¶ A workshop presented by Simon Willison at PyCon 2025. See my blog for the accompanying annotated slides. Setup Installing the packages Configuring the OpenAI API key Testing that it worked Prompting with LLM Setting a default model Running a prompt What did that do for us? Continuing the conversation Seeing it in the logs Browsing the logs with Datasette Using different models Piping in content Using system prompts Prompting with an image Using fragments Fragment plugins Prompting from Python Streaming Using attachments Using system prompts Async support Building a text to SQL tool Prototyping against the logs database Turning that into a Python function Upgrading that to a CLI tool Ways to make this better Structured data extraction Using simple schemas Something a bit more impressive Structured data extraction in Python with Pydantic Semantic search and RAG Semantic search Generating embeddings with LLM Answering questions against those PEPs RAG does not have to use semantic search! RAG is dead? Tool usage Passing Python functions to an LLM Multiple tools can run in a row Tools in Python Running calculations with simpleeval Let’s build some tools! Chat with your database, implemented using tools Security and prompt injection