Repository for the oreilly live training course: "Getting Started with Llama and Other Local Models": https://learning.oreilly.com/live-events/getting-started-with-llama-2/0636920098588/

Windows Users: See WINDOWS-SETUP.md for Windows-specific instructions and a compatible requirements file that addresses known compatibility issues.

Conda

• Install anaconda
• This repo was tested on a Mac with python=3.10.
• Create an environment: conda create -n oreilly-llama3 python=3.10
• Activate your environment with: conda activate oreilly-llama3
• Install requirements with: pip install -r requirements/requirements.txt
• (Optional) Setup your openai API key. It's only needed for the cloud-comparison cells in 3.0-tool-calling-ollama.ipynb and the live tool-calling demo (live-demo-intro-agents-tool-calling.ipynb); everything else runs locally.

Pip

1. Create a Virtual Environment: Navigate to your project directory. Make sure you hvae python3.10 installed! If using Python 3's built-in venv:

   python -m venv oreilly-llama3

   If you're using virtualenv:

   virtualenv oreilly-llama3

2. Activate the Virtual Environment:

   • On Windows:

     .\oreilly-llama3\Scripts\activate

   • On macOS and Linux:

     source oreilly-llama3/bin/activate

3. Install Dependencies from requirements.txt:

   pip install python-dotenv
   pip install -r requirements/requirements.txt

4. (Optional) Setup your openai API key. Only needed for the cloud-comparison cells in 3.0-tool-calling-ollama.ipynb and the live tool-calling demo (live-demo-intro-agents-tool-calling.ipynb).

Remember to deactivate the virtual environment once you're done by simply typing:

deactivate

• Change the .env.example file to .env and add your OpenAI API key (optional; see above).

• pip install jupyter
• python3 -m ipykernel install --user --name=oreilly-llama3

These notebooks follow a structured learning path from basics to advanced topics:

1. Quickstart with Ollama - Get started running local LLMs using Ollama

   

2. Introduction to RAG - Learn the fundamentals of RAG with interactive visualizations of embeddings and chunking

   

3. Local RAG with Gemma 4 - Build a complete local RAG system using Gemma 4 and PDF documents

   

4. Tool Calling with Ollama - Learn how to implement tool calling with local LLMs (Gmail integration example)

   

5. Local Agents: Structured Outputs to Tool Calling - Build up to local agents through structured outputs with Pydantic and tool calling

   

6. Simple Agentic RAG - Build a ReAct-based agentic RAG system from scratch

   

7. Useful Local Agents - Run Hermes Agent with Gemma 4, vibe-check agentic task quality, and learn when to route to the cloud

   

8. Fine-Tuning Llama 3: What You Need to Know - Comprehensive guide to fine-tuning concepts (LoRA, QLoRA, PEFT)

9. Fine-Tuning Walkthrough with Hugging Face - Practical fine-tuning implementation

   

10. Quantization Precision Format Code Explanation - Deep dive into model quantization

    

11. GUI Options for Local Models - LM Studio, Open WebUI (with the Docker command) and other front-ends for local models

12. Best Local LLMs in Practice (2026 Edition) - Compare and explore the best local models available

    

13. vLLM Setup Guide - Complete guide to setting up and using vLLM for high-performance inference

    

Older versions and experimental notebooks are available in the notebooks/legacy-notebooks/ directory.

• Best Local Models 2026 - Which open models to run locally by hardware tier (Gemma 4, Qwen 3.8, DeepSeek-R1, Phi-4 and others), with deployment tools and use-case picks