Toolkit

Resources

The tools worth knowing if you build with AI, grouped by job, with links to our deep dives.

Run models locally

Runtimes and apps for running open-weight models on your own hardware.

Ollama

One-command local model runner with a model library and an OpenAI-compatible API.

CLImacOSLinuxWindows

llama.cpp

The C/C++ inference engine behind GGUF. llama-server can pull models straight from Hugging Face.

GGUFCPUGPU

LM Studio

Desktop app to search, download and chat with local models, plus a local API server.

DesktopGUI

MLX LM

Apple's MLX-based package for running and fine-tuning LLMs on Apple Silicon.

Apple SiliconPython

vLLM

High-throughput serving engine for GPUs, with paged attention and an OpenAI-compatible server.

ServingGPUPython
On LeepCast:Which LLM runs on my machine?Running LLMs locally with Ollama and vLLM

Build with LLMs

Frameworks and protocols for wiring models into real apps and agents.

Model Context Protocol

Open protocol for connecting AI apps to tools and data sources through MCP servers.

AgentsProtocol

LangChain

Framework for chaining models, retrievers and tools, with LangGraph for stateful agents.

PythonTypeScriptAgents

LlamaIndex

Data framework for ingesting, indexing and querying your documents with LLMs.

RAGPython

Vercel AI SDK

TypeScript toolkit for streaming chat UIs, tool calls and structured output across providers.

TypeScriptStreaming

Chroma

Open-source embedding database that runs in-process or as a server.

Vector DBOpen source
On LeepCast:Building a RAG pipelineModel Context Protocol, explainedChoosing a vector database

Evaluate and observe

Know whether your LLM app works, and see what it does in production.

promptfoo

CLI for testing prompts and models against test cases, plus red-teaming.

EvalsCLIOpen source

Langfuse

Open-source tracing, prompt management and evals for LLM applications.

TracingSelf-host

Arize Phoenix

Open-source observability and evaluation built on OpenTelemetry.

TracingOpenTelemetry
On LeepCast:Evaluating LLM apps with evalsLLM observability in production

Learn

Free places to go deeper on how models actually work.

Hugging Face course

Free courses on transformers, agents, fine-tuning and more.

CourseFree

Anthropic courses

Hands-on notebooks covering prompting, tool use and evals with Claude.

CourseFree

Andrej Karpathy: Neural Networks Zero to Hero

Video series that builds neural nets and a GPT from scratch in code.

VideoFree
On LeepCast:Prompt engineering that works