AI Model Serving

LM Studio is a desktop application that lets you discover, download, and run open-source large language models on your local computer. It offers a chat interface, OpenAI-compatible API, and developer tools for privacy-focused AI experimentation and development.

Ollama is an open-source tool that makes it easy to run large language models locally on your computer. It offers full data privacy, offline capabilities, and optional cloud model acceleration. Ideal for developers, researchers, and privacy-conscious users building AI applications.

Open-source high-throughput LLM inference and serving engine. Uses PagedAttention for efficient KV cache management, continuous batching, and an OpenAI-compatible API for deploying open models.