Skip to main content
You are currently on a page documenting the use of Ollama models as text completion models. Many popular Ollama models are chat completion models.You may be looking for this page instead.
This page goes over how to use LangChain to interact with Ollama models.

Installation

Setup

First, follow these instructions to set up and run a local Ollama instance:
  • Download and install Ollama onto the available supported platforms (including Windows Subsystem for Linux aka WSL, macOS, and Linux)
    • macOS users can install via Homebrew with brew install ollama and start with brew services start ollama
  • Fetch available LLM model via ollama pull <name-of-model>
    • View a list of available models via the model library
    • e.g., ollama pull llama3
  • This will download the default tagged version of the model. Typically, the default points to the latest, smallest sized-parameter model.
On Mac, the models will be download to ~/.ollama/models On Linux (or WSL), the models will be stored at /usr/share/ollama/.ollama/models
  • Specify the exact version of the model of interest as such ollama pull vicuna:13b-v1.5-16k-q4_0 (View the various tags for the Vicuna model in this instance)
  • To view all pulled models, use ollama list
  • To chat directly with a model from the command line, use ollama run <name-of-model>
  • View the Ollama documentation for more commands. You can run ollama help in the terminal to see available commands.

Usage

Multi-modal

Ollama has support for multi-modal LLMs, such as bakllava and llava. ollama pull bakllava Be sure to update Ollama so that you have the most recent version to support multi-modal.

API reference

For detailed documentation of all ChatOllama features and configurations head to the API reference.