Ollama¶

Category: AI Type: Project Service Tags: llm · machine-learning Editable: Yes

Overview¶

Ollama makes it easy to run large language models (LLMs) locally within your cluster. It provides a simple API and CLI for downloading, running, and managing open-source models like Llama, Mistral, Gemma, and dozens more. The Ollama plugin installs Ollama to a shared persistent volume and runs it as a service within your project. GPU acceleration is supported for significantly faster inference.

How It Works¶

Project Service — When added to a project, Terra sets up Ollama and starts it as a persistent service. Workstations and other plugins in the same project can connect to the Ollama API to run models. The service stays running until you uninstall the plugin.

Prerequisites¶

A shared persistent volume provisioned in your project (for model storage — models can be several GB each)
For GPU acceleration: the NVIDIA GPU Operator plugin installed on the cluster and a GPU-equipped node in the cluster

Installation¶

Open Terra and navigate to the Plugin Marketplace
Search for "ollama"
Click Install
Fill in the configuration fields below
Click Confirm to deploy

Configuration¶

Install-Time Fields¶

Field	Details
`gpu`	boolean · Required · Default: `false` Enable GPU acceleration for model inference
`runtime`	string · Optional · Default: `""` Container runtime for GPU access (e.g. `nvidia`). Leave empty for CPU-only.
`destination`	string · Required · Default: `ollama` Directory path within the volume for Ollama data (no leading slash)
`install_volume`	shared-volume · Required Shared persistent volume to store Ollama models and data
`enable_node_port`	boolean · Required · Default: `false` Expose a NodePort for the Ollama API (allows access from outside the cluster). You will need to retrieve the assigned port from Kubernetes directly.

Notes¶

This plugin is editable — you can update GPU settings and node port exposure after install via Terra
Models are downloaded on demand when first requested via the Ollama API or CLI — ensure the persistent volume has enough capacity for your intended models (models range from ~1GB to 70GB+)
The destination field must not include a leading slash (e.g. use ollama, not /ollama)
Ollama is accessible within the cluster at the service ClusterIP address; workstations can connect to it using its Kubernetes service DNS name
For integration with Open WebUI, install that plugin and point it at the Ollama service endpoint

plugins/ollama/terra.yaml
resource_id: ollama
name: ollama
icon: https://raw.githubusercontent.com/juno-fx/Terra-Official-Plugins/refs/heads/main/plugins/ollama/scripts/assets/logo.jpg
description: Get up and running with large language models.
category: AI
tags:
- llm
- machine-learning
editable: true
fields:
- name: runtime
  description: This is the underlying container runtime. Normally you need to set this to the container runtime that has access to the GPU. For example, "nvidia". You can also leave this empty which will default to CPU.
  required: false
  default: ""
- name: gpu
  description: This is the underlying container runtime. Normally you need to set this to the container runtime that has access to the GPU. For example, "nvidia". You can also leave this empty which will default to CPU.
  required: true
  type: boolean
  default: false
- name: destination
  description: Destination directory (DO NOT ADD A LEADING SLASH "/")
  required: true
  default: "ollama"
- name: install_volume
  description: The name of the install volume to store the persisted data.
  required: true
  type: shared-volume
- name: enable_node_port
  description: Expose a nodeport for the Ollama service. This is required if you want to access the Ollama API from outside the cluster. NOTE, you will need to get the assigned port for kubernetes directly.
  required: true
  type: boolean
  default: false