How to fix Ollama: 500 message=Internal Server Error

With over 50K+ GitHub stars, Open WebUI has emerged as a robust, feature-rich, and self-hosted solution for AI-driven applications. Supporting multiple LLM runners like Ollama and OpenAI-compatible APIs, it allows users to operate completely offline while enjoying an extensible and user-friendly interface. However, one common issue users encounter is the error: “Ollama: 500, message=’Internal Server Error’. This blog will guide you through diagnosing and resolving this error effectively.

Understanding the Issue

The “500 Internal Server Error” often indicates a server-side problem with the Ollama service. This could stem from misconfigurations, unsupported models, or issues with the container setup in your deployment. Follow this step-by-step guide to identify and fix the problem.

Step 1. Clone the repository

Copied!git clone https://github.com/open-webui/open-webui/
git clone https://github.com/open-webui/open-webui/

Step 2. Examining the Compose file

Copied!services:
  ollama:
    volumes:
      - ollama:/root/.ollama
    container_name: ollama
    pull_policy: always
    tty: true
    restart: unless-stopped
    image: ollama/ollama:${OLLAMA_DOCKER_TAG-latest}

  open-webui:
    build:
      context: .
      args:
        OLLAMA_BASE_URL: '/ollama'
      dockerfile: Dockerfile
    image: ghcr.io/open-webui/open-webui:${WEBUI_DOCKER_TAG-main}
    container_name: open-webui
    volumes:
      - open-webui:/app/backend/data
    depends_on:
      - ollama
    ports:
      - ${OPEN_WEBUI_PORT-3001}:8080
    environment:
      - 'OLLAMA_BASE_URL=http://ollama:11434'
      - 'WEBUI_SECRET_KEY='
    extra_hosts:
      - host.docker.internal:host-gateway
    restart: unless-stopped

volumes:
  ollama: {}
  open-webui: {}
services:
  ollama:
    volumes:
      - ollama:/root/.ollama
    container_name: ollama
    pull_policy: always
    tty: true
    restart: unless-stopped
    image: ollama/ollama:${OLLAMA_DOCKER_TAG-latest}

  open-webui:
    build:
      context: .
      args:
        OLLAMA_BASE_URL: '/ollama'
      dockerfile: Dockerfile
    image: ghcr.io/open-webui/open-webui:${WEBUI_DOCKER_TAG-main}
    container_name: open-webui
    volumes:
      - open-webui:/app/backend/data
    depends_on:
      - ollama
    ports:
      - ${OPEN_WEBUI_PORT-3001}:8080
    environment:
      - 'OLLAMA_BASE_URL=http://ollama:11434'
      - 'WEBUI_SECRET_KEY='
    extra_hosts:
      - host.docker.internal:host-gateway
    restart: unless-stopped

volumes:
  ollama: Array
  open-webui: Array

This Docker Compose file defines two services, ollama and open-webui, with associated volumes for data persistence. The ollama service runs a container named ollama based on the ollama/ollama image (defaulting to the latest version if OLLAMA_DOCKER_TAG is not set). It stores its data in the ollama volume, restarts automatically unless stopped, and uses a TTY-enabled container. The open-webui service builds its image using a specified Dockerfile with build arguments, setting OLLAMA_BASE_URL to /ollama. It runs a container named open-webui, maps a local port (default 3001) to the container’s port 8080, and stores data in the open-webui volume. It depends on the ollama service, setting an environment variable for the OLLAMA_BASE_URL to point to ollama at port 11434. It also uses host.docker.internal for additional networking and restarts unless stopped. Two volumes, ollama and open-webui, are defined for persistent storage.

Key Modifications:

I changed the default port from 3000 to 3001 to avoid conflicts with my existing applications. The Compose file allows customization, so feel free to adjust as needed.

Step 3: Start the Services

Bring the services up with Docker Compose:

Copied!docker compose -f docker-compose.yaml up
docker compose -f docker-compose.yaml up

Step 4: Verify the Services

Ensure all services are running as expected:

Step 5: Accessing the Open WebUI

Step 6: Download a Model

Downloading and managing AI models is straightforward. For example, after downloading, you’ll see the Llama2 model listed:

Once downloaded, you will see the Llama2 model listed instead of your name.

Step 7: Configure Settings

Adjust the configuration for your models and application settings:

Step 8. Configure the permission

Step 9: Select the Right Model

If you encounter an issue like Ollama: 500, message='Internal Server Error', it might be due to pulling an unsupported model. Refer to this discussion for solutions

Try pulling llama3.2:1b model and it will work flawlessly.

The “Ollama: 500, message=’Internal Server Error’” error can be resolved by carefully reviewing configurations, pulling supported models, and ensuring sufficient system resources. By following this guide, you can confidently deploy and customize Open WebUI for your AI-powered projects.

Happy building!