How to Deploy Milvus with Docker Compose: A Comprehensive Guide

X (Twitter) Facebook Pinterest LinkedIn

Milvus, an open-source vector database designed to power AI applications and search engines with its ability to handle massive vector datasets, has become a go-to solution for developers looking to implement feature-rich search and AI functionalities. Combining Milvus with Docker Compose streamlines the deployment process and ensures consistency and scalability across different environments. This guide provides a detailed walkthrough on deploying Milvus using Docker Compose, enabling you to set up a powerful vector database for your applications easily.

Table of Contents

Why Docker Compose?

Docker Compose facilitates the management of multi-container Docker applications. By using a YAML file to define your application’s services, networks, and volumes, Docker Compose allows for easy configuration and automation of your containerized applications. Docker Compose offers a seamless way to manage and interconnect these services for deploying complex applications like Milvus, which may rely on multiple services (such as a database and a search engine).

Prerequisites

Before proceeding, ensure you have Docker and Docker Compose installed on your machine. Familiarity with Docker and basic vector database concepts is also recommended.

Step 1: Create the Docker Compose File

Start by creating a docker-compose.yml file in your project directory. This file will define the necessary services for Milvus, including Milvus server itself and its dependencies, such as MinIO (for storage) and Etcd (for metadata management).

version: '3.8'

services:
  etcd:
    image: bitnami/etcd:3.4.9
    environment:
      - ALLOW_NONE_AUTHENTICATION=yes
    volumes:
      - etcd-data:/bitnami/etcd

  minio:
    image: minio/minio:RELEASE.2020-12-03T00-03-10Z
    volumes:
      - minio-data:/data
    ports:
      - "9000:9000"
    environment:
      - MINIO_ACCESS_KEY=minioadmin
      - MINIO_SECRET_KEY=minioadmin
    command: server /data

  milvus:
    image: milvusdb/milvus:v2.0.0
    ports:
      - "19530:19530"
      - "19121:19121"
    depends_on:
      - minio
      - etcd
    environment:
      - ETCD_ENDPOINTS=etcd:2379
      - MINIO_ADDRESS=minio:9000
      - MINIO_ACCESS_KEY=minioadmin
      - MINIO_SECRET_KEY=minioadmin

volumes:
  etcd-data:
  minio-data:

Explanation of Docker Compose Components

version: Specifies the Docker Compose file version. Version 3.8 is used here for its support of newer features and stability.
services: Defines three services: etcd for metadata management, minio for object storage, and milvus for the vector database engine.
image: Points to the Docker image for each service. Official images are used for Etcd, MinIO, and Milvus.
volumes: Maps persistent storage for the services, ensuring data is retained across container restarts.
ports: Exposes ports to the host machine, allowing for external access to MinIO and Milvus services.
environment: Configures necessary environment variables for each service, such as authentication credentials for MinIO and connection endpoints for Milvus.
depends_on: Ensures that the milvus service starts only after the minio and etcd services are up and running.

Step 2: Deploying with Docker Compose

To deploy Milvus along with its dependencies, navigate to the directory containing your docker-compose.yml file and execute the following command:

docker compose up -d

This command pulls the required Docker images, creates the containers, and starts the services in the background.

Milvus Docker Compose Log — Screenshot: BORDERPOLAR

Step 3: Verifying the Deployment

After deploying the services, verify that Milvus is running correctly by accessing its client interface or performing a simple operation, such as creating a collection or inserting vectors. You can interact with Milvus through its REST API or one of the SDKs provided by the Milvus project.

Conclusion

Deploying Milvus with Docker Compose significantly simplifies setting up a vector database, making it accessible for development, testing, and production environments. Following the steps outlined in this guide, you can have a powerful vector database up and running, ready to enhance your applications with advanced search and AI capabilities. Whether you’re building a search engine, recommendation system, or AI-driven application, Milvus and Docker Compose provide a robust and scalable foundation for your data storage and retrieval needs.

Author
Recent Posts

Follow me

Anastasios Antoniadis

Anastasios Antoniadis is the founder and editor-in-chief of BORDERPOLAR... He is a software engineer, blogger, and avid gamer covering tech, gaming, and coding guides for over 4 years. He is a 2014 graduate of the Department of Informatics and Telecommunications of the University of Athens, an M.Sc. holder in Computer Science, and a Ph.D. student in Program Analysis.