How to Set Up S3 bucket Using Docker Compose

X (Twitter) Facebook Pinterest LinkedIn

Using Docker to set up an Amazon S3 bucket for development or testing environments is an efficient way to replicate S3’s functionality without incurring extra costs or network latency. This setup is particularly useful when developing applications that interact with S3 but need to be tested locally before deployment. In this guide, we’ll set up a local S3-compatible service using Docker and Docker Compose.

Table of Contents

Understanding the Tools

Docker: Docker is a platform for developers and sysadmins to develop, deploy, and run applications with containers. Containerization is the use of Linux containers to deploy applications.

Docker Compose: Docker Compose is a tool for defining and running multi-container Docker applications. With Compose, you use a YAML file to configure your application’s services. Then, with a single command, you create and start all the services from your configuration.

MinIO: We’ll use MinIO, a high-performance, S3-compatible object storage service that you can run on-premises. MinIO is widely adopted for its scalability, performance, and compatibility with S3 APIs.

Prerequisites

Docker and Docker Compose installed on your machine. Installation guides for Docker and Docker Compose can be found on the Docker official website.
Basic understanding of Docker, Docker Compose, and YAML syntax.

Step 1: Setting Up MinIO with Docker Compose

Create a Docker Compose File: First, create a docker-compose.yml file in your project directory. This file will define the MinIO service and specify how it should be run.

version: '3.7'
services:
  minio:
    image: minio/minio
    volumes:
      - minio_data:/data
    ports:
      - "9000:9000"
    environment:
      MINIO_ACCESS_KEY: minioadmin
      MINIO_SECRET_KEY: minioadmin
    command: server /data
volumes:
  minio_data:

In this configuration, we are setting up a service named minio using the minio/minio image. We map a volume for persistent storage and expose port 9000, which is the default MinIO console and API access port. The MINIO_ACCESS_KEY and MINIO_SECRET_KEY environment variables are used to set the access and secret keys needed to interact with the MinIO server. The command option tells MinIO to start as a server and use the /data directory for storing data.

Launch MinIO: Navigate to the directory containing your docker-compose.yml file and run the following command:

docker compose up -d

If it’s not already present, this command will download the MinIO Docker image and start the MinIO service in detached mode.

Step 2: Accessing MinIO

Once the service is up, you can access the MinIO web interface by opening a web browser and navigating to http://localhost:9000. Use the access key (minioadmin) and secret key (minioadmin) to log in.

Step 3: Using MinIO

With MinIO up and running, you can:

Create buckets using the web interface or S3 CLI tools.
Upload, download, and manage files and folders.
Use S3-compatible tools and libraries in your applications to interact with your local MinIO instance, just like you would with Amazon S3.

Step 4: Integrating with Your Application

To integrate with your application, configure your S3 client to use the endpoint http://localhost:9000 and the access and secret keys you defined. This setup allows your application to communicate with MinIO as if it were talking to Amazon S3.

Conclusion

Setting up an S3-compatible service like MinIO using Docker and Docker Compose is a straightforward process that greatly benefits development and testing. It allows developers to ensure their applications interact correctly with S3 services without connecting to Amazon’s cloud. By following the steps outlined above, you can have a local, S3-compatible environment running in minutes.

Author
Recent Posts

Follow me

Anastasios Antoniadis

Anastasios Antoniadis is the founder and editor-in-chief of BORDERPOLAR... He is a software engineer, blogger, and avid gamer covering tech, gaming, and coding guides for over 4 years. He is a 2014 graduate of the Department of Informatics and Telecommunications of the University of Athens, an M.Sc. holder in Computer Science, and a Ph.D. student in Program Analysis.