In this guide, we will learn how to install a MinIO server in Baremetal Infrastructure with Terraform in an AWS environment.

We are going to set up our own version of an Amazon S3-compatible storage service that can be configured to be accessible by multiple users. You can host your MinIO server on any host to which you have administrator access.

Introduction to MinIO

MinIO is an open-source replacement for Amazon S3 (Simple Storage Service) for backing up files, as a storage back-end for tools like a container registry, or even for hosting static websites.

MinIO describes itself as:

100% open source, enterprise-grade, Amazon S3-compatible object storage.

MinIO offers high-performance, S3 compatible object storage.Native to Kubernetes, MinIO is the only object storage suite available on every public cloud, every Kubernetes distribution, the private cloud and the edge. MinIO is software-defined and is 100% open source under GNU AGPL v3.

For this guide, we will use AWS to host an instance, which will provide a public IP address, to which other servers or your project can connect

For more details on the architecture of MinIO see the following link:

In this post you will see :

• How to Provisioning EBS volume for MinIO instance with Terraform
• How to deploy MinIO with Terraform
• Connect to the MinIO dashboard
• Create a Bucket
• Run MySQL Backup on MinIO Bucket

Prerequisites

Before you get started, you’ll need to have these things:

• Terraform > 0.13.x
• An AWS account with the IAM permissions
• AWS CLI : the AWS CLI Documentation
• AWS key-pair Create a key pair using Amazon EC2
• MySQL Database

Initial setup

• Clone the repository and install the dependencies:

$> git clone https://github.com/colussim/terraform-Minio.git
$> cd terraform-Minio
terraform-Minio $> mkdir ssh-keys
terraform-Minio $>  terraform init

Copy your AWS key-pair in ssh-keys directory. Edit the file variables.tf and modify if necessary the private_key variable

Usage

Install a MinIO server :

$> terraform apply
null_resource.minio-server (remote-exec): + sudo systemctl enable minio
null_resource.minio-server (remote-exec): Created symlink /etc/systemd/system/multi-user.target.wants/minio.service → /etc/systemd/system/minio.service.
null_resource.minio-server (remote-exec): + sudo systemctl start minio
null_resource.minio-server: Creation complete after 1m53s [id=1123250946150979358]

Apply complete! Resources: 10 added, 0 changed, 0 destroyed.
$>

If you use the terraform apply command without parameters the default values will be those defined in the variables.tf file.

variable "aws_ami" {
  description = "centos 8"
  default     = "ami-0d6e9a57f6259ba3a"
}

variable "aws_instance_type" {
  description = "Machine Type"
  default     = "t3.medium"
}

variable "private_key" {
  type        = string
  default     = "ssh-keys/id_rsa_aws"
  description = "The path to your private key"
}

variable "volume_size" {
  default = "20"
  description = "Size for volume default : 20 GB"
}

variable "zone" {
  default = "us-east-1a"
}

This will do the following :

• Install a CentOS 8 instance
• Add an EBS volume of 20 Gb
• Attach EBS volume
• Create a linux partition on this EBS volume
• Setup this partition on ext4 format
• Mount this partition on /opt/minio/data
• Install package wget
• Create a MinIO config file : /opt/minio/minio.conf
• Connects to the server via SSH and installs MinIO server and MinIO Client
• Connect to the MinIO Console

Tear down the whole Terraform plan with :

$> terraform destroy

Resources can be destroyed using the terraform destroy command, which is similar to terraform apply but it behaves as if all of the resources have been removed from the configuration.

In the network configuration of the VM (security.tf) we will allow access on port 22 and 9000 (for access to the MinIO console) in TCP.

Remote control

Connect to the MinIO Console with User : Admin and the password : Bench123

credentials are configured in the file vm_instance.tf on line 51 and 52: MINIO_ROOT_USER=Your User

MINIO_ROOT_PASSWOR=Password for This Users

By default, MinIO does not ship with TLS enabled. You can turn on this encryption by following this guide: How to secure access to MinIO server with TLS

Now you can access the dashboard on your computer at http://VM_IP_EXTERNAL:9000. The external IP address of the VM is displayed to you at the end of the Terraform deployment, otherwise you get it from the AWS console.

Create a Bucket

We will create a Bucket to store our MySQL backup, we will use the MinIO client but you can also use the console to create it.the client is installed on our server but you can also install it on your terraform workstation. Complete guide here for details.Download the client here

Add a Cloud Storage Service

Usage :

mc alias set [ALIAS] [YOUR-S3-ENDPOINT] [YOUR-ACCESS-KEY] [YOUR-SECRET-KEY] [–api API-SIGNATURE]

• ALIAS - Short name to reference Object Storage in commands.
• COS-ENDPOINT - Final node for your Object Storage instance.
• ACCESS-KEY - Access key that is assigned to your service identification data.
• SECRET-KEY - Secret key that is assigned to your service identification data.

The configuration information is stored in a JSON file, in ~/.mc/config.json.

$> /opt/minio/bin/mc alias set minio01 http://33.174.175.70:9000  Admin Bench123 --api S3v4  
Added `minio01` successfully.
$>

Create a bucket mysqlbkp

$> /opt/minio/bin/mc mb minio01/mysqlbkp
Bucket created successfully minio01/mysqlbkp
$>

Check that our bucket is well created

$> /opt/minio/bin/mc ls minio01
[2021-10-11 18:56:00 CEST]     0B mysqlbkp/
$>

Backup MySQL Database

in this example we have a database employees hosted in a MySQL instance in kubernetes cluster.

Step1 : install and configure a MinIO client in our pod

• Copy MinIO client inside the pod :

$> wget https://dl.min.io/client/mc/release/linux-amd64/mc
$> MYSQLPOD=`kubectl -n student1 get pods -l app=mysql | grep Running | grep 1/1 | awk '{print $1}'`
$> kubectl -n student1 cp mc student1/$MYSQLPOD:/usr/bin
$> kubectl -n student1 exec -it $MYSQLPOD -- chmod +x /usr/bin/mc

• Setup MinIO client :

$> kubectl -n student1 exec -it $MYSQLPOD -- /usr/bin/mc alias set minio01 http://34.174.175.70:9000 Admin Bench123 --api S3v4
Added `minio01` successfully.
$>

• Check the access to the Bucket :

$> kubectl -n student1 exec -it $MYSQLPOD -- /usr/bin/mc ls minio01
$> [2021-10-11 16:56:00 UTC]     0B mysqlbkp/
$>

Step2 : run MySQL Backup

• Copy backup script inside the pod :

$> cd mysql-scripts
$> kubectl -n student1 cp backup.sh student1/$MYSQLPOD:/var/lib/mysql
$>

• Run Backup :

$> kubectl -n student1 exec -it $MYSQLPOD -- /var/lib/mysql/backup.sh
$>

• Check backup is ok :

$> kubectl -n student1 exec -it $MYSQLPOD -- ls /tmp/employeesdb.sql
/tmp/employeesdb.sql
$>

Step3 : To upload the backup into a bucket

$> kubectl -n student1 exec -it $MYSQLPOD --  /usr/bin/mc cp /tmp/employeesdb.sql minio01/mysqlbkp
tmp/employeesdb.sql:                   160.58 MiB / 160.58 MiB ┃
$>

• Check that the backup file is uploads

$> kubectl -n student1 exec -it $MYSQLPOD --  /usr/bin/mc ls minio01/mysqlbkp
[2021-10-12 10:02:03 UTC] 161MiB employeesdb.sql
$>

Finally, it will be useful to create a cron job to regularly perform backups and upload them to S3.

Conclusion

MinIO is a great option as a primary or extended storage to your current infrastructure. With the high-performance object storage, you can store large data like database backups and archives at minimal cost without any vendor lock-in. Data recovery during any disaster strike becomes easy as you are in total control of your backup storage. MinIO server is easy to set up and can connect securely from a web interface using Let’s Encrypt SSL/TLS certificate. Lastly, and probably the most distinctive feature of the MinIO, is its S3 compatibility, making it work seamlessly with any S3 compliant tool.

Resources :

MinIO features

MinIO Client

MinIO installation

Thank You