MinIO [Link] is an open source [Link] high performance object storage API written in GO compatible with Amazon S3 cloud storage. It can handle unstructured data (photos, videos, log files, backups, container images, etc) with size up to 5 TB per object.

Its enterprise class features are capable of strip files in multiple disks and reliably recover the data even under the failure of multiple drives. And based on the implemented parity/hash logic it heals corrupted files on the fly ensuring data integrity.

Moreover, MinIO features server-site encryption, continuous site replication, bucket versioning, deletion protection, event notification, lifecycle management, access rules, work as a cluster, integration to public cloud services…


DEPLOYMENT METHODS

  • From a binary:
    • You literally download the binary version to your CPU architecture and run it with no installation required or install it from the package manager.
  • From a container:
    • With a single line command a new instance of the application can be executed. Great for evaluation and development.
  • From the source code:
    • With the usage of the Golang native features, you can easily pulls and compiles the binary that can be executed. The most secure way.

BINARY EXECUTION

  • Linux
    • export MINIO_ROOT_USER=admin
      export MINIO_ROOT_PASSWORD=strongpassword
      wget https://dl.min.io/server/minio/release/linux-amd64/minio
      chmod +x minio
      ./minio server /PATH/data --console-address ":9001"
  • ARM (Raspberry PI)
    • export MINIO_ROOT_USER=admin
      export MINIO_ROOT_PASSWORD=strongpassword
      wget https://dl.min.io/server/minio/release/linux-arm64/minio
      chmod +x minio
      ./minio server /PATH/data --console-address ":9001"
  • MacOS
    • export MINIO_ROOT_USER=admin
      export MINIO_ROOT_PASSWORD=strongpassword
      wget https://dl.min.io/server/minio/release/darwin-amd64/minio
      chmod +x minio
      ./minio server /PATH/data --console-address ":9001"
  • Windows
    • PS> Invoke-WebRequest -Uri "https://dl.min.io/server/minio/release/windows-amd64/minio.exe" -OutFile "C:\minio.exe"
      PS> setx MINIO_ROOT_USER admin
      PS> setx MINIO_ROOT_PASSWORD strongpassword
      PS> C:\minio.exe server D:\Data --console-address ":9001"

PACKAGE MANAGER INSTALLATION

  • Linux
    • Or for Debian-based distributions:
      • wget https://dl.min.io/server/minio/release/linux-amd64/minio_20220620231345.0.0_amd64.deb
        sudo dpkg -i minio_20220620231345.0.0_amd64.deb
    • Or for RHEL-based distributions:
      • wget https://dl.min.io/server/minio/release/linux-amd64/minio-20220620231345.0.0.x86_64.rpm
        sudo rpm -i minio-20220620231345.0.0.x86_64.rpm
  • ARM (Raspberry PI)
    • Or for Debian-based distributions:
      • wget https://dl.min.io/server/minio/release/linux-arm64/minio_20220620231345.0.0_arm64.deb
        sudo dpkg -i minio_20220620231345.0.0_amd64.deb
    • Or for RHEL-based distributions:
      • wget https://dl.min.io/server/minio/release/linux-arm64/minio-20220620231345.0.0.aarch64.rpm
        sudo rpm -i minio-20220620231345.0.0.x86_64.rpm
  • MacOS
    • brew install minio/stable/minio
      minio server start

DOCKER DEPLOYMENT

  • Using Docker
    • sudo docker run -p 9000:9000 -p 9001:9001 -v /PATH/data:/data quay.io/minio/minio server /PATH/data --console-address ":9001"
  • Using Podman:
    • podman run -p 9000:9000 -p 9001:9001 -v /PATH/data:/data quay.io/minio/minio server /PATH/data --console-address ":9001"

COMPILING THE SOURCE CODE

  • Using GO
    • GO111MODULE=on go install github.com/minio/minio@latest
  • Manually
    • git clone https://github.com/minio/minio.git
      cd minio/
      make
      make install

FIREWALL SETTINGS

  • UFW
    • sudo ufw allow 9000:9001/tcp
  • Firewall-CMD
    • sudo firewall-cmd --zone=public --add-port=9000-9001/tcp --permanent
      sudo firewall-cmd --reload

BONUS

  • Securing
    • sudo useradd -r minioUser -s /sbin/nologin
      sudo mkdir -p /PATH/data
      sudo chown minioUser:minioUser /PATH/data
      sudo chmod 770 /PATH/data
  • Configuration File
    • sudo nano /etc/default/minio
    • MINIO_ROOT_USER=admin
      MINIO_ROOT_PASSWORD=strongpassword 
      MINIO_VOLUMES="/PATH/data"
      MINIO_OPTS="--certs-dir /home/minioUser/.minio/certs --console-address :9001"
    • sudo chown minioUser:minioUser /etc/default/minio
      sudo chmod 770 /etc/default/minio
  • Running as a Service
    • sudo nano /etc/systemd/system/minio.service
    • [Unit]
      Description=MinIO
      Wants=network-online.target
      After=network-online.target
      
      [Service]
      WorkingDirectory=/usr/local/
      User=minioUser
      Group=minioUser
      ProtectProc=invisible
      
      EnvironmentFile=/etc/default/minio
      ExecStart=/usr/local/bin/minio server $MINIO_OPTS $MINIO_VOLUMES
      Restart=always
      LimitNOFILE=1048576
      TasksMax=infinity
      TimeoutStopSec=infinity
      SendSIGKILL=no
      
      [Install]
      WantedBy=multi-user.target
    • sudo systemctl daemon-reload
      sudo systemctl enable minio
      sudo systemctl start minio
      sudo systemctl status minio
  • MinIO Client
    • Download
      • wget https://dl.min.io/client/mc/release/linux-amd64/mc
        chmod +x mc
        ./mc --help
        sudo mv mc /usr/local/bin
        mc --help
    • Configuration
      • ./mc alias set <ALIAS> http://<YOUR-S3-ENDPOINT>:9000 <YOUR-ACCESS-KEY> <YOUR-SECRET-KEY>
        ./mc ls <ALIAS>
        
    • Basic commands (<ALIAS> = s3)
      • mc ls s3
        • List buckets.
      • mc mb s3/new-bucket
        • Make/create a new bucket.
      • mc rm s3/new-bucket
        • Remove/delete a bucket.
      • mc ls s3/bucket
        • List content of the bucket.
      • mc cp localFile s3/bucket/remoteFile
        • Copy file from Local to Remote.
      • mc cp s3/bucket/remoteFile localFile
        • Copy file from Remote to Local.
      • mc cp s3/backup/fileName s3/backup/copiedFile
        • Copy file from and to the same bucket (same for between buckets).
      • mc cp s3/backup/fileName s3/backup/newFIle
        • Copy file in the same bucket (same for between buckets).
      • mc mv s3/backup/fileName s3/backup/renamedFile
        • Rename (or move) file in the same bucket (same for between buckets).
      • mc rm s3/backup/fileToRemove
        • Remove/delete a file (object).
      • mc find s3 –name “*.zip
        • Find/search in all buckets for a file by name (also by regex, max depth, age…).
      • mc find s3/bucket –name “*.zip” –exec “mc cp {} .”
        • Spawn an external process for each matching object to copy it from to local (or any other action).
      • mc mirror backup/ s3/backup
        • Synchronize a local directory recursively with a remote location (also between remote locations).
      • mc update
        • Update the client to the latest
    • Additional commands
      • cat
      • head
      • pipe
      • sql
      • stat
      • tree
      • du
      • retention
      • legalhold
      • share
      • version
      • ilm
      • encrypt
      • event
      • watch
      • undo
      • anonymous
      • tag
      • diff
      • replicate
    • Administrative commands
      • mc admin config get s3 site
        • Get config information.
      • mc admin config set s3 site name=mys3
        • Set config information.
      • mc admin service restart s3
        • Restart the service.
    •  Upgrading
      • mc admin update s3
    • Backups
      • tar zcf - /PATH | mc pipe -q <ALIAS>/<BUCKET>/fileName.tar.gz
        mysqldump --no-tablespaces --lock-tables=false dbName | gzip -c | mc pipe -q <ALIAS>/<BUCKET>/fileName.tar.gz
    • Versioning and Life Cycle
      • mc mb --with-lock <ALIAS>/<BUCKET>
        mc version enable <ALIAS>/<BUCKET>
        mc ilm rule add --noncurrent-expire-days 30 <ALIAS>/<BUCKET>
        mc ilm rule add --expire-days 90 --noncurrent-expire-days 30 <ALIAS>/<BUCKET>
        mc ls --versions <ALIAS>/<BUCKET>
  •  Encrypting
    • I recommend using a web server between (such as Apache or NGINX – or event Varnish) the client and the MinIO server to facilitate the management of  certificates, virtual hosts, load balance traffic, cache…
    • But MinIO has its own resources for encrypted communication with self-signed certificate or public certificate (from Let’s Encrypt) using CertBot, for example:
      • sudo apt install snapd
        sudo snap install core 
        sudo snap refresh core
        sudo snap install --classic certbot
        sudo ln -s /snap/bin/certbot /usr/bin/certbot
        sudo certbot certonly --standalone --agree-tos --no-eff-email --staple-ocsp --preferred-challenges http -m [email protected] -d minio.domain.com
        sudo cp /etc/letsencrypt/live/minio.domain.com/privkey.pem /home/minioUser/.minio/certs/private.key
        sudo cp /etc/letsencrypt/live/minio.domain.com/fullchain.pem /home/minioUser/.minio/certs/public.crt
        sudo ufw allow 80
        sudo ufw allow 443
      • sudo nano /etc/default/minio
      • MINIO_SERVER_URL="https://minio.domain.com:9000"
      • sudo systemctl restart minio
  • Optional
    • export MINIO_DOMAIN=domain.com
      export MINIO_SITE_NAME=mysite
      export MINIO_SITE_REGION=global
      export MINIO_SITE_COMMENT="My Block Storage"
      export MINIO_API_REQUESTS_MAX=1600
      export MINIO_API_REQUESTS_DEADLINE=1m
      export MINIO_API_CORS_ALLOW_ORIGIN="https://domain-a.com,https://domain-b.com"
      export MINIO_API_REMOTE_TRANSPORT_DEADLINE=2h
      export MINIO_BROWSER=off
      
  • Multiple resources
    • Maps the resource based on the host address.
      • minio server https://minio-{1...4}.example.net/mnt/disk-{1...4} --console-address ":9001"
  • Directory Tree
    • While the the certificates are stored inside the home by default inside the home directory of the user that runs the binary:
      • /home/minioUser/.minio/
        • └── certs
              └── CAs
          
    • All the configuration for the instance is stored in another hidden directory inside the data directory (volume directory) as follows:
      • /PATH/data/.minio.sys/
        • ├── buckets
          ├── config
          │   ├── config.json
          │   │   └── xl.meta
          │   └── iam
          │       ├── format.json
          │       │   └── xl.meta
          │       ├── policydb
          │       │   └── users
          │       │       └── username.json
          │       │           └── xl.meta
          │       ├── sts
          │       │   ├── 2G67ZRC44SCKLVLLZVGT
          │       │   │   └── identity.json
          │       │   │       └── xl.meta
          │       │   ├── 3KEUK011NRQIOOS0AK0R
          │       │   │   └── identity.json
          │       │   │       └── xl.meta
          │       │   └── AAQ83Z1Z279SWO7VMH5T
          │       │       └── identity.json
          │       │           └── xl.meta
          │       └── users
          │           └── username
          │               └── identity.json
          │                   └── xl.meta
          ├── format.json
          ├── ilm
          │   └── deletion-journal.bin
          ├── multipart
          ├── tmp
          └── tmp-old

READ MORE

Configure a Reverse Proxy with HTTP for the MinIO [Link].

Configure a Reverse Proxy with Cache for the MinIO [Link].

Check the awesome backup tools Duplicati (open source) [Link], Duplicity [Link] and Duplicacy [Link] that are S3 compatible, among many other features.