k3s/.claude/skills/minio/s3.md

---
name: minio-s3-expert
description: 提供 MinIO 对象存储的配置、Bucket 管理及 S3 API 调用建议。
---

# MinIO S3 Object Storage Skill

## Architecture Overview

**Setup**: Caddy (HTTPS/SSL) → Traefik (routing) → MinIO (S3 storage)

- **MinIO**: S3-compatible object storage with web console
- **Caddy**: Handles HTTPS (443) with automatic SSL certificates
- **Traefik**: Routes HTTP traffic to MinIO services
- **Policy Manager**: Automatically sets new buckets to public-read (download) permission
- **Flow**: Internet → Caddy:443 (HTTPS) → Traefik:80 (HTTP) → MinIO (9000: API, 9001: Console)

## Quick Deployment Template

### 1. Complete MinIO Deployment YAML

```yaml
apiVersion: v1
kind: Namespace
metadata:
  name: minio
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: minio-data
  namespace: minio
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 50Gi
  storageClassName: local-path
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: minio
  namespace: minio
spec:
  replicas: 1
  selector:
    matchLabels:
      app: minio
  template:
    metadata:
      labels:
        app: minio
    spec:
      containers:
      - name: minio
        image: minio/minio:latest
        command:
        - /bin/sh
        - -c
        - minio server /data --console-address ":9001"
        ports:
        - containerPort: 9000
          name: api
        - containerPort: 9001
          name: console
        env:
        - name: MINIO_ROOT_USER
          value: "admin"
        - name: MINIO_ROOT_PASSWORD
          value: "your-password-here"
        - name: MINIO_SERVER_URL
          value: "https://s3.yourdomain.com"
        - name: MINIO_BROWSER_REDIRECT_URL
          value: "https://console.s3.yourdomain.com"
        volumeMounts:
        - name: data
          mountPath: /data
        livenessProbe:
          httpGet:
            path: /minio/health/live
            port: 9000
          initialDelaySeconds: 30
          periodSeconds: 10
        readinessProbe:
          httpGet:
            path: /minio/health/ready
            port: 9000
          initialDelaySeconds: 10
          periodSeconds: 5
      - name: policy-manager
        image: alpine:latest
        command:
        - /bin/sh
        - -c
        - |
          # Install MinIO Client
          wget https://dl.min.io/client/mc/release/linux-arm64/mc -O /usr/local/bin/mc
          chmod +x /usr/local/bin/mc

          # Wait for MinIO to start
          sleep 10

          # Configure mc client
          mc alias set myminio http://localhost:9000 ${MINIO_ROOT_USER} ${MINIO_ROOT_PASSWORD}

          echo "Policy manager started. Monitoring buckets..."

          # Continuously monitor and set bucket policies
          while true; do
            # Get all buckets
            mc ls myminio 2>/dev/null | awk '{print $NF}' | sed 's/\///' | while read -r BUCKET; do
              if [ -n "$BUCKET" ]; then
                # Check current policy
                POLICY_OUTPUT=$(mc anonymous get myminio/${BUCKET} 2>&1)

                # If private (contains "Access permission for" but not "download")
                if echo "$POLICY_OUTPUT" | grep -q "Access permission for" && ! echo "$POLICY_OUTPUT" | grep -q "download"; then
                  echo "Setting download policy for bucket: ${BUCKET}"
                  mc anonymous set download myminio/${BUCKET}
                fi
              fi
            done

            sleep 30
          done
        env:
        - name: MINIO_ROOT_USER
          value: "admin"
        - name: MINIO_ROOT_PASSWORD
          value: "your-password-here"
      volumes:
      - name: data
        persistentVolumeClaim:
          claimName: minio-data
---
apiVersion: v1
kind: Service
metadata:
  name: minio
  namespace: minio
spec:
  type: ClusterIP
  ports:
  - port: 9000
    targetPort: 9000
    name: api
  - port: 9001
    targetPort: 9001
    name: console
  selector:
    app: minio
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: minio-api
  namespace: minio
spec:
  ingressClassName: traefik
  rules:
  - host: s3.yourdomain.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: minio
            port:
              number: 9000
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: minio-console
  namespace: minio
spec:
  ingressClassName: traefik
  rules:
  - host: console.s3.yourdomain.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: minio
            port:
              number: 9001
```

### 2. Configuration Checklist

Before deploying, update these values in the YAML:

**Domains (4 places):**
- `s3.yourdomain.com` → Your S3 API domain
- `console.s3.yourdomain.com` → Your console domain

**Credentials (4 places):**
- `MINIO_ROOT_USER: "admin"` → Your admin username
- `MINIO_ROOT_PASSWORD: "your-password-here"` → Your admin password (min 8 chars)

**Architecture (1 place):**
- `linux-arm64` → Change based on your CPU:
  - ARM64: `linux-arm64`
  - x86_64: `linux-amd64`

**Storage (1 place):**
- `storage: 50Gi` → Adjust storage size as needed

## Deployment Steps

### 1. Prepare DNS

Point your domains to the server IP:
```bash
# Add DNS A records
s3.yourdomain.com        A    your-server-ip
console.s3.yourdomain.com A    your-server-ip
```

### 2. Configure Caddy

Add domains to Caddy ConfigMap:
```bash
kubectl edit configmap caddy-config -n default
```

Add these blocks:
```caddy
s3.yourdomain.com {
    reverse_proxy traefik.kube-system.svc.cluster.local:80 {
        header_up Host {host}
        header_up X-Real-IP {remote}
        header_up X-Forwarded-For {remote}
        header_up X-Forwarded-Proto {scheme}
    }
}

console.s3.yourdomain.com {
    reverse_proxy traefik.kube-system.svc.cluster.local:80 {
        header_up Host {host}
        header_up X-Real-IP {remote}
        header_up X-Forwarded-For {remote}
        header_up X-Forwarded-Proto {scheme}
    }
}
```

Reload Caddy:
```bash
kubectl exec -n default deployment/caddy -- caddy reload --config /etc/caddy/Caddyfile
```

### 3. Deploy MinIO

```bash
# Apply the configuration
kubectl apply -f minio.yaml

# Check deployment status
kubectl get pods -n minio

# Wait for pods to be ready
kubectl wait --for=condition=ready pod -l app=minio -n minio --timeout=300s
```

### 4. Verify Deployment

```bash
# Check MinIO logs
kubectl logs -n minio -l app=minio -c minio

# Check policy manager logs
kubectl logs -n minio -l app=minio -c policy-manager

# Check ingress
kubectl get ingress -n minio

# Check service
kubectl get svc -n minio
```

## Access MinIO

### Web Console
- URL: `https://console.s3.yourdomain.com`
- Username: Your configured `MINIO_ROOT_USER`
- Password: Your configured `MINIO_ROOT_PASSWORD`

### S3 API Endpoint
- URL: `https://s3.yourdomain.com`
- Use with AWS CLI, SDKs, or any S3-compatible client

## Bucket Policy Management

### Automatic Public-Read Policy

The policy manager sidecar automatically:
- Scans all buckets every 30 seconds
- Sets new private buckets to `download` (public-read) permission
- Allows anonymous downloads, requires auth for uploads/deletes

### Manual Policy Management

```bash
# Get pod name
POD=$(kubectl get pod -n minio -l app=minio -o jsonpath='{.items[0].metadata.name}')

# Access MinIO Client in pod
kubectl exec -n minio $POD -c policy-manager -- mc alias set myminio http://localhost:9000 admin your-password

# List buckets
kubectl exec -n minio $POD -c policy-manager -- mc ls myminio

# Check bucket policy
kubectl exec -n minio $POD -c policy-manager -- mc anonymous get myminio/bucket-name

# Set bucket to public-read (download)
kubectl exec -n minio $POD -c policy-manager -- mc anonymous set download myminio/bucket-name

# Set bucket to private
kubectl exec -n minio $POD -c policy-manager -- mc anonymous set private myminio/bucket-name

# Set bucket to public (read + write)
kubectl exec -n minio $POD -c policy-manager -- mc anonymous set public myminio/bucket-name
```

## Using MinIO

### Create Bucket via Web Console

1. Access `https://console.s3.yourdomain.com`
2. Login with credentials
3. Click "Buckets" → "Create Bucket"
4. Enter bucket name
5. Wait 30 seconds for auto-policy to apply

### Upload Files via Web Console

1. Navigate to bucket
2. Click "Upload" → "Upload File"
3. Select files
4. Files are immediately accessible via public URL

### Access Files

Public URL format:
```
https://s3.yourdomain.com/bucket-name/file-path
```

Example:
```bash
# Upload via console, then access:
curl https://s3.yourdomain.com/my-bucket/image.png
```

### Using AWS CLI

```bash
# Configure AWS CLI
aws configure set aws_access_key_id admin
aws configure set aws_secret_access_key your-password
aws configure set default.region us-east-1

# List buckets
aws --endpoint-url https://s3.yourdomain.com s3 ls

# Create bucket
aws --endpoint-url https://s3.yourdomain.com s3 mb s3://my-bucket

# Upload file
aws --endpoint-url https://s3.yourdomain.com s3 cp file.txt s3://my-bucket/

# Download file
aws --endpoint-url https://s3.yourdomain.com s3 cp s3://my-bucket/file.txt ./

# List bucket contents
aws --endpoint-url https://s3.yourdomain.com s3 ls s3://my-bucket/
```

### Using MinIO Client (mc)

```bash
# Install mc locally
wget https://dl.min.io/client/mc/release/linux-amd64/mc
chmod +x mc
sudo mv mc /usr/local/bin/

# Configure alias
mc alias set myminio https://s3.yourdomain.com admin your-password

# List buckets
mc ls myminio

# Create bucket
mc mb myminio/my-bucket

# Upload file
mc cp file.txt myminio/my-bucket/

# Download file
mc cp myminio/my-bucket/file.txt ./

# Mirror directory
mc mirror ./local-dir myminio/my-bucket/remote-dir
```

## Common Operations

### View Logs

```bash
# MinIO server logs
kubectl logs -n minio -l app=minio -c minio -f

# Policy manager logs
kubectl logs -n minio -l app=minio -c policy-manager -f

# Both containers
kubectl logs -n minio -l app=minio --all-containers -f
```

### Restart MinIO

```bash
# Graceful restart
kubectl rollout restart deployment/minio -n minio

# Force restart (delete pod)
kubectl delete pod -n minio -l app=minio
```

### Scale Storage

```bash
# Edit PVC (note: can only increase, not decrease)
kubectl edit pvc minio-data -n minio

# Update storage size
# Change: storage: 50Gi → storage: 100Gi
```

### Backup Data

```bash
# Get pod name
POD=$(kubectl get pod -n minio -l app=minio -o jsonpath='{.items[0].metadata.name}')

# Copy data from pod
kubectl cp minio/$POD:/data ./minio-backup -c minio

# Or use mc mirror
mc mirror myminio/bucket-name ./backup/bucket-name
```

### Restore Data

```bash
# Copy data to pod
kubectl cp ./minio-backup minio/$POD:/data -c minio

# Restart MinIO
kubectl rollout restart deployment/minio -n minio

# Or use mc mirror
mc mirror ./backup/bucket-name myminio/bucket-name
```

## Troubleshooting

### Pod Not Starting

```bash
# Check pod status
kubectl describe pod -n minio -l app=minio

# Check events
kubectl get events -n minio --sort-by='.lastTimestamp'

# Common issues:
# - PVC not bound (check storage class)
# - Image pull error (check network/registry)
# - Resource limits (check node resources)
```

### Cannot Access Web Console

```bash
# Check ingress
kubectl get ingress -n minio
kubectl describe ingress minio-console -n minio

# Check service
kubectl get svc -n minio

# Test from inside cluster
kubectl run -it --rm debug --image=curlimages/curl --restart=Never -- curl -v http://minio.minio.svc.cluster.local:9001

# Check Caddy logs
kubectl logs -n default -l app=caddy | grep -i s3

# Check Traefik logs
kubectl logs -n kube-system -l app.kubernetes.io/name=traefik
```

### SSL Certificate Issues

```bash
# Check Caddy certificates
kubectl exec -n default deployment/caddy -- caddy list-certificates

# Check Caddy logs for ACME
kubectl logs -n default deployment/caddy | grep -i "s3\|acme\|certificate"

# Verify DNS resolution
nslookup s3.yourdomain.com
nslookup console.s3.yourdomain.com
```

### Policy Manager Not Working

```bash
# Check policy manager logs
kubectl logs -n minio -l app=minio -c policy-manager

# Manually test mc commands
POD=$(kubectl get pod -n minio -l app=minio -o jsonpath='{.items[0].metadata.name}')
kubectl exec -n minio $POD -c policy-manager -- mc ls myminio

# Restart policy manager (restart pod)
kubectl delete pod -n minio -l app=minio
```

### Files Not Accessible

```bash
# Check bucket policy
kubectl exec -n minio $POD -c policy-manager -- mc anonymous get myminio/bucket-name

# Should show: Access permission for `myminio/bucket-name` is set to `download`

# If not, manually set
kubectl exec -n minio $POD -c policy-manager -- mc anonymous set download myminio/bucket-name

# Test access
curl -I https://s3.yourdomain.com/bucket-name/file.txt
```

## Advanced Configuration

### Custom Storage Class

```yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: minio-data
  namespace: minio
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 100Gi
  storageClassName: fast-ssd  # Custom storage class
```

### Resource Limits

```yaml
containers:
- name: minio
  image: minio/minio:latest
  resources:
    requests:
      memory: "512Mi"
      cpu: "500m"
    limits:
      memory: "2Gi"
      cpu: "2000m"
```

### Multiple Replicas (Distributed Mode)

For production, use distributed MinIO:
```yaml
# Requires 4+ nodes with persistent storage
command:
- /bin/sh
- -c
- minio server http://minio-{0...3}.minio.minio.svc.cluster.local/data --console-address ":9001"
```

### Custom Bucket Policies

Create custom policy JSON:
```json
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {"AWS": ["*"]},
      "Action": ["s3:GetObject"],
      "Resource": ["arn:aws:s3:::bucket-name/*"]
    }
  ]
}
```

Apply via mc:
```bash
kubectl exec -n minio $POD -c policy-manager -- mc anonymous set-json policy.json myminio/bucket-name
```

### Disable Auto-Policy Manager

Remove the `policy-manager` container from deployment if you want manual control.

## Best Practices

### Bucket Naming

- Use lowercase letters, numbers, hyphens
- Avoid underscores, spaces, special characters
- Keep names short and descriptive
- Example: `user-uploads`, `static-assets`, `backups-2024`

### Folder Structure

Use prefixes (folders) to organize files:
```
bucket-name/
├── user1/
│   ├── profile.jpg
│   └── documents/
├── user2/
│   └── avatar.png
└── shared/
    └── logo.png
```

### Security

- Change default credentials immediately
- Use strong passwords (16+ characters)
- Create separate access keys for applications
- Use bucket policies to restrict access
- Enable versioning for important buckets
- Regular backups of critical data

### Performance

- Use CDN for frequently accessed files
- Enable compression for text files
- Use appropriate storage class
- Monitor disk usage and scale proactively

## Quick Reference Commands

```bash
# Deploy MinIO
kubectl apply -f minio.yaml

# Check status
kubectl get pods -n minio
kubectl get svc -n minio
kubectl get ingress -n minio

# View logs
kubectl logs -n minio -l app=minio -c minio -f
kubectl logs -n minio -l app=minio -c policy-manager -f

# Restart MinIO
kubectl rollout restart deployment/minio -n minio

# Get pod name
POD=$(kubectl get pod -n minio -l app=minio -o jsonpath='{.items[0].metadata.name}')

# Access mc client
kubectl exec -n minio $POD -c policy-manager -- mc ls myminio

# Check bucket policy
kubectl exec -n minio $POD -c policy-manager -- mc anonymous get myminio/bucket-name

# Set bucket policy
kubectl exec -n minio $POD -c policy-manager -- mc anonymous set download myminio/bucket-name

# Delete deployment
kubectl delete -f minio.yaml
```

## Integration Examples

### Node.js (AWS SDK)

```javascript
const AWS = require('aws-sdk');

const s3 = new AWS.S3({
  endpoint: 'https://s3.yourdomain.com',
  accessKeyId: 'admin',
  secretAccessKey: 'your-password',
  s3ForcePathStyle: true,
  signatureVersion: 'v4'
});

// Upload file
s3.putObject({
  Bucket: 'my-bucket',
  Key: 'file.txt',
  Body: 'Hello World'
}, (err, data) => {
  if (err) console.error(err);
  else console.log('Uploaded:', data);
});

// Download file
s3.getObject({
  Bucket: 'my-bucket',
  Key: 'file.txt'
}, (err, data) => {
  if (err) console.error(err);
  else console.log('Content:', data.Body.toString());
});
```

### Python (boto3)

```python
import boto3

s3 = boto3.client('s3',
    endpoint_url='https://s3.yourdomain.com',
    aws_access_key_id='admin',
    aws_secret_access_key='your-password'
)

# Upload file
s3.upload_file('local-file.txt', 'my-bucket', 'remote-file.txt')

# Download file
s3.download_file('my-bucket', 'remote-file.txt', 'downloaded.txt')

# List objects
response = s3.list_objects_v2(Bucket='my-bucket')
for obj in response.get('Contents', []):
    print(obj['Key'])
```

### Go (minio-go)

```go
package main

import (
    "github.com/minio/minio-go/v7"
    "github.com/minio/minio-go/v7/pkg/credentials"
)

func main() {
    client, _ := minio.New("s3.yourdomain.com", &minio.Options{
        Creds:  credentials.NewStaticV4("admin", "your-password", ""),
        Secure: true,
    })

    // Upload file
    client.FPutObject(ctx, "my-bucket", "file.txt", "local-file.txt", minio.PutObjectOptions{})

    // Download file
    client.FGetObject(ctx, "my-bucket", "file.txt", "downloaded.txt", minio.GetObjectOptions{})
}
```

## Notes

- MinIO is fully S3-compatible
- Automatic SSL via Caddy
- Auto-policy sets buckets to public-read by default
- Policy manager runs every 30 seconds
- Persistent storage required for data retention
- Single replica suitable for development/small deployments
- Use distributed mode for production high-availability
- Regular backups recommended for critical data