首次提交：初始化项目
This commit is contained in:
fei
389
.claude/skills/caddy/SKILL.md
Normal file
389
.claude/skills/caddy/SKILL.md
Normal file
@@ -0,0 +1,389 @@
|
||||
---
|
||||
name: caddy-ssl-termination
|
||||
description: 专门用于 Traefik 前置 Caddy 进行 SSL 卸载的架构配置，适用于 K3s 环境。
|
||||
---
|
||||
|
||||
|
||||
# Caddy SSL Termination Skill
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
**Setup**: Traefik (routing) → Caddy (HTTPS/SSL termination) → HTTP backend
|
||||
|
||||
- **Caddy**: Handles HTTPS (443) with automatic SSL certificates, forwards to Traefik on HTTP (80)
|
||||
- **Traefik**: Routes HTTP traffic to appropriate backend services
|
||||
- **Flow**: Internet → Caddy:443 (HTTPS) → Traefik:80 (HTTP) → Backend Pods
|
||||
|
||||
## Quick Configuration Template
|
||||
|
||||
### 1. Basic Caddyfile Structure
|
||||
|
||||
```caddy
|
||||
# /etc/caddy/Caddyfile
|
||||
|
||||
# Domain configuration
|
||||
example.com {
|
||||
reverse_proxy traefik-service:80
|
||||
}
|
||||
|
||||
# Multiple domains
|
||||
app1.example.com {
|
||||
reverse_proxy traefik-service:80
|
||||
}
|
||||
|
||||
app2.example.com {
|
||||
reverse_proxy traefik-service:80
|
||||
}
|
||||
|
||||
# Wildcard subdomain (requires DNS wildcard)
|
||||
*.example.com {
|
||||
reverse_proxy traefik-service:80
|
||||
}
|
||||
```
|
||||
|
||||
### 2. ConfigMap for Caddyfile
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: caddy-config
|
||||
namespace: default
|
||||
data:
|
||||
Caddyfile: |
|
||||
# Global options
|
||||
{
|
||||
email your-email@example.com
|
||||
# Use Let's Encrypt staging for testing
|
||||
# acme_ca https://acme-staging-v02.api.letsencrypt.org/directory
|
||||
}
|
||||
|
||||
# Your domains
|
||||
example.com {
|
||||
reverse_proxy traefik-service:80 {
|
||||
header_up Host {host}
|
||||
header_up X-Real-IP {remote}
|
||||
header_up X-Forwarded-For {remote}
|
||||
header_up X-Forwarded-Proto {scheme}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 3. Caddy Deployment
|
||||
|
||||
```yaml
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: caddy
|
||||
namespace: default
|
||||
spec:
|
||||
replicas: 1
|
||||
selector:
|
||||
matchLabels:
|
||||
app: caddy
|
||||
template:tadata:
|
||||
labels:
|
||||
app: caddy
|
||||
spec:
|
||||
containers:
|
||||
- name: caddy
|
||||
image: caddy:latest
|
||||
ports:
|
||||
- containerPort: 80
|
||||
- containerPort: 443
|
||||
- containerPort: 2019 # Admin API
|
||||
volumeMounts:
|
||||
- name: config
|
||||
mountPath: /etc/caddy
|
||||
- name: data
|
||||
mountPath: /data
|
||||
- name: config-cache
|
||||
mountPath: /config
|
||||
volumes:
|
||||
- name: config
|
||||
configMap:
|
||||
name: caddy-config
|
||||
- name: data
|
||||
persistentVolumeClaim:
|
||||
claimName: caddy-data
|
||||
- name: config-cache
|
||||
emptyDir: {}
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: caddy
|
||||
namespace: default
|
||||
spec:
|
||||
type: LoadBalancer # or NodePort
|
||||
ports:
|
||||
- name: http
|
||||
port: 80
|
||||
targetPort: 80
|
||||
- name: https
|
||||
port: 443
|
||||
targetPort: 443
|
||||
selector:
|
||||
app: caddy
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: PersistentVolumeClaim
|
||||
metadata:
|
||||
name: caddy-data
|
||||
namespace: default
|
||||
spec:
|
||||
accessModes:
|
||||
- ReadWriteOnce
|
||||
resources:
|
||||
requests:
|
||||
storage: 1Gi
|
||||
```
|
||||
|
||||
## Common Operations
|
||||
|
||||
### Reload Configuration
|
||||
|
||||
After updating the ConfigMap:
|
||||
|
||||
```bash
|
||||
# Method 1: Reload via exec (preserves connections)
|
||||
kubectl exec -n default deployment/caddy -- caddy reload --config /etc/caddy/Caddyfile
|
||||
|
||||
# Method 2: Restart pod (brief downtime)
|
||||
kubectl rollout restart deployment/caddy -n default
|
||||
|
||||
# Method 3: Delete pod (auto-recreates)
|
||||
kubectl delete pod -n default -l app=caddy
|
||||
```
|
||||
|
||||
### Update Caddyfile
|
||||
|
||||
```bash
|
||||
# Edit ConfigMap
|
||||
kubectl edit configmap caddy-config -n default
|
||||
|
||||
# Or apply updated file
|
||||
kubectl apply -f caddy-configmap.yaml
|
||||
|
||||
# Then reload
|
||||
kubectl exec -n default deployment/caddy -- caddy reload --config /etc/caddy/Caddyfile
|
||||
```
|
||||
|
||||
### View Logs
|
||||
|
||||
```bash
|
||||
# Follow logs
|
||||
kubectl logs -n default -f deployment/caddy
|
||||
|
||||
# Check SSL certificate issues
|
||||
kubectl logs -n default deployment/caddy | grep -i "certificate\|acme\|tls"
|
||||
```
|
||||
|
||||
### Check Configuration
|
||||
|
||||
```bash
|
||||
# Validate Caddyfile syntax
|
||||
kubectl exec -n default deployment/caddy -- caddy validate --config /etc/caddy/Caddyfile
|
||||
|
||||
# View current config via API
|
||||
kubectl exec -n default deployment/caddy -- curl localhost:2019/config/
|
||||
```
|
||||
|
||||
## Adding New Domain
|
||||
|
||||
### Step-by-step Process
|
||||
|
||||
1. **Update DNS**: Point new domain to Caddy's LoadBalancer IP
|
||||
```bash
|
||||
kubectl get svc caddy -n default -o jsonpath='{.status.loadBalancer.ingress[0].ip}'
|
||||
```
|
||||
|
||||
2. **Update ConfigMap**: Add new domain block
|
||||
```bash
|
||||
kubectl edit configmap caddy-config -n default
|
||||
```
|
||||
|
||||
Add:
|
||||
```caddy
|
||||
newapp.example.com {
|
||||
reverse_proxy traefik-service:80 {
|
||||
header_up Host {host}
|
||||
header_up X-Real-IP {remote}
|
||||
header_up X-Forwarded-For {remote}
|
||||
header_up X-Forwarded-Proto {scheme}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
3. **Reload Caddy**:
|
||||
```bash
|
||||
kubectl exec -n default deployment/caddy -- caddy reload --config /etc/caddy/Caddyfile
|
||||
```
|
||||
|
||||
4. **Verify**: Check logs for certificate acquisition
|
||||
```bash
|
||||
kubectl logs -n default deployment/caddy | tail -20
|
||||
```
|
||||
|
||||
## Traefik Integration
|
||||
|
||||
### Traefik IngressRoute Example
|
||||
|
||||
```yaml
|
||||
apiVersion: traefik.containo.us/v1alpha1
|
||||
kind: IngressRoute
|
||||
metadata:
|
||||
name: myapp
|
||||
namespace: default
|
||||
spec:
|
||||
entryPoints:
|
||||
- web # HTTP only, Caddy handles HTTPS
|
||||
routes:
|
||||
- match: Host(`myapp.example.com`)
|
||||
kind: Rule
|
||||
services:
|
||||
- name: myapp-service
|
||||
port: 8080
|
||||
```
|
||||
|
||||
### Important Notes
|
||||
|
||||
- Traefik should listen on HTTP (80) only
|
||||
- Caddy handles all HTTPS/SSL
|
||||
- Use `Host()` matcher in Traefik to route by domain
|
||||
- Caddy forwards the original `Host` header to Traefik
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### SSL Certificate Issues
|
||||
|
||||
```bash
|
||||
# Check certificate status
|
||||
kubectl exec -n default deployment/caddy -- caddy list-certificates
|
||||
|
||||
# View ACME logs
|
||||
kubectl logs -n default deployment/caddy | grep -i acme
|
||||
|
||||
# Common issues:
|
||||
# - Port 80/443 not accessible from internet
|
||||
# - DNS not pointing to correct IP
|
||||
# - Rate limit hit (use staging CA for testing)
|
||||
```
|
||||
|
||||
### Configuration Errors
|
||||
|
||||
```bash
|
||||
# Test config before reload
|
||||
kubectl exec -n default deployment/caddy -- caddy validate --config /etc/caddy/Caddyfile
|
||||
|
||||
# Check for syntax errors
|
||||
kubectl logs -n default deployment/caddy | grep -i error
|
||||
```
|
||||
|
||||
### Connection Issues
|
||||
|
||||
```bash
|
||||
# Test from inside cluster
|
||||
kubectl run -it --rm debug --image=curlimages/curl --restart=Never -- curl -v http://traefik-service:80
|
||||
|
||||
# Check if Caddy can reach Traefik
|
||||
kubectl exec -n default deployment/caddy -- curl -v http://traefik-service:80
|
||||
```
|
||||
|
||||
## Advanced Configurations
|
||||
|
||||
### Custom TLS Settings
|
||||
|
||||
```caddy
|
||||
example.com {
|
||||
tls {
|
||||
protocols tls1.2 tls1.3
|
||||
ciphers TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384 TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
|
||||
}
|
||||
reverse_proxy traefik-service:80
|
||||
}
|
||||
```
|
||||
|
||||
### Rate Limiting
|
||||
|
||||
```caddy
|
||||
example.com {
|
||||
rate_limit {
|
||||
zone dynamic {
|
||||
key {remote_host}
|
||||
events 100
|
||||
window 1m
|
||||
}
|
||||
}
|
||||
reverse_proxy traefik-service:80
|
||||
}
|
||||
```
|
||||
|
||||
### Custom Error Pages
|
||||
|
||||
```caddy
|
||||
example.com {
|
||||
handle_errors {
|
||||
respond "{err.status_code} {err.status_text}"
|
||||
}
|
||||
reverse_proxy traefik-service:80
|
||||
}
|
||||
```
|
||||
|
||||
### Health Checks
|
||||
|
||||
```caddy
|
||||
example.com {
|
||||
reverse_proxy traefik-service:80 {
|
||||
health_uri /health
|
||||
health_interval 10s
|
||||
health_timeout 5s
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## Quick Reference Commands
|
||||
|
||||
```bash
|
||||
# Get Caddy pod name
|
||||
kubectl get pods -n default -l app=caddy
|
||||
|
||||
# Reload config
|
||||
kubectl exec -n default deployment/caddy -- caddy reload --config /etc/caddy/Caddyfile
|
||||
|
||||
# View current config
|
||||
kubectl exec -n default deployment/caddy -- cat /etc/caddy/Caddyfile
|
||||
|
||||
# Check certificates
|
||||
kubectl exec -n default deployment/caddy -- caddy list-certificates
|
||||
|
||||
# Restart Caddy
|
||||
kubectl rollout restart deployment/caddy -n default
|
||||
|
||||
# Watch logs
|
||||
kubectl logs -n default -f deployment/caddy
|
||||
|
||||
# Get LoadBalancer IP
|
||||
kubectl get svc caddy -n default -o jsonpath='{.status.loadBalancer.ingress[0].ip}'
|
||||
```
|
||||
|
||||
## Checklist for New Service
|
||||
|
||||
- [ ] DNS A/AAAA record points to Caddy LoadBalancer IP
|
||||
- [ ] Domain added to Caddy ConfigMap
|
||||
- [ ] Caddy reloaded successfully
|
||||
- [ ] Traefik IngressRoute created (HTTP only)
|
||||
- [ ] Backend service accessible from Traefik
|
||||
- [ ] SSL certificate acquired (check logs)
|
||||
- [ ] HTTPS access working from browser
|
||||
- [ ] HTTP redirects to HTTPS (Caddy default behavior)
|
||||
|
||||
## Notes
|
||||
|
||||
- Caddy automatically obtains and renews SSL certificates
|
||||
- Certificates stored in `/data` volume (must be persistent)
|
||||
- Reload is graceful - no connection drops
|
||||
- Caddy handles HTTP→HTTPS redirect automatically
|
||||
- Use staging CA for testing to avoid rate limits
|
||||
- Wildcard certificates require DNS challenge (more complex setup)
|
||||
14
.claude/skills/environment/sudo-pref.md
Normal file
14
.claude/skills/environment/sudo-pref.md
Normal file
@@ -0,0 +1,14 @@
|
||||
---
|
||||
name: sudo-automated-preference
|
||||
description: 强制 Claude Code 在需要 sudo 权限执行命令时，使用特定的管道方式自动输入密码。
|
||||
---
|
||||
|
||||
# Sudo Automation Skill
|
||||
|
||||
## Usage Guidelines
|
||||
> 当你需要执行任何需要 `sudo` 权限的 shell 命令时，请严格遵守此格式。
|
||||
|
||||
## 强制命令格式
|
||||
严禁直接使用 `sudo <command>`，必须使用以下格式：
|
||||
```bash
|
||||
echo "1" | sudo -S <command>
|
||||
789
.claude/skills/minio/s3.md
Normal file
789
.claude/skills/minio/s3.md
Normal file
@@ -0,0 +1,789 @@
|
||||
---
|
||||
name: minio-s3-expert
|
||||
description: 提供 MinIO 对象存储的配置、Bucket 管理及 S3 API 调用建议。
|
||||
---
|
||||
|
||||
# MinIO S3 Object Storage Skill
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
**Setup**: Caddy (HTTPS/SSL) → Traefik (routing) → MinIO (S3 storage)
|
||||
|
||||
- **MinIO**: S3-compatible object storage with web console
|
||||
- **Caddy**: Handles HTTPS (443) with automatic SSL certificates
|
||||
- **Traefik**: Routes HTTP traffic to MinIO services
|
||||
- **Policy Manager**: Automatically sets new buckets to public-read (download) permission
|
||||
- **Flow**: Internet → Caddy:443 (HTTPS) → Traefik:80 (HTTP) → MinIO (9000: API, 9001: Console)
|
||||
|
||||
## Quick Deployment Template
|
||||
|
||||
### 1. Complete MinIO Deployment YAML
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: Namespace
|
||||
metadata:
|
||||
name: minio
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: PersistentVolumeClaim
|
||||
metadata:
|
||||
name: minio-data
|
||||
namespace: minio
|
||||
spec:
|
||||
accessModes:
|
||||
- ReadWriteOnce
|
||||
resources:
|
||||
requests:
|
||||
storage: 50Gi
|
||||
storageClassName: local-path
|
||||
---
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: minio
|
||||
namespace: minio
|
||||
spec:
|
||||
replicas: 1
|
||||
selector:
|
||||
matchLabels:
|
||||
app: minio
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: minio
|
||||
spec:
|
||||
containers:
|
||||
- name: minio
|
||||
image: minio/minio:latest
|
||||
command:
|
||||
- /bin/sh
|
||||
- -c
|
||||
- minio server /data --console-address ":9001"
|
||||
ports:
|
||||
- containerPort: 9000
|
||||
name: api
|
||||
- containerPort: 9001
|
||||
name: console
|
||||
env:
|
||||
- name: MINIO_ROOT_USER
|
||||
value: "admin"
|
||||
- name: MINIO_ROOT_PASSWORD
|
||||
value: "your-password-here"
|
||||
- name: MINIO_SERVER_URL
|
||||
value: "https://s3.yourdomain.com"
|
||||
- name: MINIO_BROWSER_REDIRECT_URL
|
||||
value: "https://console.s3.yourdomain.com"
|
||||
volumeMounts:
|
||||
- name: data
|
||||
mountPath: /data
|
||||
livenessProbe:
|
||||
httpGet:
|
||||
path: /minio/health/live
|
||||
port: 9000
|
||||
initialDelaySeconds: 30
|
||||
periodSeconds: 10
|
||||
readinessProbe:
|
||||
httpGet:
|
||||
path: /minio/health/ready
|
||||
port: 9000
|
||||
initialDelaySeconds: 10
|
||||
periodSeconds: 5
|
||||
- name: policy-manager
|
||||
image: alpine:latest
|
||||
command:
|
||||
- /bin/sh
|
||||
- -c
|
||||
- |
|
||||
# Install MinIO Client
|
||||
wget https://dl.min.io/client/mc/release/linux-arm64/mc -O /usr/local/bin/mc
|
||||
chmod +x /usr/local/bin/mc
|
||||
|
||||
# Wait for MinIO to start
|
||||
sleep 10
|
||||
|
||||
# Configure mc client
|
||||
mc alias set myminio http://localhost:9000 ${MINIO_ROOT_USER} ${MINIO_ROOT_PASSWORD}
|
||||
|
||||
echo "Policy manager started. Monitoring buckets..."
|
||||
|
||||
# Continuously monitor and set bucket policies
|
||||
while true; do
|
||||
# Get all buckets
|
||||
mc ls myminio 2>/dev/null | awk '{print $NF}' | sed 's/\///' | while read -r BUCKET; do
|
||||
if [ -n "$BUCKET" ]; then
|
||||
# Check current policy
|
||||
POLICY_OUTPUT=$(mc anonymous get myminio/${BUCKET} 2>&1)
|
||||
|
||||
# If private (contains "Access permission for" but not "download")
|
||||
if echo "$POLICY_OUTPUT" | grep -q "Access permission for" && ! echo "$POLICY_OUTPUT" | grep -q "download"; then
|
||||
echo "Setting download policy for bucket: ${BUCKET}"
|
||||
mc anonymous set download myminio/${BUCKET}
|
||||
fi
|
||||
fi
|
||||
done
|
||||
|
||||
sleep 30
|
||||
done
|
||||
env:
|
||||
- name: MINIO_ROOT_USER
|
||||
value: "admin"
|
||||
- name: MINIO_ROOT_PASSWORD
|
||||
value: "your-password-here"
|
||||
volumes:
|
||||
- name: data
|
||||
persistentVolumeClaim:
|
||||
claimName: minio-data
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: minio
|
||||
namespace: minio
|
||||
spec:
|
||||
type: ClusterIP
|
||||
ports:
|
||||
- port: 9000
|
||||
targetPort: 9000
|
||||
name: api
|
||||
- port: 9001
|
||||
targetPort: 9001
|
||||
name: console
|
||||
selector:
|
||||
app: minio
|
||||
---
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: Ingress
|
||||
metadata:
|
||||
name: minio-api
|
||||
namespace: minio
|
||||
spec:
|
||||
ingressClassName: traefik
|
||||
rules:
|
||||
- host: s3.yourdomain.com
|
||||
http:
|
||||
paths:
|
||||
- path: /
|
||||
pathType: Prefix
|
||||
backend:
|
||||
service:
|
||||
name: minio
|
||||
port:
|
||||
number: 9000
|
||||
---
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: Ingress
|
||||
metadata:
|
||||
name: minio-console
|
||||
namespace: minio
|
||||
spec:
|
||||
ingressClassName: traefik
|
||||
rules:
|
||||
- host: console.s3.yourdomain.com
|
||||
http:
|
||||
paths:
|
||||
- path: /
|
||||
pathType: Prefix
|
||||
backend:
|
||||
service:
|
||||
name: minio
|
||||
port:
|
||||
number: 9001
|
||||
```
|
||||
|
||||
### 2. Configuration Checklist
|
||||
|
||||
Before deploying, update these values in the YAML:
|
||||
|
||||
**Domains (4 places):**
|
||||
- `s3.yourdomain.com` → Your S3 API domain
|
||||
- `console.s3.yourdomain.com` → Your console domain
|
||||
|
||||
**Credentials (4 places):**
|
||||
- `MINIO_ROOT_USER: "admin"` → Your admin username
|
||||
- `MINIO_ROOT_PASSWORD: "your-password-here"` → Your admin password (min 8 chars)
|
||||
|
||||
**Architecture (1 place):**
|
||||
- `linux-arm64` → Change based on your CPU:
|
||||
- ARM64: `linux-arm64`
|
||||
- x86_64: `linux-amd64`
|
||||
|
||||
**Storage (1 place):**
|
||||
- `storage: 50Gi` → Adjust storage size as needed
|
||||
|
||||
## Deployment Steps
|
||||
|
||||
### 1. Prepare DNS
|
||||
|
||||
Point your domains to the server IP:
|
||||
```bash
|
||||
# Add DNS A records
|
||||
s3.yourdomain.com A your-server-ip
|
||||
console.s3.yourdomain.com A your-server-ip
|
||||
```
|
||||
|
||||
### 2. Configure Caddy
|
||||
|
||||
Add domains to Caddy ConfigMap:
|
||||
```bash
|
||||
kubectl edit configmap caddy-config -n default
|
||||
```
|
||||
|
||||
Add these blocks:
|
||||
```caddy
|
||||
s3.yourdomain.com {
|
||||
reverse_proxy traefik.kube-system.svc.cluster.local:80 {
|
||||
header_up Host {host}
|
||||
header_up X-Real-IP {remote}
|
||||
header_up X-Forwarded-For {remote}
|
||||
header_up X-Forwarded-Proto {scheme}
|
||||
}
|
||||
}
|
||||
|
||||
console.s3.yourdomain.com {
|
||||
reverse_proxy traefik.kube-system.svc.cluster.local:80 {
|
||||
header_up Host {host}
|
||||
header_up X-Real-IP {remote}
|
||||
header_up X-Forwarded-For {remote}
|
||||
header_up X-Forwarded-Proto {scheme}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Reload Caddy:
|
||||
```bash
|
||||
kubectl exec -n default deployment/caddy -- caddy reload --config /etc/caddy/Caddyfile
|
||||
```
|
||||
|
||||
### 3. Deploy MinIO
|
||||
|
||||
```bash
|
||||
# Apply the configuration
|
||||
kubectl apply -f minio.yaml
|
||||
|
||||
# Check deployment status
|
||||
kubectl get pods -n minio
|
||||
|
||||
# Wait for pods to be ready
|
||||
kubectl wait --for=condition=ready pod -l app=minio -n minio --timeout=300s
|
||||
```
|
||||
|
||||
### 4. Verify Deployment
|
||||
|
||||
```bash
|
||||
# Check MinIO logs
|
||||
kubectl logs -n minio -l app=minio -c minio
|
||||
|
||||
# Check policy manager logs
|
||||
kubectl logs -n minio -l app=minio -c policy-manager
|
||||
|
||||
# Check ingress
|
||||
kubectl get ingress -n minio
|
||||
|
||||
# Check service
|
||||
kubectl get svc -n minio
|
||||
```
|
||||
|
||||
## Access MinIO
|
||||
|
||||
### Web Console
|
||||
- URL: `https://console.s3.yourdomain.com`
|
||||
- Username: Your configured `MINIO_ROOT_USER`
|
||||
- Password: Your configured `MINIO_ROOT_PASSWORD`
|
||||
|
||||
### S3 API Endpoint
|
||||
- URL: `https://s3.yourdomain.com`
|
||||
- Use with AWS CLI, SDKs, or any S3-compatible client
|
||||
|
||||
## Bucket Policy Management
|
||||
|
||||
### Automatic Public-Read Policy
|
||||
|
||||
The policy manager sidecar automatically:
|
||||
- Scans all buckets every 30 seconds
|
||||
- Sets new private buckets to `download` (public-read) permission
|
||||
- Allows anonymous downloads, requires auth for uploads/deletes
|
||||
|
||||
### Manual Policy Management
|
||||
|
||||
```bash
|
||||
# Get pod name
|
||||
POD=$(kubectl get pod -n minio -l app=minio -o jsonpath='{.items[0].metadata.name}')
|
||||
|
||||
# Access MinIO Client in pod
|
||||
kubectl exec -n minio $POD -c policy-manager -- mc alias set myminio http://localhost:9000 admin your-password
|
||||
|
||||
# List buckets
|
||||
kubectl exec -n minio $POD -c policy-manager -- mc ls myminio
|
||||
|
||||
# Check bucket policy
|
||||
kubectl exec -n minio $POD -c policy-manager -- mc anonymous get myminio/bucket-name
|
||||
|
||||
# Set bucket to public-read (download)
|
||||
kubectl exec -n minio $POD -c policy-manager -- mc anonymous set download myminio/bucket-name
|
||||
|
||||
# Set bucket to private
|
||||
kubectl exec -n minio $POD -c policy-manager -- mc anonymous set private myminio/bucket-name
|
||||
|
||||
# Set bucket to public (read + write)
|
||||
kubectl exec -n minio $POD -c policy-manager -- mc anonymous set public myminio/bucket-name
|
||||
```
|
||||
|
||||
## Using MinIO
|
||||
|
||||
### Create Bucket via Web Console
|
||||
|
||||
1. Access `https://console.s3.yourdomain.com`
|
||||
2. Login with credentials
|
||||
3. Click "Buckets" → "Create Bucket"
|
||||
4. Enter bucket name
|
||||
5. Wait 30 seconds for auto-policy to apply
|
||||
|
||||
### Upload Files via Web Console
|
||||
|
||||
1. Navigate to bucket
|
||||
2. Click "Upload" → "Upload File"
|
||||
3. Select files
|
||||
4. Files are immediately accessible via public URL
|
||||
|
||||
### Access Files
|
||||
|
||||
Public URL format:
|
||||
```
|
||||
https://s3.yourdomain.com/bucket-name/file-path
|
||||
```
|
||||
|
||||
Example:
|
||||
```bash
|
||||
# Upload via console, then access:
|
||||
curl https://s3.yourdomain.com/my-bucket/image.png
|
||||
```
|
||||
|
||||
### Using AWS CLI
|
||||
|
||||
```bash
|
||||
# Configure AWS CLI
|
||||
aws configure set aws_access_key_id admin
|
||||
aws configure set aws_secret_access_key your-password
|
||||
aws configure set default.region us-east-1
|
||||
|
||||
# List buckets
|
||||
aws --endpoint-url https://s3.yourdomain.com s3 ls
|
||||
|
||||
# Create bucket
|
||||
aws --endpoint-url https://s3.yourdomain.com s3 mb s3://my-bucket
|
||||
|
||||
# Upload file
|
||||
aws --endpoint-url https://s3.yourdomain.com s3 cp file.txt s3://my-bucket/
|
||||
|
||||
# Download file
|
||||
aws --endpoint-url https://s3.yourdomain.com s3 cp s3://my-bucket/file.txt ./
|
||||
|
||||
# List bucket contents
|
||||
aws --endpoint-url https://s3.yourdomain.com s3 ls s3://my-bucket/
|
||||
```
|
||||
|
||||
### Using MinIO Client (mc)
|
||||
|
||||
```bash
|
||||
# Install mc locally
|
||||
wget https://dl.min.io/client/mc/release/linux-amd64/mc
|
||||
chmod +x mc
|
||||
sudo mv mc /usr/local/bin/
|
||||
|
||||
# Configure alias
|
||||
mc alias set myminio https://s3.yourdomain.com admin your-password
|
||||
|
||||
# List buckets
|
||||
mc ls myminio
|
||||
|
||||
# Create bucket
|
||||
mc mb myminio/my-bucket
|
||||
|
||||
# Upload file
|
||||
mc cp file.txt myminio/my-bucket/
|
||||
|
||||
# Download file
|
||||
mc cp myminio/my-bucket/file.txt ./
|
||||
|
||||
# Mirror directory
|
||||
mc mirror ./local-dir myminio/my-bucket/remote-dir
|
||||
```
|
||||
|
||||
## Common Operations
|
||||
|
||||
### View Logs
|
||||
|
||||
```bash
|
||||
# MinIO server logs
|
||||
kubectl logs -n minio -l app=minio -c minio -f
|
||||
|
||||
# Policy manager logs
|
||||
kubectl logs -n minio -l app=minio -c policy-manager -f
|
||||
|
||||
# Both containers
|
||||
kubectl logs -n minio -l app=minio --all-containers -f
|
||||
```
|
||||
|
||||
### Restart MinIO
|
||||
|
||||
```bash
|
||||
# Graceful restart
|
||||
kubectl rollout restart deployment/minio -n minio
|
||||
|
||||
# Force restart (delete pod)
|
||||
kubectl delete pod -n minio -l app=minio
|
||||
```
|
||||
|
||||
### Scale Storage
|
||||
|
||||
```bash
|
||||
# Edit PVC (note: can only increase, not decrease)
|
||||
kubectl edit pvc minio-data -n minio
|
||||
|
||||
# Update storage size
|
||||
# Change: storage: 50Gi → storage: 100Gi
|
||||
```
|
||||
|
||||
### Backup Data
|
||||
|
||||
```bash
|
||||
# Get pod name
|
||||
POD=$(kubectl get pod -n minio -l app=minio -o jsonpath='{.items[0].metadata.name}')
|
||||
|
||||
# Copy data from pod
|
||||
kubectl cp minio/$POD:/data ./minio-backup -c minio
|
||||
|
||||
# Or use mc mirror
|
||||
mc mirror myminio/bucket-name ./backup/bucket-name
|
||||
```
|
||||
|
||||
### Restore Data
|
||||
|
||||
```bash
|
||||
# Copy data to pod
|
||||
kubectl cp ./minio-backup minio/$POD:/data -c minio
|
||||
|
||||
# Restart MinIO
|
||||
kubectl rollout restart deployment/minio -n minio
|
||||
|
||||
# Or use mc mirror
|
||||
mc mirror ./backup/bucket-name myminio/bucket-name
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Pod Not Starting
|
||||
|
||||
```bash
|
||||
# Check pod status
|
||||
kubectl describe pod -n minio -l app=minio
|
||||
|
||||
# Check events
|
||||
kubectl get events -n minio --sort-by='.lastTimestamp'
|
||||
|
||||
# Common issues:
|
||||
# - PVC not bound (check storage class)
|
||||
# - Image pull error (check network/registry)
|
||||
# - Resource limits (check node resources)
|
||||
```
|
||||
|
||||
### Cannot Access Web Console
|
||||
|
||||
```bash
|
||||
# Check ingress
|
||||
kubectl get ingress -n minio
|
||||
kubectl describe ingress minio-console -n minio
|
||||
|
||||
# Check service
|
||||
kubectl get svc -n minio
|
||||
|
||||
# Test from inside cluster
|
||||
kubectl run -it --rm debug --image=curlimages/curl --restart=Never -- curl -v http://minio.minio.svc.cluster.local:9001
|
||||
|
||||
# Check Caddy logs
|
||||
kubectl logs -n default -l app=caddy | grep -i s3
|
||||
|
||||
# Check Traefik logs
|
||||
kubectl logs -n kube-system -l app.kubernetes.io/name=traefik
|
||||
```
|
||||
|
||||
### SSL Certificate Issues
|
||||
|
||||
```bash
|
||||
# Check Caddy certificates
|
||||
kubectl exec -n default deployment/caddy -- caddy list-certificates
|
||||
|
||||
# Check Caddy logs for ACME
|
||||
kubectl logs -n default deployment/caddy | grep -i "s3\|acme\|certificate"
|
||||
|
||||
# Verify DNS resolution
|
||||
nslookup s3.yourdomain.com
|
||||
nslookup console.s3.yourdomain.com
|
||||
```
|
||||
|
||||
### Policy Manager Not Working
|
||||
|
||||
```bash
|
||||
# Check policy manager logs
|
||||
kubectl logs -n minio -l app=minio -c policy-manager
|
||||
|
||||
# Manually test mc commands
|
||||
POD=$(kubectl get pod -n minio -l app=minio -o jsonpath='{.items[0].metadata.name}')
|
||||
kubectl exec -n minio $POD -c policy-manager -- mc ls myminio
|
||||
|
||||
# Restart policy manager (restart pod)
|
||||
kubectl delete pod -n minio -l app=minio
|
||||
```
|
||||
|
||||
### Files Not Accessible
|
||||
|
||||
```bash
|
||||
# Check bucket policy
|
||||
kubectl exec -n minio $POD -c policy-manager -- mc anonymous get myminio/bucket-name
|
||||
|
||||
# Should show: Access permission for `myminio/bucket-name` is set to `download`
|
||||
|
||||
# If not, manually set
|
||||
kubectl exec -n minio $POD -c policy-manager -- mc anonymous set download myminio/bucket-name
|
||||
|
||||
# Test access
|
||||
curl -I https://s3.yourdomain.com/bucket-name/file.txt
|
||||
```
|
||||
|
||||
## Advanced Configuration
|
||||
|
||||
### Custom Storage Class
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: PersistentVolumeClaim
|
||||
metadata:
|
||||
name: minio-data
|
||||
namespace: minio
|
||||
spec:
|
||||
accessModes:
|
||||
- ReadWriteOnce
|
||||
resources:
|
||||
requests:
|
||||
storage: 100Gi
|
||||
storageClassName: fast-ssd # Custom storage class
|
||||
```
|
||||
|
||||
### Resource Limits
|
||||
|
||||
```yaml
|
||||
containers:
|
||||
- name: minio
|
||||
image: minio/minio:latest
|
||||
resources:
|
||||
requests:
|
||||
memory: "512Mi"
|
||||
cpu: "500m"
|
||||
limits:
|
||||
memory: "2Gi"
|
||||
cpu: "2000m"
|
||||
```
|
||||
|
||||
### Multiple Replicas (Distributed Mode)
|
||||
|
||||
For production, use distributed MinIO:
|
||||
```yaml
|
||||
# Requires 4+ nodes with persistent storage
|
||||
command:
|
||||
- /bin/sh
|
||||
- -c
|
||||
- minio server http://minio-{0...3}.minio.minio.svc.cluster.local/data --console-address ":9001"
|
||||
```
|
||||
|
||||
### Custom Bucket Policies
|
||||
|
||||
Create custom policy JSON:
|
||||
```json
|
||||
{
|
||||
"Version": "2012-10-17",
|
||||
"Statement": [
|
||||
{
|
||||
"Effect": "Allow",
|
||||
"Principal": {"AWS": ["*"]},
|
||||
"Action": ["s3:GetObject"],
|
||||
"Resource": ["arn:aws:s3:::bucket-name/*"]
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Apply via mc:
|
||||
```bash
|
||||
kubectl exec -n minio $POD -c policy-manager -- mc anonymous set-json policy.json myminio/bucket-name
|
||||
```
|
||||
|
||||
### Disable Auto-Policy Manager
|
||||
|
||||
Remove the `policy-manager` container from deployment if you want manual control.
|
||||
|
||||
## Best Practices
|
||||
|
||||
### Bucket Naming
|
||||
|
||||
- Use lowercase letters, numbers, hyphens
|
||||
- Avoid underscores, spaces, special characters
|
||||
- Keep names short and descriptive
|
||||
- Example: `user-uploads`, `static-assets`, `backups-2024`
|
||||
|
||||
### Folder Structure
|
||||
|
||||
Use prefixes (folders) to organize files:
|
||||
```
|
||||
bucket-name/
|
||||
├── user1/
|
||||
│ ├── profile.jpg
|
||||
│ └── documents/
|
||||
├── user2/
|
||||
│ └── avatar.png
|
||||
└── shared/
|
||||
└── logo.png
|
||||
```
|
||||
|
||||
### Security
|
||||
|
||||
- Change default credentials immediately
|
||||
- Use strong passwords (16+ characters)
|
||||
- Create separate access keys for applications
|
||||
- Use bucket policies to restrict access
|
||||
- Enable versioning for important buckets
|
||||
- Regular backups of critical data
|
||||
|
||||
### Performance
|
||||
|
||||
- Use CDN for frequently accessed files
|
||||
- Enable compression for text files
|
||||
- Use appropriate storage class
|
||||
- Monitor disk usage and scale proactively
|
||||
|
||||
## Quick Reference Commands
|
||||
|
||||
```bash
|
||||
# Deploy MinIO
|
||||
kubectl apply -f minio.yaml
|
||||
|
||||
# Check status
|
||||
kubectl get pods -n minio
|
||||
kubectl get svc -n minio
|
||||
kubectl get ingress -n minio
|
||||
|
||||
# View logs
|
||||
kubectl logs -n minio -l app=minio -c minio -f
|
||||
kubectl logs -n minio -l app=minio -c policy-manager -f
|
||||
|
||||
# Restart MinIO
|
||||
kubectl rollout restart deployment/minio -n minio
|
||||
|
||||
# Get pod name
|
||||
POD=$(kubectl get pod -n minio -l app=minio -o jsonpath='{.items[0].metadata.name}')
|
||||
|
||||
# Access mc client
|
||||
kubectl exec -n minio $POD -c policy-manager -- mc ls myminio
|
||||
|
||||
# Check bucket policy
|
||||
kubectl exec -n minio $POD -c policy-manager -- mc anonymous get myminio/bucket-name
|
||||
|
||||
# Set bucket policy
|
||||
kubectl exec -n minio $POD -c policy-manager -- mc anonymous set download myminio/bucket-name
|
||||
|
||||
# Delete deployment
|
||||
kubectl delete -f minio.yaml
|
||||
```
|
||||
|
||||
## Integration Examples
|
||||
|
||||
### Node.js (AWS SDK)
|
||||
|
||||
```javascript
|
||||
const AWS = require('aws-sdk');
|
||||
|
||||
const s3 = new AWS.S3({
|
||||
endpoint: 'https://s3.yourdomain.com',
|
||||
accessKeyId: 'admin',
|
||||
secretAccessKey: 'your-password',
|
||||
s3ForcePathStyle: true,
|
||||
signatureVersion: 'v4'
|
||||
});
|
||||
|
||||
// Upload file
|
||||
s3.putObject({
|
||||
Bucket: 'my-bucket',
|
||||
Key: 'file.txt',
|
||||
Body: 'Hello World'
|
||||
}, (err, data) => {
|
||||
if (err) console.error(err);
|
||||
else console.log('Uploaded:', data);
|
||||
});
|
||||
|
||||
// Download file
|
||||
s3.getObject({
|
||||
Bucket: 'my-bucket',
|
||||
Key: 'file.txt'
|
||||
}, (err, data) => {
|
||||
if (err) console.error(err);
|
||||
else console.log('Content:', data.Body.toString());
|
||||
});
|
||||
```
|
||||
|
||||
### Python (boto3)
|
||||
|
||||
```python
|
||||
import boto3
|
||||
|
||||
s3 = boto3.client('s3',
|
||||
endpoint_url='https://s3.yourdomain.com',
|
||||
aws_access_key_id='admin',
|
||||
aws_secret_access_key='your-password'
|
||||
)
|
||||
|
||||
# Upload file
|
||||
s3.upload_file('local-file.txt', 'my-bucket', 'remote-file.txt')
|
||||
|
||||
# Download file
|
||||
s3.download_file('my-bucket', 'remote-file.txt', 'downloaded.txt')
|
||||
|
||||
# List objects
|
||||
response = s3.list_objects_v2(Bucket='my-bucket')
|
||||
for obj in response.get('Contents', []):
|
||||
print(obj['Key'])
|
||||
```
|
||||
|
||||
### Go (minio-go)
|
||||
|
||||
```go
|
||||
package main
|
||||
|
||||
import (
|
||||
"github.com/minio/minio-go/v7"
|
||||
"github.com/minio/minio-go/v7/pkg/credentials"
|
||||
)
|
||||
|
||||
func main() {
|
||||
client, _ := minio.New("s3.yourdomain.com", &minio.Options{
|
||||
Creds: credentials.NewStaticV4("admin", "your-password", ""),
|
||||
Secure: true,
|
||||
})
|
||||
|
||||
// Upload file
|
||||
client.FPutObject(ctx, "my-bucket", "file.txt", "local-file.txt", minio.PutObjectOptions{})
|
||||
|
||||
// Download file
|
||||
client.FGetObject(ctx, "my-bucket", "file.txt", "downloaded.txt", minio.GetObjectOptions{})
|
||||
}
|
||||
```
|
||||
|
||||
## Notes
|
||||
|
||||
- MinIO is fully S3-compatible
|
||||
- Automatic SSL via Caddy
|
||||
- Auto-policy sets buckets to public-read by default
|
||||
- Policy manager runs every 30 seconds
|
||||
- Persistent storage required for data retention
|
||||
- Single replica suitable for development/small deployments
|
||||
- Use distributed mode for production high-availability
|
||||
- Regular backups recommended for critical data
|
||||
Reference in New Issue
Block a user