首次提交：初始化项目
This commit is contained in:
fei
752
K3S-CADDY-TRAEFIK-SETUP.md
Normal file
752
K3S-CADDY-TRAEFIK-SETUP.md
Normal file
@@ -0,0 +1,752 @@
|
||||
# K3s + Caddy + Traefik 部署文档
|
||||
|
||||
## 架构概述
|
||||
|
||||
```
|
||||
互联网
|
||||
↓
|
||||
Caddy (宿主机 80/443) - SSL 终止 + 自动证书
|
||||
↓
|
||||
Traefik (ClusterIP) - Ingress 路由
|
||||
↓
|
||||
应用 Services
|
||||
↓
|
||||
应用 Pods
|
||||
```
|
||||
|
||||
**职责分工：**
|
||||
- **Caddy**: SSL/TLS 终止，自动获取和续期 Let's Encrypt 证书
|
||||
- **Traefik**: Kubernetes Ingress 控制器，根据 Ingress 规则路由流量
|
||||
- **应用**: 通过 Service 和 Ingress 暴露服务
|
||||
|
||||
---
|
||||
|
||||
## 前置要求
|
||||
|
||||
### 系统要求
|
||||
- Linux 系统（Ubuntu/Debian/CentOS 等）
|
||||
- 至少 2GB RAM
|
||||
- sudo 权限
|
||||
- 域名已解析到服务器 IP
|
||||
|
||||
### 端口要求
|
||||
- 80 (HTTP)
|
||||
- 443 (HTTPS)
|
||||
- 6443 (K3s API，可选)
|
||||
|
||||
---
|
||||
|
||||
## 安装步骤
|
||||
|
||||
### 1. 安装 K3s
|
||||
|
||||
```bash
|
||||
# 下载并安装 k3s
|
||||
curl -sfL https://get.k3s.io -o /tmp/k3s-install.sh
|
||||
sudo sh /tmp/k3s-install.sh
|
||||
|
||||
# 验证安装
|
||||
sudo kubectl get nodes
|
||||
```
|
||||
|
||||
**预期输出：**
|
||||
```
|
||||
NAME STATUS ROLES AGE VERSION
|
||||
host Ready control-plane 10s v1.34.3+k3s1
|
||||
```
|
||||
|
||||
### 2. 配置 Traefik 为 ClusterIP
|
||||
|
||||
K3s 默认安装 Traefik 为 LoadBalancer 类型，需要改为 ClusterIP 以便 Caddy 使用宿主机端口。
|
||||
|
||||
```bash
|
||||
# 将 Traefik Service 改为 ClusterIP
|
||||
sudo kubectl patch svc traefik -n kube-system -p '{"spec":{"type":"ClusterIP"}}'
|
||||
|
||||
# 验证
|
||||
sudo kubectl get svc -n kube-system traefik
|
||||
```
|
||||
|
||||
**预期输出：**
|
||||
```
|
||||
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S)
|
||||
traefik ClusterIP 10.43.x.x <none> 80/TCP,443/TCP
|
||||
```
|
||||
|
||||
### 3. 部署 Caddy
|
||||
|
||||
创建 Caddy 配置文件 `caddy-deployment.yaml`：
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: caddy-config
|
||||
namespace: default
|
||||
data:
|
||||
Caddyfile: |
|
||||
{
|
||||
email your-email@example.com
|
||||
}
|
||||
|
||||
# 添加你的域名配置
|
||||
app1.example.com {
|
||||
reverse_proxy traefik.kube-system.svc.cluster.local:80
|
||||
}
|
||||
|
||||
app2.example.com {
|
||||
reverse_proxy traefik.kube-system.svc.cluster.local:80
|
||||
}
|
||||
---
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: caddy
|
||||
namespace: default
|
||||
spec:
|
||||
replicas: 1
|
||||
selector:
|
||||
matchLabels:
|
||||
app: caddy
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: caddy
|
||||
spec:
|
||||
containers:
|
||||
- name: caddy
|
||||
image: caddy:latest
|
||||
ports:
|
||||
- containerPort: 80
|
||||
hostPort: 80
|
||||
- containerPort: 443
|
||||
hostPort: 443
|
||||
volumeMounts:
|
||||
- name: config
|
||||
mountPath: /etc/caddy
|
||||
- name: data
|
||||
mountPath: /data
|
||||
volumes:
|
||||
- name: config
|
||||
configMap:
|
||||
name: caddy-config
|
||||
- name: data
|
||||
emptyDir: {}
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: caddy
|
||||
namespace: default
|
||||
spec:
|
||||
selector:
|
||||
app: caddy
|
||||
ports:
|
||||
- name: http
|
||||
port: 80
|
||||
targetPort: 80
|
||||
- name: https
|
||||
port: 443
|
||||
targetPort: 443
|
||||
type: ClusterIP
|
||||
```
|
||||
|
||||
**部署 Caddy：**
|
||||
|
||||
```bash
|
||||
# 应用配置
|
||||
sudo kubectl apply -f caddy-deployment.yaml
|
||||
|
||||
# 验证 Caddy 运行状态
|
||||
sudo kubectl get pods -n default -l app=caddy
|
||||
```
|
||||
|
||||
### 4. 部署测试应用
|
||||
|
||||
创建测试应用 `test-app.yaml`：
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: ConfigMap
|
||||
metadata:
|
||||
name: test-html
|
||||
namespace: default
|
||||
data:
|
||||
index.html: |
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head>
|
||||
<title>K3s Test</title>
|
||||
<style>
|
||||
body {
|
||||
font-family: Arial, sans-serif;
|
||||
display: flex;
|
||||
justify-content: center;
|
||||
align-items: center;
|
||||
height: 100vh;
|
||||
margin: 0;
|
||||
background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
|
||||
color: white;
|
||||
}
|
||||
.container {
|
||||
text-align: center;
|
||||
padding: 40px;
|
||||
background: rgba(255, 255, 255, 0.1);
|
||||
border-radius: 10px;
|
||||
}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<div class="container">
|
||||
<h1>K3s + Caddy + Traefik</h1>
|
||||
<p>SSL Termination Working!</p>
|
||||
</div>
|
||||
</body>
|
||||
</html>
|
||||
---
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: test-app
|
||||
namespace: default
|
||||
spec:
|
||||
replicas: 1
|
||||
selector:
|
||||
matchLabels:
|
||||
app: test-app
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: test-app
|
||||
spec:
|
||||
containers:
|
||||
- name: nginx
|
||||
image: nginx:alpine
|
||||
ports:
|
||||
- containerPort: 80
|
||||
volumeMounts:
|
||||
- name: html
|
||||
mountPath: /usr/share/nginx/html
|
||||
volumes:
|
||||
- name: html
|
||||
configMap:
|
||||
name: test-html
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: test-app
|
||||
namespace: default
|
||||
spec:
|
||||
selector:
|
||||
app: test-app
|
||||
ports:
|
||||
- port: 80
|
||||
targetPort: 80
|
||||
type: ClusterIP
|
||||
```
|
||||
|
||||
**部署应用：**
|
||||
|
||||
```bash
|
||||
sudo kubectl apply -f test-app.yaml
|
||||
```
|
||||
|
||||
### 5. 创建 Ingress 规则
|
||||
|
||||
创建 Ingress 配置 `test-ingress.yaml`：
|
||||
|
||||
```yaml
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: Ingress
|
||||
metadata:
|
||||
name: test-app-ingress
|
||||
namespace: default
|
||||
annotations:
|
||||
traefik.ingress.kubernetes.io/router.entrypoints: web
|
||||
spec:
|
||||
rules:
|
||||
- host: app1.example.com
|
||||
http:
|
||||
paths:
|
||||
- path: /
|
||||
pathType: Prefix
|
||||
backend:
|
||||
service:
|
||||
name: test-app
|
||||
port:
|
||||
number: 80
|
||||
```
|
||||
|
||||
**应用 Ingress：**
|
||||
|
||||
```bash
|
||||
sudo kubectl apply -f test-ingress.yaml
|
||||
|
||||
# 查看 Ingress 状态
|
||||
sudo kubectl get ingress -n default
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 验证部署
|
||||
|
||||
### 1. 检查所有组件状态
|
||||
|
||||
```bash
|
||||
# 查看所有 Pods
|
||||
sudo kubectl get pods -A
|
||||
|
||||
# 查看 Services
|
||||
sudo kubectl get svc -A
|
||||
|
||||
# 查看 Ingress
|
||||
sudo kubectl get ingress -A
|
||||
```
|
||||
|
||||
### 2. 测试 HTTP 访问
|
||||
|
||||
```bash
|
||||
# 测试 HTTP（会自动重定向到 HTTPS）
|
||||
curl -I http://app1.example.com
|
||||
```
|
||||
|
||||
**预期输出：**
|
||||
```
|
||||
HTTP/1.1 308 Permanent Redirect
|
||||
Location: https://app1.example.com/
|
||||
```
|
||||
|
||||
### 3. 测试 HTTPS 访问
|
||||
|
||||
```bash
|
||||
# 测试 HTTPS
|
||||
curl https://app1.example.com
|
||||
```
|
||||
|
||||
**预期输出：** HTML 页面内容
|
||||
|
||||
### 4. 检查 SSL 证书
|
||||
|
||||
```bash
|
||||
# 查看证书信息
|
||||
echo | openssl s_client -connect app1.example.com:443 -servername app1.example.com 2>/dev/null | openssl x509 -noout -dates
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 添加新应用
|
||||
|
||||
### 步骤 1: 更新 Caddy 配置
|
||||
|
||||
```bash
|
||||
# 编辑 Caddy ConfigMap
|
||||
sudo kubectl edit configmap caddy-config -n default
|
||||
```
|
||||
|
||||
在 Caddyfile 中添加新域名：
|
||||
|
||||
```
|
||||
newapp.example.com {
|
||||
reverse_proxy traefik.kube-system.svc.cluster.local:80
|
||||
}
|
||||
```
|
||||
|
||||
### 步骤 2: 重启 Caddy
|
||||
|
||||
```bash
|
||||
sudo kubectl rollout restart deployment caddy -n default
|
||||
```
|
||||
|
||||
### 步骤 3: 部署应用
|
||||
|
||||
```bash
|
||||
# 创建应用的 Deployment 和 Service
|
||||
sudo kubectl apply -f your-app.yaml
|
||||
```
|
||||
|
||||
### 步骤 4: 创建 Ingress
|
||||
|
||||
```yaml
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: Ingress
|
||||
metadata:
|
||||
name: newapp-ingress
|
||||
namespace: default
|
||||
spec:
|
||||
rules:
|
||||
- host: newapp.example.com
|
||||
http:
|
||||
paths:
|
||||
- path: /
|
||||
pathType: Prefix
|
||||
backend:
|
||||
service:
|
||||
name: your-service-name
|
||||
port:
|
||||
number: 80
|
||||
```
|
||||
|
||||
```bash
|
||||
sudo kubectl apply -f newapp-ingress.yaml
|
||||
```
|
||||
|
||||
### 步骤 5: 等待证书获取
|
||||
|
||||
```bash
|
||||
# 查看 Caddy 日志，确认证书获取成功
|
||||
sudo kubectl logs -n default -l app=caddy --tail=50
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 常用管理命令
|
||||
|
||||
### 查看日志
|
||||
|
||||
```bash
|
||||
# Caddy 日志
|
||||
sudo kubectl logs -n default -l app=caddy -f
|
||||
|
||||
# Traefik 日志
|
||||
sudo kubectl logs -n kube-system -l app.kubernetes.io/name=traefik -f
|
||||
|
||||
# 应用日志
|
||||
sudo kubectl logs -n default -l app=your-app -f
|
||||
```
|
||||
|
||||
### 重启服务
|
||||
|
||||
```bash
|
||||
# 重启 Caddy
|
||||
sudo kubectl rollout restart deployment caddy -n default
|
||||
|
||||
# 重启应用
|
||||
sudo kubectl rollout restart deployment your-app -n default
|
||||
```
|
||||
|
||||
### 查看资源状态
|
||||
|
||||
```bash
|
||||
# 查看所有资源
|
||||
sudo kubectl get all,ingress -n default
|
||||
|
||||
# 查看 ConfigMap
|
||||
sudo kubectl get configmap -n default
|
||||
|
||||
# 查看详细信息
|
||||
sudo kubectl describe pod <pod-name> -n default
|
||||
```
|
||||
|
||||
### 更新配置
|
||||
|
||||
```bash
|
||||
# 编辑 Caddy 配置
|
||||
sudo kubectl edit configmap caddy-config -n default
|
||||
|
||||
# 编辑 Ingress
|
||||
sudo kubectl edit ingress <ingress-name> -n default
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 故障排查
|
||||
|
||||
### 问题 1: Caddy Pod 无法启动（Pending 状态）
|
||||
|
||||
**症状：**
|
||||
```bash
|
||||
sudo kubectl get pods -n default
|
||||
# caddy pod 显示 Pending
|
||||
```
|
||||
|
||||
**原因：** 端口 80/443 被占用
|
||||
|
||||
**解决方案：**
|
||||
```bash
|
||||
# 检查端口占用
|
||||
sudo ss -tlnp | grep -E ':(80|443)\s'
|
||||
|
||||
# 如果 Traefik 占用，确保已改为 ClusterIP
|
||||
sudo kubectl get svc -n kube-system traefik
|
||||
|
||||
# 如果是 LoadBalancer，执行：
|
||||
sudo kubectl patch svc traefik -n kube-system -p '{"spec":{"type":"ClusterIP"}}'
|
||||
```
|
||||
|
||||
### 问题 2: SSL 证书获取失败
|
||||
|
||||
**症状：** 访问 HTTPS 时证书错误
|
||||
|
||||
**检查步骤：**
|
||||
|
||||
```bash
|
||||
# 1. 查看 Caddy 日志
|
||||
sudo kubectl logs -n default -l app=caddy --tail=100
|
||||
|
||||
# 2. 确认域名解析正确
|
||||
nslookup your-domain.com
|
||||
|
||||
# 3. 确认端口 80 可访问（Let's Encrypt 需要）
|
||||
curl -I http://your-domain.com
|
||||
```
|
||||
|
||||
**常见原因：**
|
||||
- 域名未正确解析到服务器 IP
|
||||
- 防火墙阻止 80/443 端口
|
||||
- Let's Encrypt 速率限制（每周最多 5 次失败）
|
||||
|
||||
### 问题 3: 访问域名返回 404
|
||||
|
||||
**症状：** HTTPS 可访问，但返回 404
|
||||
|
||||
**检查步骤：**
|
||||
|
||||
```bash
|
||||
# 1. 检查 Ingress 配置
|
||||
sudo kubectl get ingress -n default
|
||||
sudo kubectl describe ingress <ingress-name> -n default
|
||||
|
||||
# 2. 检查 Service
|
||||
sudo kubectl get svc -n default
|
||||
|
||||
# 3. 检查 Pod 状态
|
||||
sudo kubectl get pods -n default
|
||||
|
||||
# 4. 测试 Traefik 路由
|
||||
sudo kubectl exec -n default deployment/caddy -- \
|
||||
curl -H "Host: your-domain.com" http://traefik.kube-system.svc.cluster.local:80
|
||||
```
|
||||
|
||||
**解决方案：**
|
||||
- 确认 Ingress 的 host 和 service name 正确
|
||||
- 确认 Service selector 匹配 Pod labels
|
||||
- 确认 Pod 正在运行
|
||||
|
||||
### 问题 4: 证书自动续期失败
|
||||
|
||||
**检查：**
|
||||
|
||||
```bash
|
||||
# 查看证书过期时间
|
||||
echo | openssl s_client -connect your-domain.com:443 -servername your-domain.com 2>/dev/null | openssl x509 -noout -dates
|
||||
|
||||
# 查看 Caddy 日志中的续期信息
|
||||
sudo kubectl logs -n default -l app=caddy | grep -i renew
|
||||
```
|
||||
|
||||
**解决方案：**
|
||||
- Caddy 会自动续期，通常无需手动干预
|
||||
- 如果失败，检查域名解析和端口访问
|
||||
- 必要时重启 Caddy: `sudo kubectl rollout restart deployment caddy -n default`
|
||||
|
||||
---
|
||||
|
||||
## 高级配置
|
||||
|
||||
### 使用持久化存储保存证书
|
||||
|
||||
默认配置使用 `emptyDir`，Pod 重启后证书会丢失。生产环境建议使用持久化存储：
|
||||
|
||||
```yaml
|
||||
apiVersion: v1
|
||||
kind: PersistentVolumeClaim
|
||||
metadata:
|
||||
name: caddy-data
|
||||
namespace: default
|
||||
spec:
|
||||
accessModes:
|
||||
- ReadWriteOnce
|
||||
resources:
|
||||
requests:
|
||||
storage: 1Gi
|
||||
---
|
||||
# 在 Deployment 中使用 PVC
|
||||
spec:
|
||||
template:
|
||||
spec:
|
||||
volumes:
|
||||
- name: data
|
||||
persistentVolumeClaim:
|
||||
claimName: caddy-data
|
||||
```
|
||||
|
||||
### 配置 Caddy 日志级别
|
||||
|
||||
在 Caddyfile 中添加：
|
||||
|
||||
```
|
||||
{
|
||||
email your-email@example.com
|
||||
log {
|
||||
level DEBUG
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 配置自定义 TLS 选项
|
||||
|
||||
```
|
||||
your-domain.com {
|
||||
tls {
|
||||
protocols tls1.2 tls1.3
|
||||
ciphers TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384 TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
|
||||
}
|
||||
reverse_proxy traefik.kube-system.svc.cluster.local:80
|
||||
}
|
||||
```
|
||||
|
||||
### 配置 Traefik 中间件
|
||||
|
||||
为 Ingress 添加中间件（如重定向、认证等）：
|
||||
|
||||
```yaml
|
||||
apiVersion: traefik.containo.us/v1alpha1
|
||||
kind: Middleware
|
||||
metadata:
|
||||
name: redirect-https
|
||||
namespace: default
|
||||
spec:
|
||||
redirectScheme:
|
||||
scheme: https
|
||||
permanent: true
|
||||
---
|
||||
apiVersion: networking.k8s.io/v1
|
||||
kind: Ingress
|
||||
metadata:
|
||||
name: app-ingress
|
||||
annotations:
|
||||
traefik.ingress.kubernetes.io/router.middlewares: default-redirect-https@kubernetescrd
|
||||
spec:
|
||||
rules:
|
||||
- host: app.example.com
|
||||
http:
|
||||
paths:
|
||||
- path: /
|
||||
pathType: Prefix
|
||||
backend:
|
||||
service:
|
||||
name: app-service
|
||||
port:
|
||||
number: 80
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 安全建议
|
||||
|
||||
1. **限制 K3s API 访问**
|
||||
```bash
|
||||
# 仅允许本地访问
|
||||
sudo kubectl config set-cluster default --server=https://127.0.0.1:6443
|
||||
```
|
||||
|
||||
2. **配置防火墙**
|
||||
```bash
|
||||
# 仅开放必要端口
|
||||
sudo ufw allow 80/tcp
|
||||
sudo ufw allow 443/tcp
|
||||
sudo ufw allow 22/tcp
|
||||
sudo ufw enable
|
||||
```
|
||||
|
||||
3. **定期更新**
|
||||
```bash
|
||||
# 更新 K3s
|
||||
curl -sfL https://get.k3s.io | sudo sh -
|
||||
|
||||
# 更新 Caddy
|
||||
sudo kubectl set image deployment/caddy caddy=caddy:latest -n default
|
||||
```
|
||||
|
||||
4. **备份证书数据**
|
||||
```bash
|
||||
# 如果使用 PVC，定期备份
|
||||
sudo kubectl exec -n default deployment/caddy -- tar czf - /data > caddy-backup.tar.gz
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 性能优化
|
||||
|
||||
### 1. 增加 Caddy 副本数（可选）
|
||||
|
||||
```bash
|
||||
sudo kubectl scale deployment caddy --replicas=2 -n default
|
||||
```
|
||||
|
||||
**注意：** 使用 hostPort 时，每个节点只能运行一个副本。
|
||||
|
||||
### 2. 配置资源限制
|
||||
|
||||
```yaml
|
||||
spec:
|
||||
template:
|
||||
spec:
|
||||
containers:
|
||||
- name: caddy
|
||||
resources:
|
||||
requests:
|
||||
memory: "128Mi"
|
||||
cpu: "100m"
|
||||
limits:
|
||||
memory: "256Mi"
|
||||
cpu: "500m"
|
||||
```
|
||||
|
||||
### 3. 启用 HTTP/3
|
||||
|
||||
Caddy 默认启用 HTTP/3，确保 UDP 443 端口开放：
|
||||
|
||||
```bash
|
||||
sudo ufw allow 443/udp
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 卸载
|
||||
|
||||
```bash
|
||||
# 删除应用
|
||||
sudo kubectl delete -f test-app.yaml
|
||||
sudo kubectl delete -f test-ingress.yaml
|
||||
|
||||
# 删除 Caddy
|
||||
sudo kubectl delete -f caddy-deployment.yaml
|
||||
|
||||
# 卸载 K3s
|
||||
sudo /usr/local/bin/k3s-uninstall.sh
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 参考资源
|
||||
|
||||
- [K3s 官方文档](https://docs.k3s.io/)
|
||||
- [Caddy 官方文档](https://caddyserver.com/docs/)
|
||||
- [Traefik 官方文档](https://doc.traefik.io/traefik/)
|
||||
- [Kubernetes Ingress 文档](https://kubernetes.io/docs/concepts/services-networking/ingress/)
|
||||
|
||||
---
|
||||
|
||||
## 总结
|
||||
|
||||
这个架构的优势：
|
||||
|
||||
✅ **自动化 SSL 管理** - Caddy 自动获取和续期证书
|
||||
✅ **灵活的路由** - Traefik 提供强大的 Ingress 功能
|
||||
✅ **简单部署** - 一键部署，无需手动配置证书
|
||||
✅ **高可用** - 支持多副本和负载均衡
|
||||
✅ **易于维护** - 声明式配置，版本控制友好
|
||||
|
||||
适用场景：
|
||||
- 个人项目和小型应用
|
||||
- 开发和测试环境
|
||||
- 需要快速部署的场景
|
||||
- 多域名多应用托管
|
||||
|
||||
---
|
||||
|
||||
**文档版本：** 1.0
|
||||
**最后更新：** 2026-01-29
|
||||
**适用于：** K3s v1.34+, Caddy v2.8+, Traefik v3.5+
|
||||
Reference in New Issue
Block a user