168 lines
2.6 KiB
Markdown
168 lines
2.6 KiB
Markdown
|
# 部署指南
|
|||
|
|
|
|||
|
|
## 快速部署
|
|||
|
|
|
|||
|
|
### Linux/Mac
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
# 添加执行权限
|
|||
|
|
chmod +x deploy.sh
|
|||
|
|
|
|||
|
|
# 构建镜像
|
|||
|
|
./deploy.sh build
|
|||
|
|
|
|||
|
|
# 启动服务
|
|||
|
|
./deploy.sh up
|
|||
|
|
|
|||
|
|
# 查看日志
|
|||
|
|
./deploy.sh logs
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### Windows
|
|||
|
|
|
|||
|
|
```cmd
|
|||
|
|
REM 构建镜像
|
|||
|
|
deploy.bat build
|
|||
|
|
|
|||
|
|
REM 启动服务
|
|||
|
|
deploy.bat up
|
|||
|
|
|
|||
|
|
REM 查看日志
|
|||
|
|
deploy.bat logs
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
## 部署命令
|
|||
|
|
|
|||
|
|
| 命令 | 说明 |
|
|||
|
|
|------|------|
|
|||
|
|
| `build` | 构建Docker镜像 |
|
|||
|
|
| `up` | 启动所有服务（Kafka + App） |
|
|||
|
|
| `down` | 停止所有服务 |
|
|||
|
|
| `restart` | 重启应用服务 |
|
|||
|
|
| `logs` | 查看应用日志 |
|
|||
|
|
| `status` | 查看服务状态 |
|
|||
|
|
| `clean` | 清理无用镜像 |
|
|||
|
|
|
|||
|
|
## 手动部署
|
|||
|
|
|
|||
|
|
### 1. 构建镜像
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
docker build -t job-crawler:latest .
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### 2. 推送到私有仓库（可选）
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
# 打标签
|
|||
|
|
docker tag job-crawler:latest your-registry.com/job-crawler:latest
|
|||
|
|
|
|||
|
|
# 推送
|
|||
|
|
docker push your-registry.com/job-crawler:latest
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### 3. 启动服务
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
# 使用 docker-compose（推荐）
|
|||
|
|
docker-compose up -d
|
|||
|
|
|
|||
|
|
# 或单独运行容器
|
|||
|
|
docker run -d \
|
|||
|
|
--name job-crawler \
|
|||
|
|
-p 8000:8000 \
|
|||
|
|
-v $(pwd)/config:/app/config:ro \
|
|||
|
|
-v job-data:/app/data \
|
|||
|
|
-e CONFIG_PATH=/app/config/config.yml \
|
|||
|
|
job-crawler:latest
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
## 配置说明
|
|||
|
|
|
|||
|
|
### 1. 复制配置模板
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
cp config/config.yml.docker config/config.yml
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### 2. 编辑配置文件
|
|||
|
|
|
|||
|
|
```yaml
|
|||
|
|
api:
|
|||
|
|
username: "your_username" # 修改为你的账号
|
|||
|
|
password: "your_password" # 修改为你的密码
|
|||
|
|
tasks:
|
|||
|
|
- id: "your-task-id" # 修改为你的任务ID
|
|||
|
|
name: "任务名称"
|
|||
|
|
enabled: true
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
## 生产环境建议
|
|||
|
|
|
|||
|
|
### 1. 使用外部Kafka
|
|||
|
|
|
|||
|
|
修改 `config/config.yml`:
|
|||
|
|
|
|||
|
|
```yaml
|
|||
|
|
kafka:
|
|||
|
|
bootstrap_servers: your-kafka-host:9092
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
修改 `docker-compose.yml`，移除 zookeeper 和 kafka 服务。
|
|||
|
|
|
|||
|
|
### 2. 数据持久化
|
|||
|
|
|
|||
|
|
确保挂载数据卷：
|
|||
|
|
|
|||
|
|
```yaml
|
|||
|
|
volumes:
|
|||
|
|
- ./data:/app/data # 本地目录
|
|||
|
|
# 或使用命名卷
|
|||
|
|
- app_data:/app/data
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### 3. 日志收集
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
# 查看日志
|
|||
|
|
docker-compose logs -f app
|
|||
|
|
|
|||
|
|
# 导出日志
|
|||
|
|
docker-compose logs app > app.log
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### 4. 健康检查
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
# 检查服务健康
|
|||
|
|
curl http://localhost:8000/health
|
|||
|
|
|
|||
|
|
# 检查采集状态
|
|||
|
|
curl http://localhost:8000/status
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
## 常见问题
|
|||
|
|
|
|||
|
|
### Q: Kafka连接失败
|
|||
|
|
|
|||
|
|
检查 `config.yml` 中的 `kafka.bootstrap_servers`:
|
|||
|
|
- Docker内部: `kafka:29092`
|
|||
|
|
- 本地开发: `localhost:9092`
|
|||
|
|
|
|||
|
|
### Q: 配置文件未生效
|
|||
|
|
|
|||
|
|
确保配置文件正确挂载：
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
docker exec job-crawler cat /app/config/config.yml
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### Q: 数据丢失
|
|||
|
|
|
|||
|
|
检查数据卷是否正确挂载：
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
docker volume ls
|
|||
|
|
docker volume inspect job_crawler_app_data
|
|||
|
|
```
|