docker部署python项目、爬虫注意事项【持续更新】

爱被打了一巴掌 2022-11-06 00:56 431阅读 0赞

**python项目部署在docker，记录一下出现的问题**

## pip下载安装包慢、超时、失败的问题 ##

[解决办法][Link 1]

## 时区不正确的问题 ##

**异常提示**  
Timezone offset does not match system offset: 0 != 28800. Please, check your config files.  
**解决办法**

1.  修改代码中BlockingScheduler、BackgroundScheduler指定时区为timezone=“Asia/Shanghai”

# 示例
        block_scheduler = BlockingScheduler(timezone="Asia/Shanghai")

1.  程序启动时映射/etc/localtime:/etc/localtime

# 示例
    docker run -d -v /etc/localtime:/etc/localtime -p 8080:8080 crawl:1.0

## izihawa-commons安装失败 ##

`ERROR: Could not find a version that satisfies the requirement izihawa-commons>=0.0.10 (from clickhouse) (from versions: none)`

`ERROR: No matching distribution found for izihawa-commons>=0.0.10`

### 解决方案： ###

*  使用豆瓣镜像源  
    `修改Dockerfile在pip -r 前加上如下指令`

RUN pip install -i http://pypi.douban.com/simple/ --trusted-host pypi.douban.com izihawa-commons

## 使用flask启动时报错 ##

`ValueError: Timezone offset does not match system offset: 0 != 28800. Please, check your config files`

*  ***原因分析：在启动时挂载了/etc/localtime  
    使用`date -R`查看确实是+0800时区，代码层面虽然指定了timezone，但是没有生效***
 *  修改前代码

app = Flask(__name__)
    scheduler = APScheduler(BackgroundScheduler(timezone="Asia/Shanghai"))

*  查看源码：

timezone = self.app.config.get('SCHEDULER_TIMEZONE')
            if timezone:
                options['timezone'] = timezone
    
            self._scheduler.configure(**options)

*  源码位置截图  
    ![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQxMDcwMzkz_size_16_color_FFFFFF_t_70]
 *  修改代码：在配置中加入`SCHEDULER_TIMEZONE = "Asia/Shanghai"`

class Config:
        JOBS = [
            { 
                "id": "nk_job_detail_1",
                "func": "service:run",
                "args": None,
                "trigger": "cron",
                "day": "*/1",
                "hour": "1"
            }
        ]
    
        SCHEDULER_API_ENABLED = True
        # 加入配置
        SCHEDULER_TIMEZONE = "Asia/Shanghai"

# 项目中有python包，docker启动报错 #

`ModuleNotFoundError: No module named xxx`  
例如下目录结构  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQxMDcwMzkz_size_16_color_FFFFFF_t_70 1]  
启动py文件中已经加入了系统环境变量，而且`python -m site`打印可以看出当前项目确实已经加入到path路径下

*  解决办法：  
    修改dockerfile 文件

FROM python:3.9
    
    RUN mkdir /app
    # 重点加add指令
    ADD . /app
    WORKDIR /app
    RUN ["python","-m","pip","install","--upgrade","pip"]
    
    RUN ["python","-m","site"]
    COPY requirements.txt requirements.txt
    RUN pip install -i https://mirrors.aliyun.com/pypi/simple/ --trusted-host mirrors.aliyun.com --default-timeout=20 --no-cache-dir -r requirements.txt
    #RUN pip install -r requirements.txt
    
    COPY ./* /app/
    
    VOLUME ["/data/mo"]
    
    CMD [ "python", "/app/app.py" ]

修改点`ADD . /app`

[Link 1]: https://blog.csdn.net/qq_41070393/article/details/113881271
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQxMDcwMzkz_size_16_color_FFFFFF_t_70]: /images/20221023/8ab5684a95654befa179999b90bebb2e.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQxMDcwMzkz_size_16_color_FFFFFF_t_70 1]: /images/20221023/c21c961115f84983a71c99090a5ed827.png