{"status":false,"msg":"您操作太频繁,请稍后再访问","clientIp":"113.94.81.141"}

古城微笑少年丶 2024-04-17 16:58 16阅读 0赞

### 在爬取拉勾网的时候报错\{"status":false,"msg":"您操作太频繁,请稍后再访问","clientIp":"113.94.81.141"\} ###

需要获取搜索结果那一页的cookie 以及header里面需要添加Accept、Referer、User-Agent

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpOTM2NzU_size_16_color_FFFFFF_t_70][]

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpOTM2NzU_size_16_color_FFFFFF_t_70 1][]

import requests 
    
    url = 'https://www.lagou.com/jobs/positionAjax.json?city=%E5%B9%BF%E5%B7%9E&needAddtionalResult=false'
    
    payload={
        'first': 'true',
        'pn': '1',
        'kd': '自然语言处理'   
    }
    
    header = {
        'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.86 Safari/537.36',
        'Referer': 'https://www.lagou.com/jobs/list_%E8%87%AA%E7%84%B6%E8%AF%AD%E8%A8%80%E5%A4%84%E7%90%86?px=default&city=%E5%B9%BF%E5%B7%9E',
        'Accept': 'application/json, text/javascript, */*; q=0.01'
    }
    
    urls ='https://www.lagou.com/jobs/list_%E8%87%AA%E7%84%B6%E8%AF%AD%E8%A8%80%E5%A4%84%E7%90%86?city=%E5%B9%BF%E5%B7%9E&cl=false&fromSearch=true&labelWords=&suginput='
    s = requests.Session()
    # 获取搜索页的cookies
    s.get(urls, headers=header, timeout=3)
    # 为此次获取的cookies
    cookie = s.cookies
    # 获取此次文本
    response = s.post(url, data=payload, headers=header, cookies=cookie, timeout=5).text
    print(response)

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpOTM2NzU_size_16_color_FFFFFF_t_70 2][]

[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpOTM2NzU_size_16_color_FFFFFF_t_70]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/17/1554935216f4466bbb41f830c0a481a1.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpOTM2NzU_size_16_color_FFFFFF_t_70 1]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/17/5d395770966246409f5b2cc9b4288525.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2xpOTM2NzU_size_16_color_FFFFFF_t_70 2]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/17/5952c1fe645a4cc1911155a731dd2378.png