Requests与BeautifulSoup

- 日理万妓 2021-09-30 07:26 329阅读 0赞

一.requests 模块

Python标准库中提供了：urllib、urllib2、httplib等模块以供Http请求，但是，它的 API 太渣了。它是为另一个时代、另一个互联网所创建的。它需要巨量的工作，甚至包括各种方法覆盖，来完成最简单的任务。

Requests 是使用 Apache2 Licensed 许可证的 基于Python开发的HTTP 库，其在Python内置模块的基础上进行了高度的封装，从而使得Pythoner进行网络请求时，变得美好了许多，使用Requests可以轻而易举的完成浏览器可有的任何操作。

1.GET请求

1.无参数
    
    import requests
    
    ret = requests.get('https://github.com/timeline.json')
    
    print(ret.url)
    print(ret.text)
    
    
    
    2.有参数
    
    import requests
    
    payload = {'k1': 'v1', 'k2': 'v2'}
    ret = requests.get("http://www.baidu.com", params=payload)
    
    print(ret.url)
    print(ret.text)
    #http://www.baidu.com/?k1=v1&k2=v2    params(参数：k1=v1&k2=v2 )

2.POST请求

requests.post(
        url='xxx',
        params={'k1':'v1','nid':888},
        cookies={},
        headers={},
    
        #数据格式
        data={},
        json={}
    
    )
    
    
    #注意：请求头
      默认请求头：application/x-www-form-urlencoded,  浏览器显示的数据就是  Form Data
    
    requests.post(url='',data={},headers={'content-type': 'application/json'})  # 若还是用data={}， 后端拿不到值
    
    requests.post(url='',json={})  # {'content-type': 'application/json'}  #用requests.body
    
    
    
    示例：
    import requests
    import json
    
    url = 'https://www.jiemian.com/'
    payload = {'some': 'data'}
    headers = {'content-type': 'application/json'}
    
    ret = requests.post(url, data=json.dumps(payload), headers=headers)
    
    print(ret.cookies)
    print(ret.text)

3.其他请求

requests.get(url, params=None, **kwargs)
    requests.post(url, data=None, json=None, **kwargs)
    requests.put(url, data=None, **kwargs)
    requests.head(url, **kwargs)
    requests.delete(url, **kwargs)
    requests.patch(url, data=None, **kwargs)
    requests.options(url, **kwargs)
      
    # 以上方法均是在此方法的基础上构建
    requests.request(method, url, **kwargs)
    
    
    
    #额外补充
    $.post()
    
    $.ajax({
        type:'PSOT',
    })
    
    $.ajax({
        type:'GET',
        dataType:'JSON',
    })
    
    # 等价关系

更多参数：

![ContractedBlock.gif][] ![ExpandedBlockStart.gif][]

def request(method, url, **kwargs):
        """Constructs and sends a :class:`Request <Request>`.
     
        :param method: method for the new :class:`Request` object.
        :param url: URL for the new :class:`Request` object.
        :param params: (optional) Dictionary or bytes to be sent in the query string for the :class:`Request`.
        :param data: (optional) Dictionary, bytes, or file-like object to send in the body of the :class:`Request`.
        :param json: (optional) json data to send in the body of the :class:`Request`.
        :param headers: (optional) Dictionary of HTTP Headers to send with the :class:`Request`.
        :param cookies: (optional) Dict or CookieJar object to send with the :class:`Request`.
        :param files: (optional) Dictionary of ``'name': file-like-objects`` (or ``{'name': file-tuple}``) for multipart encoding upload.
            ``file-tuple`` can be a 2-tuple ``('filename', fileobj)``, 3-tuple ``('filename', fileobj, 'content_type')``
            or a 4-tuple ``('filename', fileobj, 'content_type', custom_headers)``, where ``'content-type'`` is a string
            defining the content type of the given file and ``custom_headers`` a dict-like object containing additional headers
            to add for the file.
        :param auth: (optional) Auth tuple to enable Basic/Digest/Custom HTTP Auth.
        :param timeout: (optional) How long to wait for the server to send data
            before giving up, as a float, or a :ref:`(connect timeout, read
            timeout) <timeouts>` tuple.
        :type timeout: float or tuple
        :param allow_redirects: (optional) Boolean. Set to True if POST/PUT/DELETE redirect following is allowed.
        :type allow_redirects: bool
        :param proxies: (optional) Dictionary mapping protocol to the URL of the proxy. #代理
        :param verify: (optional) whether the SSL cert will be verified. A CA_BUNDLE path can also be provided. Defaults to ``True``.
        :param stream: (optional) if ``False``, the response content will be immediately downloaded.
        :param cert: (optional) if String, path to ssl client cert file (.pem). If Tuple, ('cert', 'key') pair.
        :return: :class:`Response <Response>` object
        :rtype: requests.Response
     
        Usage::
     
          >>> import requests
          >>> req = requests.request('GET', 'http://httpbin.org/get')
          <Response [200]>
        """

参数示例：

![ContractedBlock.gif][] ![ExpandedBlockStart.gif][]

def param_method_url():
        # requests.request(method='get', url='http://127.0.0.1:8000/test/')
        # requests.request(method='post', url='http://127.0.0.1:8000/test/')
        pass
    
    
    def param_param():
        # - 可以是字典
        # - 可以是字符串
        # - 可以是字节（ascii编码以内）
    
        # requests.request(method='get',
        # url='http://127.0.0.1:8000/test/',
        # params={'k1': 'v1', 'k2': '水电费'})
    
        # requests.request(method='get',
        # url='http://127.0.0.1:8000/test/',
        # params="k1=v1&k2=水电费&k3=v3&k3=vv3")
    
        # requests.request(method='get',
        # url='http://127.0.0.1:8000/test/',
        # params=bytes("k1=v1&k2=k2&k3=v3&k3=vv3", encoding='utf8'))
    
        # 错误
        # requests.request(method='get',
        # url='http://127.0.0.1:8000/test/',
        # params=bytes("k1=v1&k2=水电费&k3=v3&k3=vv3", encoding='utf8'))
        pass
    
    
    def param_data():
        # 可以是字典
        # 可以是字符串
        # 可以是字节
        # 可以是文件对象
    
        # requests.request(method='POST',
        # url='http://127.0.0.1:8000/test/',
        # data={'k1': 'v1', 'k2': '水电费'})
    
        # requests.request(method='POST',
        # url='http://127.0.0.1:8000/test/',
        # data="k1=v1; k2=v2; k3=v3; k3=v4"
        # )
    
        # requests.request(method='POST',
        # url='http://127.0.0.1:8000/test/',
        # data="k1=v1;k2=v2;k3=v3;k3=v4",
        # headers={'Content-Type': 'application/x-www-form-urlencoded'}
        # )
    
        # requests.request(method='POST',
        # url='http://127.0.0.1:8000/test/',
        # data=open('data_file.py', mode='r', encoding='utf-8'), # 文件内容是：k1=v1;k2=v2;k3=v3;k3=v4
        # headers={'Content-Type': 'application/x-www-form-urlencoded'}
        # )
        pass
    
    
    def param_json():
        # 将json中对应的数据进行序列化成一个字符串，json.dumps(...)
        # 然后发送到服务器端的body中，并且Content-Type是 {'Content-Type': 'application/json'}
        requests.request(method='POST',
                         url='http://127.0.0.1:8000/test/',
                         json={
         'k1': 'v1', 'k2': '水电费'})
    
    
    def param_headers():
        # 发送请求头到服务器端
        requests.request(method='POST',
                         url='http://127.0.0.1:8000/test/',
                         json={
         'k1': 'v1', 'k2': '水电费'},
                         headers={
         'Content-Type': 'application/x-www-form-urlencoded'}
                         )
    
    
    def param_cookies():
        # 发送Cookie到服务器端
        requests.request(method='POST',
                         url='http://127.0.0.1:8000/test/',
                         data={
         'k1': 'v1', 'k2': 'v2'},
                         cookies={
         'cook1': 'value1'},
                         )
        # 也可以使用CookieJar（字典形式就是在此基础上封装）
        from http.cookiejar import CookieJar
        from http.cookiejar import Cookie
    
        obj = CookieJar()
        obj.set_cookie(Cookie(version=0, name='c1', value='v1', port=None, domain='', path='/', secure=False, expires=None,
                              discard=True, comment=None, comment_url=None, rest={
         'HttpOnly': None}, rfc2109=False,
                              port_specified=False, domain_specified=False, domain_initial_dot=False, path_specified=False)
                       )
        requests.request(method='POST',
                         url='http://127.0.0.1:8000/test/',
                         data={
         'k1': 'v1', 'k2': 'v2'},
                         cookies=obj)
    
    
    def param_files():
        # 发送文件
        # file_dict = {
          
        # 'f1': open('readme', 'rb')
        # }
        # requests.request(method='POST',
        # url='http://127.0.0.1:8000/test/',
        # files=file_dict)
    
        # 发送文件，定制文件名
        # file_dict = {
          
        # 'f1': ('test.txt', open('readme', 'rb'))
        # }
        # requests.request(method='POST',
        # url='http://127.0.0.1:8000/test/',
        # files=file_dict)
    
        # 发送文件，定制文件名
        # file_dict = {
          
        # 'f1': ('test.txt', "hahsfaksfa9kasdjflaksdjf")
        # }
        # requests.request(method='POST',
        # url='http://127.0.0.1:8000/test/',
        # files=file_dict)
    
        # 发送文件，定制文件名
        # file_dict = {
          
        #     'f1': ('test.txt', "hahsfaksfa9kasdjflaksdjf", 'application/text', {'k1': '0'})
        # }
        # requests.request(method='POST',
        #                  url='http://127.0.0.1:8000/test/',
        #                  files=file_dict)
    
        pass
    
    
    def param_auth():
        from requests.auth import HTTPBasicAuth, HTTPDigestAuth
    
        ret = requests.get('https://api.github.com/user', auth=HTTPBasicAuth('wupeiqi', 'sdfasdfasdf'))
        print(ret.text)
    
        # ret = requests.get('http://192.168.1.1',
        # auth=HTTPBasicAuth('admin', 'admin'))
        # ret.encoding = 'gbk'
        # print(ret.text)
    
        # ret = requests.get('http://httpbin.org/digest-auth/auth/user/pass', auth=HTTPDigestAuth('user', 'pass'))
        # print(ret)
        #
    
    
    def param_timeout():
        # ret = requests.get('http://google.com/', timeout=1)
        # print(ret)
    
        # ret = requests.get('http://google.com/', timeout=(5, 1))
        # print(ret)
        pass
    
    
    def param_allow_redirects():
        ret = requests.get('http://127.0.0.1:8000/test/', allow_redirects=False)
        print(ret.text)
    
    
    #代理
    def param_proxies():
        # proxies = {
          
        # "http": "61.172.249.96:80",
        # "https": "http://61.185.219.126:3128",
        # }
    
        # proxies = {'http://10.20.1.128': 'http://10.10.1.10:5323'}
    
        # ret = requests.get("http://www.proxy360.cn/Proxy", proxies=proxies)
        # print(ret.headers)
    
    
        # from requests.auth import HTTPProxyAuth
        #
        # proxyDict = {
          
        # 'http': '77.75.105.165',
        # 'https': '77.75.105.165'
        # }
        # auth = HTTPProxyAuth('username', 'mypassword')
        #
        # r = requests.get("http://www.google.com", proxies=proxyDict, auth=auth)
        # print(r.text)
    
        pass
    
    
    #证书开始
    def param_stream(): 
        ret = requests.get('http://127.0.0.1:8000/test/', stream=True)
        print(ret.content)
        ret.close()  
    
        # from contextlib import closing  #自动关闭文件
        # with closing(requests.get('http://httpbin.org/get', stream=True)) as r:
        # # 在此处理响应。
        # for i in r.iter_content():
        # print(i)
    
    
    # 知乎，可带可不带，【知乎，xxxx】
    # True,带
    # False,不太
    # 必须带证书，证书是个 文件（xxx.pem）
    # requests.get('http://httpbin.org/get', stream=True,cert="xxxx.pem")
    
    #结束
    
    
    
    
    def requests_session():
        import requests
    
        session = requests.Session()
    
        ### 1、首先登陆任何页面，获取cookie
    
        i1 = session.get(url="http://dig.chouti.com/help/service")
    
        ### 2、用户登陆，携带上一次的cookie，后台对cookie中的 gpsd 进行授权
        i2 = session.post(
            url="http://dig.chouti.com/login",
            data={
                'phone': "8615131255089",
                'password': "xxxxxx",
                'oneMonth': ""
            }
        )
    
        i3 = session.post(
            url="http://dig.chouti.com/link/vote?linksId=8589623",
        )
        print(i3.text)

### 3.相应Response ###

1.response属性

![ContractedBlock.gif][] ![ExpandedBlockStart.gif][]

import requests
    respone=requests.get('http://www.jianshu.com')
    # respone属性
    print(respone.text)
    print(respone.content)
    
    print(respone.status_code)
    print(respone.headers)
    print(respone.cookies)
    print(respone.cookies.get_dict())
    print(respone.cookies.items())
    
    print(respone.url)
    print(respone.history)
    
    print(respone.encoding)
    
    #关闭：response.close()
    from contextlib import closing
    with closing(requests.get('xxx',stream=True)) as response:
        for line in response.iter_content():
        pass

**2、编码问题**

![ContractedBlock.gif][] ![ExpandedBlockStart.gif][]

#编码问题
    import requests
    
    response=requests.get('http://www.autohome.com/news')
    # response.encoding='gbk' #汽车之家网站返回的页面内容为gb2312编码的，而requests的默认编码为ISO-8859-1，如果不设置成gbk则中文乱码
    
    print(response.text)

**3、获取二进制数据**

![ContractedBlock.gif][] ![ExpandedBlockStart.gif][]

import requests
    
    response=requests.get('https://timgsa.baidu.com/timg?image&quality=80&size=b9999_10000&sec=1509868306530&di=712e4ef3ab258b36e9f4b48e85a81c9d&imgtype=0&src=http%3A%2F%2Fc.hiphotos.baidu.com%2Fimage%2Fpic%2Fitem%2F11385343fbf2b211e1fb58a1c08065380dd78e0c.jpg')
    
    with open('a.jpg','wb') as f:
        f.write(response.content)

![ContractedBlock.gif][] ![ExpandedBlockStart.gif][]

#stream参数:一点一点的取,比如下载视频时,如果视频100G,用response.content然后一下子写到文件中是不合理的
    
    import requests
    
    response=requests.get('https://gss3.baidu.com/6LZ0ej3k1Qd3ote6lo7D0j9wehsv/tieba-smallvideo-transcode/1767502_56ec685f9c7ec542eeaf6eac93a65dc7_6fe25cd1347c_3.mp4',
                          stream=True)
    
    with open('b.mp4','wb') as f:
        for line in response.iter_content():
            f.write(line)

获取二进制流

**4、解析json**

![ContractedBlock.gif][] ![ExpandedBlockStart.gif][]

#解析json
    import requests
    response=requests.get('http://httpbin.org/get')
    
    import json
    res1=json.loads(response.text) #太麻烦
    
    res2=response.json() #直接获取json数据
    
    
    print(res1 == res2) #True

**5、Redirection and History**

BeautifulSoup

BeautifulSoup是一个模块，该模块用于接收一个HTML或XML字符串，然后将其进行格式化，之后便可以使用他提供的方法进行快速查找指定元素，从而使得在HTML或XML中查找指定元素变得简单。

from bs4 import BeautifulSoup
    
    
    html_doc = """
    <html>
    <head><title>The Dormouse's story</title></head>
    <body>
    asdf
    <div class="title">
        <b>The Dormouse's story总共</b>
        <h1>f</h1>
    </div>
    <div class="story">从前，有三个小女孩，她们的名字分别是
        <a class="sister0" id="link1">艾薇儿<span>FFF</span>ie</a>,
        <a href="http://example.com/lacie" class="sister" id="link2">Lacie</a> and
        <a href="http://example.com/tillie" class="sister" id="link3">Tillie</a>;
        and they lived at the bottom of a well.
    </div>
    ad<br/>sf
    <p class="story">...</p>
    </body>
    </html></html>
    """
    
    soup = BeautifulSoup(html_doc,'html.parser')

1.name，标签名称

# 找到第一个a标签
    tag = soup.find(name='a')
    # 找到所有的a标签
    tag2 = soup.find_all('a')
    # 找到id＝link2的标签
    tag3 = soup.select('#link2')

2. attr，标签属性

tag = soup.find('a')
    attr = tag.attrs
    #print(attr)  #{'class': ['sister0'], 'id': 'link1'}
    
    tag.attrs = {'ik':12345}
    #设置后，  <a ik="12345">艾薇儿<span>f</span>ie</a>,
    
    tag.attrs['id'] = 'iiiii'  #设置
    print(soup)

3. children,所有子标签

body = soup.find('body')
    
    v = body.children

4. children,所有子子孙孙标签

body = soup.find('body')
    
    v = body.descendants

5. clear,将标签的所有子标签全部清空（保留标签名）

tag = soup.find('body')
    tag.clear()
    print(soup)

6. decompose,递归的删除所有的标签

tag = soup.find('body')
    tag.decompose()
    
    print(soup)

7. extract,递归的删除所有的标签，并获取删除的标签

body = soup.find('body')
    v = body.extract()
    print(v)
    print('---'*10)
    print(soup)

8. decode,转换为字符串（含当前标签）；decode\_contents（不含当前标签）

tag = soup.find(class_= 'story')
    print(tag)
    
    v = tag.decode()
    print(v)
    
    v = tag.decode_contents()
    print(v)

9. encode,转换为字节（含当前标签）；encode\_contents（不含当前标签）

tag = soup.find(class_= 'story')
    print(tag)
    
    v = tag.encode()
    print(v)
    
    v = tag.encode_contents()
    print(v)

10. find,获取匹配的第一个标签

tag = soup.find('a')
    print(tag)
    
    # tag = soup.find(name='a',attrs={'class':'sister'},recursive=True,text='Lacie')
    tag = soup.find(name='a',class_='sister',recursive=True,text='Lacie')
    print(tag)

11. find\_all,获取匹配的所有标签

# tags = soup.find_all('a')
    # print(tags)
     
    # tags = soup.find_all('a',limit=1)
    # print(tags)
     
    # tags = soup.find_all(name='a', attrs={'class': 'sister'}, recursive=True, text='Lacie')
    # # tags = soup.find(name='a', class_='sister', recursive=True, text='Lacie')
    # print(tags)
     
     
    # ####### 列表 #######
    # v = soup.find_all(name=['a','div'])
    # print(v)
     
    # v = soup.find_all(class_=['sister0', 'sister'])
    # print(v)
     
    # v = soup.find_all(text=['Tillie'])
    # print(v, type(v[0]))
     
     
    # v = soup.find_all(id=['link1','link2'])
    # print(v)
     
    # v = soup.find_all(href=['link1','link2'])
    # print(v)
    
     
    # ####### 正则 #######
    import re
    # rep = re.compile('p')
    # rep = re.compile('^p')
    # v = soup.find_all(name=rep)
    # print(v)
     
    # rep = re.compile('sister.*')
    # v = soup.find_all(class_=rep)
    # print(v)
     
    # rep = re.compile('xxxx.*')
    # v = soup.find_all(href=rep)
    # print(v)
     
    # ####### 方法筛选 #######
    # def func(tag):
    # return tag.has_attr('class') and tag.has_attr('id')
    # v = soup.find_all(name=func)
    # print(v)
     
     
    # ## get,获取标签属性
    # tag = soup.find('a')
    # v = tag.get('id')
    # print(v)

12. has\_attr,检查标签是否具有该属性

tag = soup.find('a')
    v = tag.has_attr('id')
    
    print(v)  #True

13. get\_text,获取标签内部文本内容

tag = soup.find('a')
    v = tag.get_text('id')
    
    print(v)

14. index,检查标签在某标签中的索引位置

# tag = soup.find('body')
    # v = tag.index(tag.find('div'))
    # print(v)
     
    # tag = soup.find('body')
    # for i,v in enumerate(tag):
    # print(i,v)

15. is\_empty\_element,是否是空标签(是否可以是空)或者自闭合标签，

判断是否是如下标签：'br' , 'hr', 'input', 'img', 'meta','spacer', 'link', 'frame', 'base'

# tag = soup.find('br')
    # v = tag.is_empty_element
    # print(v)

16. 当前的关联标签

# soup.next
    # soup.next_element
    # soup.next_elements
    # soup.next_sibling
    # soup.next_siblings
     
    #
    # tag.previous
    # tag.previous_element
    # tag.previous_elements
    # tag.previous_sibling
    # tag.previous_siblings
     
    #
    # tag.parent
    # tag.parents

17. 查找某标签的关联标签

# tag.find_next(...)
    # tag.find_all_next(...)
    # tag.find_next_sibling(...)
    # tag.find_next_siblings(...)
     
    # tag.find_previous(...)
    # tag.find_all_previous(...)
    # tag.find_previous_sibling(...)
    # tag.find_previous_siblings(...)
     
    # tag.find_parent(...)
    # tag.find_parents(...)
     
    # 参数同find_all

18. select,select\_one, CSS选择器

soup.select("title")
     
    soup.select("p nth-of-type(3)")
     
    soup.select("body a")
     
    soup.select("html head title")
     
    tag = soup.select("span,a")
     
    soup.select("head > title")
     
    soup.select("p > a")
     
    soup.select("p > a:nth-of-type(2)")
     
    soup.select("p > #link1")
     
    soup.select("body > a")
     
    soup.select("#link1 ~ .sister")
     
    soup.select("#link1 + .sister")
     
    soup.select(".sister")
     
    soup.select("[class~=sister]")
     
    soup.select("#link1")
     
    soup.select("a#link2")
     
    soup.select('a[href]')
     
    soup.select('a[href="http://example.com/elsie"]')
     
    soup.select('a[href^="http://example.com/"]')
     
    soup.select('a[href$="tillie"]')
     
    soup.select('a[href*=".com/el"]')
     
     
    from bs4.element import Tag
     
    def default_candidate_generator(tag):
        for child in tag.descendants:
            if not isinstance(child, Tag):
                continue
            if not child.has_attr('href'):
                continue
            yield child
     
    tags = soup.find('body').select("a", _candidate_generator=default_candidate_generator)
    print(type(tags), tags)
     
    from bs4.element import Tag
    def default_candidate_generator(tag):
        for child in tag.descendants:
            if not isinstance(child, Tag):
                continue
            if not child.has_attr('href'):
                continue
            yield child
     
    tags = soup.find('body').select("a", _candidate_generator=default_candidate_generator, limit=1)
    print(type(tags), tags)

19. 标签的内容

# tag = soup.find('span')
    # print(tag.string)          # 获取
    # tag.string = 'new content' # 设置
    # print(soup)
     
    # tag = soup.find('body')
    # print(tag.string)
    # tag.string = 'xxx'
    # print(soup)
     
    # tag = soup.find('body')
    # v = tag.stripped_strings  # 递归内部获取所有标签的文本
    # print(v)

20.append在当前标签内部追加一个标签

# tag = soup.find('body')
    # tag.append(soup.find('a'))
    # print(soup)
    #
    # from bs4.element import Tag
    # obj = Tag(name='i',attrs={'id': 'it'})
    # obj.string = '我是一个新来的'
    # tag = soup.find('body')
    # tag.append(obj)
    # print(soup)

21.insert在当前标签内部指定位置插入一个标签

# from bs4.element import Tag
    # obj = Tag(name='i', attrs={'id': 'it'})
    # obj.string = '我是一个新来的'
    # tag = soup.find('body')
    # tag.insert(2, obj)
    # print(soup)

22. insert\_after,insert\_before 在当前标签后面或前面插入

# from bs4.element import Tag
    # obj = Tag(name='i', attrs={'id': 'it'})
    # obj.string = '我是一个新来的'
    # tag = soup.find('body')
    # # tag.insert_before(obj)
    # tag.insert_after(obj)
    # print(soup)

23. replace\_with 在当前标签替换为指定标签

# from bs4.element import Tag
    # obj = Tag(name='i', attrs={'id': 'it'})
    # obj.string = '我是一个新来的'
    # tag = soup.find('div')
    # tag.replace_with(obj)
    # print(soup)

24. 创建标签之间的关系

# tag = soup.find('div')
    # a = soup.find('a')
    # tag.setup(previous_sibling=a)
    # print(tag.previous_sibling)

25. wrap，将指定标签把当前标签包裹起来

# from bs4.element import Tag
    # obj1 = Tag(name='div', attrs={'id': 'it'})
    # obj1.string = '我是一个新来的'
    #
    # tag = soup.find('a')
    # v = tag.wrap(obj1)
    # print(soup)
     
    # tag = soup.find('a')
    # v = tag.wrap(soup.find('p'))
    # print(soup)

26. unwrap，去掉当前标签，将保留其包裹的标签

# tag = soup.find('a')
    # v = tag.unwrap()
    # print(soup)

转载于:https://www.cnblogs.com/zhaochangbo/p/7546083.html

[ContractedBlock.gif]: https://images.cnblogs.com/OutliningIndicators/ContractedBlock.gif
[ExpandedBlockStart.gif]: /images/20210725/7c2821c144b64cdabc54d399d2e95d51.png

发表评论取消回复

表情：

评论列表（有 0 条评论，329人围观）

还没有评论，来说两句吧...

相关阅读

相关 Python爬虫实践：使用requests和BeautifulSoup的案例

在Python中，我们可以使用两个库来编写爬虫：requests用于发送HTTP请求获取网页内容，而BeautifulSoup则负责解析这些内容并提取我们需要的信息。以下是

太过爱你忘了你带给我的痛/ 2025年01月02日 06:33/ 0 赞/ 23 阅读

相关 Python爬虫开发：requests和BeautifulSoup实践案例

在Python爬虫开发中，requests库用于发送HTTP请求，获取响应内容；而BeautifulSoup库则用于解析HTML文档，提取我们需要的信息。以下是一个简单的r

╰半夏微凉°/ 2024年12月27日 05:18/ 0 赞/ 24 阅读

相关 Python爬虫实战：基于requests和BeautifulSoup的问题

在Python的爬虫实践中，经常会遇到使用requests库获取网页内容以及BeautifulSoup进行解析的问题。以下是一些常见问题及解决方案： 1. **无法获取网页内

女爷i/ 2024年12月21日 21:18/ 0 赞/ 29 阅读

相关 Python爬虫实践：requests+BeautifulSoup案例

在Python爬虫中，requests和BeautifulSoup是两个常用库。requests用于发送HTTP请求获取响应数据；BeautifulSoup则用于解析HTML文

忘是亡心i/ 2024年12月19日 02:51/ 0 赞/ 42 阅读

相关 python中request请求库与BeautifulSoup解析库的用法

python中request请求库与BeautifulSoup解析库的用法 request 安装打开cmd窗口，检查python环境，需要python3.7

古城微笑少年丶/ 2024年03月27日 08:55/ 0 赞/ 88 阅读

相关 Python3爬虫使用requests和BeautifulSoup

Python3爬虫是一种非常强大的工具，可以帮助我们从互联网上获取各种数据。本文将从入门到精通，为大家介绍Python3爬虫的基础知识、常用库以及实战案例。一、Python

快来打我*/ 2024年03月24日 01:28/ 0 赞/ 96 阅读

相关 Python下基于requests及BeautifulSoup构建网络爬虫

功能说明在Python下面可使用requests模块请求某个url获取响应的html文件，接着使用BeautifulSoup解析某个html。案例假设我要[

喜欢ヅ旅行/ 2022年06月08日 13:40/ 0 赞/ 247 阅读

相关 Requests，BeautifulSoup，自动登录实例

requests Python标准库中提供了：urllib、urllib2、httplib等模块以供Http请求，但是，它的 API 太渣了。它是为另一个时代、另一个互联

小咪咪/ 2022年05月28日 11:29/ 0 赞/ 248 阅读

相关 pycharm导入requests和beautifulSoup

1、在pycharm界面 , File -> Setting或(Ctrl + Alt + S) 进入 setting 界面 ![watermark_type_ZmFu

╰半夏微凉°/ 2022年04月23日 09:52/ 0 赞/ 217 阅读

相关 Requests与BeautifulSoup

一.requests 模块 Python标准库中提供了：urllib、urllib2、httplib等模块以供Http请求，但是，它的 API 太渣了。它是为另一个时代、另一

- 日理万妓/ 2021年09月30日 07:26/ 0 赞/ 330 阅读