Java爬虫学习笔记 JSOUP

客官°小女子只卖身不卖艺 2023-01-02 05:20 142阅读 0赞

### Java爬虫 学习笔记 JSOUP ###

*  添加依赖
 *  解析document
 *   *  从字符串解析
     *  从fragment解析
     *  从URL解析
     *  从文件解析
 *  传参
 *  参考

# 添加依赖 #

[https://jsoup.org/download][https_jsoup.org_download]

<dependency>
      
      <groupId>org.jsoup</groupId>
      <artifactId>jsoup</artifactId>
      <version>1.13.1</version>
    </dependency>

# 解析document #

## 从字符串解析 ##

String html = "<html><head><title>First parse</title></head><body><p>Parsed HTML into a doc.</p></body></html>";
    Document doc = Jsoup.parse(html);

## 从fragment解析 ##

String html = "<div><p>Lorem ipsum.</p>";
    Document doc = Jsoup.parseBodyFragment(html);
    Element body = doc.body();

## 从URL解析 ##

// 发 get 请求，拿到 doc 对象。
    Document doc = Jsoup.connect("https://jerryjin.blog.csdn.net").get();
    // 通过选择器获取博客名称
    String blogTitle = doc.select(".title-blog").text();
    System.out.println(blogTitle); // 湖边的小屋遗址

*  添加请求头

Document doc = Jsoup.connect("https://jerryjin.blog.csdn.net")
            .userAgent("Mozilla")
            .cookie("auth", "token")
            .timeout(3000)
            .get();
    String blogTitle = doc.select(".title-blog").text();
    System.out.println(blogTitle);

## 从文件解析 ##

[Jsoup.parse(输入流, 字符集, baseUri解析相对路径用)][Jsoup.parse_ _ baseUri]

File input = new File("/tmp/input.html");
    Document doc = Jsoup.parse(input, "UTF-8", "http://example.com/");

# 传参 #

Document doc = Jsoup.connect("http://example.com")
      .data("query", "Java") // 传参
      .userAgent("Mozilla")
      .cookie("auth", "token")
      .timeout(3000)
      .post();

*  多参数

Map<String, String> data = new HashMap<>();
    data.put("username", "笨笨");
    data.put("passwd", "9527");
            
    Document doc = Jsoup.connect("http://example.com")
      .data(data) // 传参
      .userAgent("Mozilla")
      .cookie("auth", "token")
      .timeout(3000)
      .post();

# 参考 #

[JSOUP 官网 - 首页][JSOUP _ -]  
[JSOUP 官网 - 选择器语法 selector][JSOUP _ - _ selector]  
[JSOUP 官网 - DOM 操作方法：getElementByXX、attr、append 等等][JSOUP _ - DOM _getElementByXX_attr_append]  
[JSOUP 官网 - 设置属性值][JSOUP _ - 1]

[JSOUP 官网 - 参考文档 Elements][JSOUP _ - _ Elements]  
[JSOUP 官网 - 参考文档 Package org.jsoup.nodes][JSOUP _ - _ Package org.jsoup.nodes]  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2p4NTIw_size_16_color_FFFFFF_t_70]

[https_jsoup.org_download]: https://jsoup.org/download
[Jsoup.parse_ _ baseUri]: https://jsoup.org/apidocs/org/jsoup/Jsoup.html#parse%28java.io.File,java.lang.String,java.lang.String%29
[JSOUP _ -]: https://jsoup.org/
[JSOUP _ - _ selector]: https://jsoup.org/cookbook/extracting-data/selector-syntax
[JSOUP _ - DOM _getElementByXX_attr_append]: https://jsoup.org/cookbook/extracting-data/dom-navigation
[JSOUP _ - 1]: https://jsoup.org/cookbook/modifying-data/set-attributes
[JSOUP _ - _ Elements]: https://jsoup.org/apidocs/org/jsoup/select/Elements.html
[JSOUP _ - _ Package org.jsoup.nodes]: https://jsoup.org/apidocs/org/jsoup/nodes/package-summary.html
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2p4NTIw_size_16_color_FFFFFF_t_70]: /images/20221120/57ca184d53f843e181bb1547cebe9809.png