Java解决jdk低版本证书问题 sun.security.validator.ValidatorException: PKIX path building failed

墨蓝 2024-04-19 10:30 94阅读 0赞

# 一、问题 #

这几天发现项目的微博抓取没数据展示了，查了一下数据库，原来上个月26号就停了

奇了个怪的，然后就追溯了一下日志

找到了错误日志：

[ERROR][2019-09-16 06:02:46,943][com.spider.controler.weibo.WeiBoParser.parse(WeiBoParser.java:40)]get weibo user home error!url=http://weibo.com/u/3886607933?refer_flag=1001030101_&is_all=1
    javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    	at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
    	at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1904)
    	at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:279)
    	at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:273)
    	at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1446)
    	at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:209)
    	at sun.security.ssl.Handshaker.processLoop(Handshaker.java:901)
    	at sun.security.ssl.Handshaker.process_record(Handshaker.java:837)
    	at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1023)
    	at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1332)
    	at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1359)
    	at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1343)
    	at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:394)
    	at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:353)
    	at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:141)
    	at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:353)
    	at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:380)
    	at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)
    	at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
    	at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
    	at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
    	at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
    	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:71)
    	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:220)
    	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:164)
    	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:139)
    	at org.bigdatacn.sfgk.wlzbs.controler.weibo.WeiBoUserHomeParser.parse(WeiBoUserHomeParser.java:38)
    	at org.bigdatacn.sfgk.wlzbs.controler.weibo.task.WeiBoUrlTask.crawl(WeiBoUrlTask.java:127)
    	at org.bigdatacn.sfgk.wlzbs.controler.weibo.task.WeiBoUrlTask.crawlJob(WeiBoUrlTask.java:101)
    	at org.bigdatacn.sfgk.wlzbs.controler.weibo.task.WeiBoUrlTask.grab(WeiBoUrlTask.java:71)
    	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    	at java.lang.reflect.Method.invoke(Method.java:606)
    	at org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:65)
    	at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
    	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
    	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304)
    	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178)
    	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
    	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    	at java.lang.Thread.run(Thread.java:745)
    Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    	at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:385)
    	at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292)
    	at sun.security.validator.Validator.validate(Validator.java:260)
    	at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:326)
    	at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:231)
    	at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:126)
    	at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1428)
    	... 38 more
    Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    	at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:196)
    	at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:268)
    	at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:380)
    	... 44 more

然后就在本地项目测试发现竟然是正常的

![20190917133852402.png][]

观察了一下，本地用的jdk8测试，没有出现问题，然后服务器上是jdk7，所以出现了这个问题，这是由于本地与服务器所使用的SSL/TLS版本不一致。服务器使用的TLS版本高，而客户端支持的TLS版本低。**Java 8默认支持TLSv1.2版本**。

微博网页都变https了

# 二、解决方案 #

### 1、升级jdk ###

具体的就不写了，随便百度就能找到

因为有人是windows，有人linux，方法也不一样

### 2、生成证书 ###

/* 
     * Copyright 2006 Sun Microsystems, Inc.  All Rights Reserved. 
     * 
     * Redistribution and use in source and binary forms, with or without 
     * modification, are permitted provided that the following conditions 
     * are met: 
     * 
     *   - Redistributions of source code must retain the above copyright 
     *     notice, this list of conditions and the following disclaimer. 
     * 
     *   - Redistributions in binary form must reproduce the above copyright 
     *     notice, this list of conditions and the following disclaimer in the 
     *     documentation and/or other materials provided with the distribution. 
     * 
     *   - Neither the name of Sun Microsystems nor the names of its 
     *     contributors may be used to endorse or promote products derived 
     *     from this software without specific prior written permission. 
     * 
     * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS 
     * IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, 
     * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR 
     * PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL THE COPYRIGHT OWNER OR 
     * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, 
     * EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, 
     * PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR 
     * PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF 
     * LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING 
     * NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS 
     * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 
     */  
      
    import java.io.BufferedReader;  
    import java.io.File;  
    import java.io.FileInputStream;  
    import java.io.FileOutputStream;  
    import java.io.InputStream;  
    import java.io.InputStreamReader;  
    import java.io.OutputStream;  
    import java.security.KeyStore;  
    import java.security.MessageDigest;  
    import java.security.cert.CertificateException;  
    import java.security.cert.X509Certificate;  
      
    import javax.net.ssl.SSLContext;  
    import javax.net.ssl.SSLException;  
    import javax.net.ssl.SSLSocket;  
    import javax.net.ssl.SSLSocketFactory;  
    import javax.net.ssl.TrustManager;  
    import javax.net.ssl.TrustManagerFactory;  
    import javax.net.ssl.X509TrustManager;  
      
    public class InstallCert {  
      
        public static void main(String[] args) throws Exception {  
            String host;  
            int port;  
            char[] passphrase;  
            if ((args.length == 1) || (args.length == 2)) {  
                String[] c = args[0].split(":");  
                host = c[0];  
                port = (c.length == 1) ? 443 : Integer.parseInt(c[1]);  
                String p = (args.length == 1) ? "changeit" : args[1];  
                passphrase = p.toCharArray();  
            } else {  
                System.out  
                        .println("Usage: java InstallCert <host>[:port] [passphrase]");  
                return;  
            }  
      
            File file = new File("jssecacerts");  
            if (file.isFile() == false) {  
                char SEP = File.separatorChar;  
                File dir = new File(System.getProperty("java.home") + SEP + "lib"  
                        + SEP + "security");  
                file = new File(dir, "jssecacerts");  
                if (file.isFile() == false) {  
                    file = new File(dir, "cacerts");  
                }  
            }  
            System.out.println("Loading KeyStore " + file + "...");  
            InputStream in = new FileInputStream(file);  
            KeyStore ks = KeyStore.getInstance(KeyStore.getDefaultType());  
            ks.load(in, passphrase);  
            in.close();  
      
            SSLContext context = SSLContext.getInstance("TLS");  
            TrustManagerFactory tmf = TrustManagerFactory  
                    .getInstance(TrustManagerFactory.getDefaultAlgorithm());  
            tmf.init(ks);  
            X509TrustManager defaultTrustManager = (X509TrustManager) tmf  
                    .getTrustManagers()[0];  
            SavingTrustManager tm = new SavingTrustManager(defaultTrustManager);  
            context.init(null, new TrustManager[] { tm }, null);  
            SSLSocketFactory factory = context.getSocketFactory();  
      
            System.out  
                    .println("Opening connection to " + host + ":" + port + "...");  
            SSLSocket socket = (SSLSocket) factory.createSocket(host, port);  
            socket.setSoTimeout(10000);  
            try {  
                System.out.println("Starting SSL handshake...");  
                socket.startHandshake();  
                socket.close();  
                System.out.println();  
                System.out.println("No errors, certificate is already trusted");  
            } catch (SSLException e) {  
                System.out.println();  
                e.printStackTrace(System.out);  
            }  
      
            X509Certificate[] chain = tm.chain;  
            if (chain == null) {  
                System.out.println("Could not obtain server certificate chain");  
                return;  
            }  
      
            BufferedReader reader = new BufferedReader(new InputStreamReader(  
                    System.in));  
      
            System.out.println();  
            System.out.println("Server sent " + chain.length + " certificate(s):");  
            System.out.println();  
            MessageDigest sha1 = MessageDigest.getInstance("SHA1");  
            MessageDigest md5 = MessageDigest.getInstance("MD5");  
            for (int i = 0; i < chain.length; i++) {  
                X509Certificate cert = chain[i];  
                System.out.println(" " + (i + 1) + " Subject "  
                        + cert.getSubjectDN());  
                System.out.println("   Issuer  " + cert.getIssuerDN());  
                sha1.update(cert.getEncoded());  
                System.out.println("   sha1    " + toHexString(sha1.digest()));  
                md5.update(cert.getEncoded());  
                System.out.println("   md5     " + toHexString(md5.digest()));  
                System.out.println();  
            }  
      
            System.out  
                    .println("Enter certificate to add to trusted keystore or 'q' to quit: [1]");  
            String line = reader.readLine().trim();  
            int k;  
            try {  
                k = (line.length() == 0) ? 0 : Integer.parseInt(line) - 1;  
            } catch (NumberFormatException e) {  
                System.out.println("KeyStore not changed");  
                return;  
            }  
      
            X509Certificate cert = chain[k];  
            String alias = host + "-" + (k + 1);  
            ks.setCertificateEntry(alias, cert);  
      
            OutputStream out = new FileOutputStream("jssecacerts");  
            ks.store(out, passphrase);  
            out.close();  
      
            System.out.println();  
            System.out.println(cert);  
            System.out.println();  
            System.out  
                    .println("Added certificate to keystore 'jssecacerts' using alias '"  
                            + alias + "'");  
        }  
      
        private static final char[] HEXDIGITS = "0123456789abcdef".toCharArray();  
      
        private static String toHexString(byte[] bytes) {  
            StringBuilder sb = new StringBuilder(bytes.length * 3);  
            for (int b : bytes) {  
                b &= 0xff;  
                sb.append(HEXDIGITS[b >> 4]);  
                sb.append(HEXDIGITS[b & 15]);  
                sb.append(' ');  
            }  
            return sb.toString();  
        }  
      
        private static class SavingTrustManager implements X509TrustManager {  
      
            private final X509TrustManager tm;  
            private X509Certificate[] chain;  
      
            SavingTrustManager(X509TrustManager tm) {  
                this.tm = tm;  
            }  
      
            public X509Certificate[] getAcceptedIssuers() {  
                throw new UnsupportedOperationException();  
            }  
      
            public void checkClientTrusted(X509Certificate[] chain, String authType)  
                    throws CertificateException {  
                throw new UnsupportedOperationException();  
            }  
      
            public void checkServerTrusted(X509Certificate[] chain, String authType)  
                    throws CertificateException {  
                this.chain = chain;  
                tm.checkServerTrusted(chain, authType);  
            }  
        }  
      
    }

在电脑某个你熟悉的地方新建一个txt

把上面的代码放到txt中，然后保存，重命名文件为

InstallCert.java

然后编译一下java文件

在文件夹内按住shift然后右键

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTI5OTgzMDY_size_16_color_FFFFFF_t_70][]

选择在此处打开命令窗口

编译：

javac InstallCert.java

运行：

java InstallCert www.weibo.com

**注：我是因为微博抓不到生成微博证书**

**所以 java InstallCert (要生成证书的网站)**

输完命令之后回车出现如图信息

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTI5OTgzMDY_size_16_color_FFFFFF_t_70 1][]

根据提示输入 1 然后回车

[
    [
      Version: V3
      Subject: CN=*.weibo.com, O="Sina.com Technology (China) Co., Ltd.", L=Beijin
     ST=Beijing, C=CN
      Signature Algorithm: SHA384withECDSA, OID = 1.2.840.10045.4.3.3
    
      Key:  Sun EC public key, 256 bits
      public x coord: 123315875432783152221904536036809075583890703163686656991850
    536779801316622
      public y coord: 833422678118437284553618318631202115878810846320782411162031
    483753735996056
      parameters: secp256r1 [NIST P-256, X9.62 prime256v1] (1.2.840.10045.3.1.7)
      Validity: [From: Fri Aug 16 10:12:01 GMT+08:00 2019,
                   To: Thu Oct 07 20:59:24 GMT+08:00 2021]
      Issuer: CN=GlobalSign ECC OV SSL CA 2018, O=GlobalSign nv-sa, C=BE
      SerialNumber: [    33aac3b1 aea0f9c3 8a6a1f7a]
    
    Certificate Extensions: 10
    [1]: ObjectId: 1.3.6.1.4.1.11129.2.4.2 Criticality=false
    Extension unknown: DER encoded OCTET string =
    0000: 04 82 01 6E 04 82 01 6A   01 68 00 76 00 5C DC 43  ...n...j.h.v.\.C
    0010: 92 FE E6 AB 45 44 B1 5E   9A D4 56 E6 10 37 FB D5  ....ED.^..V..7..
    0020: FA 47 DC A1 73 94 B2 5E   E6 F6 C7 0E CA 00 00 01  .G..s..^........
    0030: 6C 98 32 3A E2 00 00 04   03 00 47 30 45 02 21 00  l.2:......G0E.!.
    0040: 99 03 0C 54 C6 34 07 FE   DA E5 C1 D1 85 45 4F 2A  ...T.4.......EO*
    0050: 50 94 52 94 56 9B 2A E8   CF F1 0D 90 A8 5E 10 98  P.R.V.*......^..
    0060: 02 20 2F 40 4B C1 BE 4D   A5 11 45 49 F5 0C 73 66  . /@K..M..EI..sf
    0070: DE DC 63 43 26 35 83 3B   B7 BF F7 A9 4C 5D F2 74  ..cC&5.;....L].t
    0080: CD CE 00 76 00 EE 4B BD   B7 75 CE 60 BA E1 42 69  ...v..K..u.`..Bi
    0090: 1F AB E1 9E 66 A3 0F 7E   5F B0 72 D8 83 00 C4 7B  ....f..._.r.....
    00A0: 89 7A A8 FD CB 00 00 01   6C 98 32 37 BB 00 00 04  .z......l.27....
    00B0: 03 00 47 30 45 02 20 5E   E3 90 81 5B 9E A9 BB B6  ..G0E. ^...[....
    00C0: 19 11 86 86 61 FA 9F 06   84 39 FC 5D B7 86 DC 19  ....a....9.]....
    00D0: B8 D9 3A 81 04 92 7C 02   21 00 F5 3A B9 D2 17 29  ..:.....!..:...)
    00E0: C9 80 80 56 FB 2A 80 9B   89 BD 04 DC 47 B7 84 74  ...V.*......G..t
    00F0: 1C 57 86 31 1C 74 38 5E   4D 14 00 76 00 55 81 D4  .W.1.t8^M..v.U..
    0100: C2 16 90 36 01 4A EA 0B   9B 57 3C 53 F0 C0 E4 38  ...6.J...W<S...8
    0110: 78 70 25 08 17 2F A3 AA   1D 07 13 D3 0C 00 00 01  xp%../..........
    0120: 6C 98 32 3A 81 00 00 04   03 00 47 30 45 02 20 46  l.2:......G0E. F
    0130: AD C1 D1 5E D3 46 B1 ED   21 07 8F 3C 86 C0 09 39  ...^.F..!..<...9
    0140: B2 05 81 2E 29 34 D3 9F   4D 3F A8 08 0A D2 85 02  ....)4..M?......
    0150: 21 00 B9 30 E5 70 B9 24   D0 E2 1B 7D F3 B1 12 70  !..0.p.$.......p
    0160: 38 40 D9 64 BE 21 5C E9   CE 2F 85 4B C2 7C E2 1D  8@.d.!\../.K....
    0170: DB 6A                                              .j
    
    
    [2]: ObjectId: 1.3.6.1.5.5.7.1.1 Criticality=false
    AuthorityInfoAccess [
      [
       accessMethod: caIssuers
       accessLocation: URIName: http://secure.globalsign.com/cacert/gseccovsslca20
    .crt
    ,
       accessMethod: ocsp
       accessLocation: URIName: http://ocsp.globalsign.com/gseccovsslca2018
    ]
    ]
    
    [3]: ObjectId: 2.5.29.35 Criticality=false
    AuthorityKeyIdentifier [
    KeyIdentifier [
    0000: 58 7B 8E 75 2A FE 61 80   AA 90 40 01 AE D6 E8 07  X..u*.a...@.....
    0010: 46 6E 3F 48                                        Fn?H
    ]
    ]
    
    [4]: ObjectId: 2.5.29.19 Criticality=false
    BasicConstraints:[
      CA:false
      PathLen: undefined
    ]
    
    [5]: ObjectId: 2.5.29.31 Criticality=false
    CRLDistributionPoints [
      [DistributionPoint:
         [URIName: http://crl.globalsign.com/gseccovsslca2018.crl]
    ]]
    
    [6]: ObjectId: 2.5.29.32 Criticality=false
    CertificatePolicies [
      [CertificatePolicyId: [1.3.6.1.4.1.4146.1.20]
    [PolicyQualifierInfo: [
      qualifierID: 1.3.6.1.5.5.7.2.1
      qualifier: 0000: 16 26 68 74 74 70 73 3A   2F 2F 77 77 77 2E 67 6C  .&https:
    www.gl
    0010: 6F 62 61 6C 73 69 67 6E   2E 63 6F 6D 2F 72 65 70  obalsign.com/rep
    0020: 6F 73 69 74 6F 72 79 2F                            ository/
    
    ]]  ]
      [CertificatePolicyId: [2.23.140.1.2.2]
    []  ]
    ]
    
    [7]: ObjectId: 2.5.29.37 Criticality=false
    ExtendedKeyUsages [
      serverAuth
      clientAuth
    ]
    
    [8]: ObjectId: 2.5.29.15 Criticality=true
    KeyUsage [
      DigitalSignature
      Key_Agreement
    ]
    
    [9]: ObjectId: 2.5.29.17 Criticality=false
    SubjectAlternativeName [
      DNSName: *.weibo.com
      DNSName: weibo.com
    ]
    
    [10]: ObjectId: 2.5.29.14 Criticality=false
    SubjectKeyIdentifier [
    KeyIdentifier [
    0000: 87 2D 65 1E A9 9D 2A 42   03 72 AE 21 78 D6 0C 2B  .-e...*B.r.!x..+
    0010: 51 72 EF 45                                        Qr.E
    ]
    ]
    
    ]
      Algorithm: [SHA384withECDSA]
      Signature:
    0000: 30 65 02 31 00 C2 0F B3   42 B1 02 CB 8C 06 35 23  0e.1....B.....5#
    0010: 2F 81 5E 67 0B 82 8A 8A   DF DA F7 AB 1B EA D1 60  /.^g...........`
    0020: DE BC DA 56 98 C6 44 55   7F 96 C1 4C 55 21 72 1B  ...V..DU...LU!r.
    0030: FB 0C 65 87 16 02 30 5A   DC 52 C4 9C 92 9F D4 1B  ..e...0Z.R......
    0040: 4E 0D 7C 3D 13 6E D7 45   8E 08 AA ED 63 0F A5 1D  N..=.n.E....c...
    0050: 55 22 3A 43 EA 2D 70 22   C8 DA 05 F4 A4 5D F5 5C  U":C.-p".....].\
    0060: 50 28 6B CA 17 8F 01                               P(k....
    
    ]
    
    Added certificate to keystore 'jssecacerts' using alias 'www.weibo.com-1'

在当前目录会有一个名为jssecacerts的文件

### 3、导入证书 ###

复制证书文件到jdk安装路径\\jre\\lib\\security文件夹下

windows/Linux都是复制到jdk安装路径\\jre\\lib\\security

复制证书到这个目录下

比如我的Linux：![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTI5OTgzMDY_size_16_color_FFFFFF_t_70 2][]

# 三、其他问题 #

如果不想生成证书的话可以尝试java抓取时忽略掉证书。jsoup在调用前先执行下以下忽略证书请求就可以了。

try {
                //先调用下忽略https证书的再请求才可以
                HttpsUrlValidator.retrieveResponseFromServer(url); 
                doc = Jsoup
                        .connect(url)
                        .header("User-Agent",rand_agents)
                        .timeout(10000).get();
                body = doc.getElementsByTag("body").html();
            } catch (Exception e) {
                log.info(e.getMessage());
            }

HttpsUrlValidator 类：

import java.io.BufferedReader;
    import java.io.IOException;
    import java.io.InputStreamReader;
    import java.net.HttpURLConnection;
    import java.net.URL;
    
    import javax.net.ssl.HostnameVerifier;
    import javax.net.ssl.HttpsURLConnection;
    import javax.net.ssl.SSLSession;
    
    
    public class HttpsUrlValidator {
    
        static HostnameVerifier hv = new HostnameVerifier() {
            public boolean verify(String urlHostName, SSLSession session) {
                System.out.println("Warning: URL Host: " + urlHostName + " vs. "
                                   + session.getPeerHost());
                return true;
            }
        };
    
        public final static String retrieveResponseFromServer(final String url) {
            HttpURLConnection connection = null;
            
            try {
                URL validationUrl = new URL(url);
                trustAllHttpsCertificates();
                HttpsURLConnection.setDefaultHostnameVerifier(hv);
    
                connection = (HttpURLConnection) validationUrl.openConnection();
                final BufferedReader in = new BufferedReader(new InputStreamReader(
                        connection.getInputStream()));
    
                String line;
                final StringBuffer stringBuffer = new StringBuffer(255);
    
                synchronized (stringBuffer) {
                    while ((line = in.readLine()) != null) {
                        stringBuffer.append(line);
                        stringBuffer.append("\n");
                    }
                    return stringBuffer.toString();
                }
    
            } catch (final IOException e) {
                System.out.println(e.getMessage());
                return null;
            } catch (final Exception e1){
                System.out.println(e1.getMessage());
                return null;
            }finally {
                if (connection != null) {
                    connection.disconnect();
                }
            }
        }
        
        public static void trustAllHttpsCertificates() throws Exception {
            javax.net.ssl.TrustManager[] trustAllCerts = new javax.net.ssl.TrustManager[1];
            javax.net.ssl.TrustManager tm = new miTM();
            trustAllCerts[0] = tm;
            javax.net.ssl.SSLContext sc = javax.net.ssl.SSLContext
                    .getInstance("SSL");
            sc.init(null, trustAllCerts, null);
            javax.net.ssl.HttpsURLConnection.setDefaultSSLSocketFactory(sc
                    .getSocketFactory());
        }
    
        static class miTM implements javax.net.ssl.TrustManager,
                javax.net.ssl.X509TrustManager {
            public java.security.cert.X509Certificate[] getAcceptedIssuers() {
                return null;
            }
    
            public boolean isServerTrusted(
                    java.security.cert.X509Certificate[] certs) {
                return true;
            }
    
            public boolean isClientTrusted(
                    java.security.cert.X509Certificate[] certs) {
                return true;
            }
    
            public void checkServerTrusted(
                    java.security.cert.X509Certificate[] certs, String authType)
                    throws java.security.cert.CertificateException {
                return;
            }
    
            public void checkClientTrusted(
                    java.security.cert.X509Certificate[] certs, String authType)
                    throws java.security.cert.CertificateException {
                return;
            }
        }
    
    }

[20190917133852402.png]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/18/d73ecb17eb7d49498f8bd5af7f8940a1.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTI5OTgzMDY_size_16_color_FFFFFF_t_70]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/18/12e5eb6d8ee74a7ab28698d276678627.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTI5OTgzMDY_size_16_color_FFFFFF_t_70 1]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/18/ea1c00bb84214b2a8a20ae13b4ed2024.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTI5OTgzMDY_size_16_color_FFFFFF_t_70 2]: https://image.dandelioncloud.cn/pgy_files/images/2024/04/18/d10164db5a204481a9d4eeac21e12e0a.png