用 Java 实现断点续传 (HTTP)（转）

系统管理员 2021-05-12 11:50 456阅读 0赞

## 断点续传的原理 ##

其实断点续传的原理很简单，就是在 Http 的请求上和一般的下载有所不同而已。   
打个比方，浏览器请求服务器上的一个文时，所发出的请求如下：   
假设服务器域名为 wwww.sjtu.edu.cn，文件名为 down.zip。   
GET /down.zip HTTP/1.1   
Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/vnd.ms-   
excel, application/msword, application/vnd.ms-powerpoint, \*/\*   
Accept-Language: zh-cn   
Accept-Encoding: gzip, deflate   
User-Agent: Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)   
Connection: Keep-Alive

服务器收到请求后，按要求寻找请求的文件，提取文件的信息，然后返回给浏览器，返回信息如下：

200   
Content-Length=106786028   
Accept-Ranges=bytes   
Date=Mon, 30 Apr 2001 12:56:11 GMT   
ETag=W/"02ca57e173c11:95b"   
Content-Type=application/octet-stream   
Server=Microsoft-IIS/5.0   
Last-Modified=Mon, 30 Apr 2001 12:56:11 GMT

所谓断点续传，也就是要从文件已经下载的地方开始继续下载。所以在客户端浏览器传给 Web 服务器的时候要多加一条信息 -- 从哪里开始。   
下面是用自己编的一个"浏览器"来传递请求信息给 Web 服务器，要求从 2000070 字节开始。   
GET /down.zip HTTP/1.0   
User-Agent: NetFox   
RANGE: bytes=2000070-   
Accept: text/html, image/gif, image/jpeg, \*; q=.2, \*/\*; q=.2

仔细看一下就会发现多了一行 RANGE: bytes=2000070-   
这一行的意思就是告诉服务器 down.zip 这个文件从 2000070 字节开始传，前面的字节不用传了。   
服务器收到这个请求以后，返回的信息如下：   
206   
Content-Length=106786028   
Content-Range=bytes 2000070-106786027/106786028   
Date=Mon, 30 Apr 2001 12:55:20 GMT   
ETag=W/"02ca57e173c11:95b"   
Content-Type=application/octet-stream   
Server=Microsoft-IIS/5.0   
Last-Modified=Mon, 30 Apr 2001 12:55:20 GMT

和前面服务器返回的信息比较一下，就会发现增加了一行：   
Content-Range=bytes 2000070-106786027/106786028   
返回的代码也改为 206 了，而不再是 200 了。

知道了以上原理，就可以进行断点续传的编程了。

--------------------

## Java 实现断点续传的关键几点 ##

1.  (1) 用什么方法实现提交 RANGE: bytes=2000070-。   
    当然用最原始的 Socket 是肯定能完成的，不过那样太费事了，其实 Java 的 net 包中提供了这种功能。代码如下：   
      
    URL url = new URL("http://www.sjtu.edu.cn/down.zip");   
    HttpURLConnection httpConnection = (HttpURLConnection)url.openConnection();   
      
    // 设置 User-Agent   
    httpConnection.setRequestProperty("User-Agent","NetFox");   
    // 设置断点续传的开始位置   
    httpConnection.setRequestProperty("RANGE","bytes=2000070");   
    // 获得输入流   
    InputStream input = httpConnection.getInputStream();   
    
    
    从输入流中取出的字节流就是 down.zip 文件从 2000070 开始的字节流。 大家看，其实断点续传用 Java 实现起来还是很简单的吧。 接下来要做的事就是怎么保存获得的流到文件中去了。
2.  保存文件采用的方法。   
    我采用的是 IO 包中的 RandAccessFile 类。   
    操作相当简单，假设从 2000070 处开始保存文件，代码如下：   
    RandomAccess oSavedFile = new RandomAccessFile("down.zip","rw");   
    long nPos = 2000070;   
    // 定位文件指针到 nPos 位置   
    oSavedFile.seek(nPos);   
    byte\[\] b = new byte\[1024\];   
    int nRead;   
    // 从输入流中读入字节流，然后写到文件中   
    while((nRead=input.read(b,0,1024)) > 0)   
    \{   
    oSavedFile.write(b,0,nRead);   
    \}

怎么样，也很简单吧。 接下来要做的就是整合成一个完整的程序了。包括一系列的线程控制等等。

--------------------

## 断点续传内核的实现 ##

主要用了 6 个类，包括一个测试类。   
SiteFileFetch.java 负责整个文件的抓取，控制内部线程 (FileSplitterFetch 类 )。   
FileSplitterFetch.java 负责部分文件的抓取。   
FileAccess.java 负责文件的存储。   
SiteInfoBean.java 要抓取的文件的信息，如文件保存的目录，名字，抓取文件的 URL 等。   
Utility.java 工具类，放一些简单的方法。   
TestMethod.java 测试类。

下面是源程序：

![ExpandedBlockStart.gif][]

![复制代码][copycode.gif]

1 /* 
      2  /*
      3  * SiteFileFetch.java 
      4  */ 
      5  package NetFox; 
      6  import java.io.*; 
      7  import java.net.*; 
      8  public class SiteFileFetch extends Thread { 
      9  SiteInfoBean siteInfoBean = null; // 文件信息 Bean 
     10  long[] nStartPos; // 开始位置
     11  long[] nEndPos; // 结束位置
     12  FileSplitterFetch[] fileSplitterFetch; // 子线程对象
     13  long nFileLength; // 文件长度
     14  boolean bFirst = true; // 是否第一次取文件
     15  boolean bStop = false; // 停止标志
     16  File tmpFile; // 文件下载的临时信息
     17  DataOutputStream output; // 输出到文件的输出流
     18  public SiteFileFetch(SiteInfoBean bean) throws IOException 
     19  { 
     20  siteInfoBean = bean; 
     21  //tmpFile = File.createTempFile ("zhong","1111",new File(bean.getSFilePath())); 
     22  tmpFile = new File(bean.getSFilePath()+File.separator + bean.getSFileName()+".info");
     23  if(tmpFile.exists ()) 
     24  { 
     25  bFirst = false; 
     26  read_nPos(); 
     27  } 
     28  else 
     29  { 
     30  nStartPos = new long[bean.getNSplitter()]; 
     31  nEndPos = new long[bean.getNSplitter()]; 
     32  } 
     33  } 
     34  public void run() 
     35  { 
     36  // 获得文件长度
     37  // 分割文件
     38  // 实例 FileSplitterFetch 
     39  // 启动 FileSplitterFetch 线程
     40  // 等待子线程返回
     41  try{ 
     42  if(bFirst) 
     43  { 
     44  nFileLength = getFileSize(); 
     45  if(nFileLength == -1) 
     46  { 
     47  System.err.println("File Length is not known!"); 
     48  } 
     49  else if(nFileLength == -2) 
     50  { 
     51  System.err.println("File is not access!"); 
     52  } 
     53  else 
     54  { 
     55  for(int i=0;i<nStartPos.length;i++) 
     56  { 
     57  nStartPos[i] = (long)(i*(nFileLength/nStartPos.length)); 
     58  } 
     59  for(int i=0;i<nEndPos.length-1;i++) 
     60  { 
     61  nEndPos[i] = nStartPos[i+1]; 
     62  } 
     63  nEndPos[nEndPos.length-1] = nFileLength; 
     64  } 
     65  } 
     66  // 启动子线程
     67  fileSplitterFetch = new FileSplitterFetch[nStartPos.length]; 
     68  for(int i=0;i<nStartPos.length;i++) 
     69  { 
     70  fileSplitterFetch[i] = new FileSplitterFetch(siteInfoBean.getSSiteURL(), 
     71  siteInfoBean.getSFilePath() + File.separator + siteInfoBean.getSFileName(), 
     72  nStartPos[i],nEndPos[i],i); 
     73  Utility.log("Thread " + i + " , nStartPos = " + nStartPos[i] + ", nEndPos = " 
     74  + nEndPos[i]); 
     75  fileSplitterFetch[i].start(); 
     76  } 
     77  // fileSplitterFetch[nPos.length-1] = new FileSplitterFetch(siteInfoBean.getSSiteURL(),
     78  siteInfoBean.getSFilePath() + File.separator 
     79  + siteInfoBean.getSFileName(),nPos[nPos.length-1],nFileLength,nPos.length-1); 
     80  // Utility.log("Thread " +(nPos.length-1) + ",nStartPos = "+nPos[nPos.length-1]+",
     81  nEndPos = " + nFileLength); 
     82  // fileSplitterFetch[nPos.length-1].start(); 
     83  // 等待子线程结束
     84  //int count = 0; 
     85  // 是否结束 while 循环
     86  boolean breakWhile = false; 
     87  while(!bStop) 
     88  { 
     89  write_nPos(); 
     90  Utility.sleep(500); 
     91  breakWhile = true; 
     92  for(int i=0;i<nStartPos.length;i++) 
     93  { 
     94  if(!fileSplitterFetch[i].bDownOver) 
     95  { 
     96  breakWhile = false; 
     97  break; 
     98  } 
     99  } 
    100  if(breakWhile) 
    101  break; 
    102  //count++; 
    103  //if(count>4) 
    104  // siteStop(); 
    105  } 
    106  System.err.println("文件下载结束！"); 
    107  } 
    108  catch(Exception e){e.printStackTrace ();} 
    109  } 
    110  // 获得文件长度
    111  public long getFileSize() 
    112  { 
    113  int nFileLength = -1; 
    114  try{ 
    115  URL url = new URL(siteInfoBean.getSSiteURL()); 
    116  HttpURLConnection httpConnection = (HttpURLConnection)url.openConnection ();
    117  httpConnection.setRequestProperty("User-Agent","NetFox"); 
    118  int responseCode=httpConnection.getResponseCode(); 
    119  if(responseCode>=400) 
    120  { 
    121  processErrorCode(responseCode); 
    122  return -2; //-2 represent access is error 
    123  } 
    124  String sHeader; 
    125  for(int i=1;;i++) 
    126  { 
    127  //DataInputStream in = new DataInputStream(httpConnection.getInputStream ()); 
    128  //Utility.log(in.readLine()); 
    129  sHeader=httpConnection.getHeaderFieldKey(i); 
    130  if(sHeader!=null) 
    131  { 
    132  if(sHeader.equals("Content-Length")) 
    133  { 
    134  nFileLength = Integer.parseInt(httpConnection.getHeaderField(sHeader)); 
    135  break; 
    136  } 
    137  } 
    138  else 
    139  break; 
    140  } 
    141  } 
    142  catch(IOException e){e.printStackTrace ();} 
    143  catch(Exception e){e.printStackTrace ();} 
    144  Utility.log(nFileLength); 
    145  return nFileLength; 
    146  } 
    147  // 保存下载信息（文件指针位置）
    148  private void write_nPos() 
    149  { 
    150  try{ 
    151  output = new DataOutputStream(new FileOutputStream(tmpFile)); 
    152  output.writeInt(nStartPos.length); 
    153  for(int i=0;i<nStartPos.length;i++) 
    154  { 
    155  // output.writeLong(nPos[i]); 
    156  output.writeLong(fileSplitterFetch[i].nStartPos); 
    157  output.writeLong(fileSplitterFetch[i].nEndPos); 
    158  } 
    159  output.close(); 
    160  } 
    161  catch(IOException e){e.printStackTrace ();} 
    162  catch(Exception e){e.printStackTrace ();} 
    163  } 
    164  // 读取保存的下载信息（文件指针位置）
    165  private void read_nPos() 
    166  { 
    167  try{ 
    168  DataInputStream input = new DataInputStream(new FileInputStream(tmpFile)); 
    169  int nCount = input.readInt(); 
    170  nStartPos = new long[nCount]; 
    171  nEndPos = new long[nCount]; 
    172  for(int i=0;i<nStartPos.length;i++) 
    173  { 
    174  nStartPos[i] = input.readLong(); 
    175  nEndPos[i] = input.readLong(); 
    176  } 
    177  input.close(); 
    178  } 
    179  catch(IOException e){e.printStackTrace ();} 
    180  catch(Exception e){e.printStackTrace ();} 
    181  } 
    182  private void processErrorCode(int nErrorCode) 
    183  { 
    184  System.err.println("Error Code : " + nErrorCode); 
    185  } 
    186  // 停止文件下载
    187  public void siteStop() 
    188  { 
    189  bStop = true; 
    190  for(int i=0;i<nStartPos.length;i++) 
    191  fileSplitterFetch[i].splitterStop(); 
    192  } 
    193  }

![复制代码][copycode.gif]

![ExpandedBlockStart.gif][]

![复制代码][copycode.gif]

1 /* 
      2  **FileSplitterFetch.java 
      3  */ 
      4  package NetFox; 
      5  import java.io.*; 
      6  import java.net.*; 
      7  public class FileSplitterFetch extends Thread { 
      8  String sURL; //File URL 
      9  long nStartPos; //File Snippet Start Position 
     10  long nEndPos; //File Snippet End Position 
     11  int nThreadID; //Thread's ID 
     12  boolean bDownOver = false; //Downing is over 
     13  boolean bStop = false; //Stop identical 
     14  FileAccessI fileAccessI = null; //File Access interface 
     15  public FileSplitterFetch(String sURL,String sName,long nStart,long nEnd,int id)
     16  throws IOException 
     17  { 
     18  this.sURL = sURL; 
     19  this.nStartPos = nStart; 
     20  this.nEndPos = nEnd; 
     21  nThreadID = id; 
     22  fileAccessI = new FileAccessI(sName,nStartPos); 
     23  } 
     24  public void run() 
     25  { 
     26  while(nStartPos < nEndPos && !bStop) 
     27  { 
     28  try{ 
     29  URL url = new URL(sURL); 
     30  HttpURLConnection httpConnection = (HttpURLConnection)url.openConnection (); 
     31  httpConnection.setRequestProperty("User-Agent","NetFox"); 
     32  String sProperty = "bytes="+nStartPos+"-"; 
     33  httpConnection.setRequestProperty("RANGE",sProperty); 
     34  Utility.log(sProperty); 
     35  InputStream input = httpConnection.getInputStream(); 
     36  //logResponseHead(httpConnection); 
     37  byte[] b = new byte[1024]; 
     38  int nRead; 
     39  while((nRead=input.read(b,0,1024)) > 0 && nStartPos < nEndPos 
     40  && !bStop) 
     41  { 
     42  nStartPos += fileAccessI.write(b,0,nRead); 
     43  //if(nThreadID == 1) 
     44  // Utility.log("nStartPos = " + nStartPos + ", nEndPos = " + nEndPos); 
     45  } 
     46  Utility.log("Thread " + nThreadID + " is over!"); 
     47  bDownOver = true; 
     48  //nPos = fileAccessI.write (b,0,nRead); 
     49  } 
     50  catch(Exception e){e.printStackTrace ();} 
     51  } 
     52  } 
     53  // 打印回应的头信息
     54  public void logResponseHead(HttpURLConnection con) 
     55  { 
     56  for(int i=1;;i++) 
     57  { 
     58  String header=con.getHeaderFieldKey(i); 
     59  if(header!=null) 
     60  //responseHeaders.put(header,httpConnection.getHeaderField(header)); 
     61  Utility.log(header+" : "+con.getHeaderField(header)); 
     62  else 
     63  break; 
     64  } 
     65  } 
     66  public void splitterStop() 
     67  { 
     68  bStop = true; 
     69  } 
     70  } 
     71  
     72  /* 
     73  **FileAccess.java 
     74  */ 
     75  package NetFox; 
     76  import java.io.*; 
     77  public class FileAccessI implements Serializable{ 
     78  RandomAccessFile oSavedFile; 
     79  long nPos; 
     80  public FileAccessI() throws IOException 
     81  { 
     82  this("",0); 
     83  } 
     84  public FileAccessI(String sName,long nPos) throws IOException 
     85  { 
     86  oSavedFile = new RandomAccessFile(sName,"rw"); 
     87  this.nPos = nPos; 
     88  oSavedFile.seek(nPos); 
     89  } 
     90  public synchronized int write(byte[] b,int nStart,int nLen) 
     91  { 
     92  int n = -1; 
     93  try{ 
     94  oSavedFile.write(b,nStart,nLen); 
     95  n = nLen; 
     96  } 
     97  catch(IOException e) 
     98  { 
     99  e.printStackTrace (); 
    100  } 
    101  return n; 
    102  } 
    103  } 
    104  
    105  /* 
    106  **SiteInfoBean.java 
    107  */ 
    108  package NetFox; 
    109  public class SiteInfoBean { 
    110  private String sSiteURL; //Site's URL 
    111  private String sFilePath; //Saved File's Path 
    112  private String sFileName; //Saved File's Name 
    113  private int nSplitter; //Count of Splited Downloading File 
    114  public SiteInfoBean() 
    115  { 
    116  //default value of nSplitter is 5 
    117  this("","","",5); 
    118  } 
    119  public SiteInfoBean(String sURL,String sPath,String sName,int nSpiltter)
    120  { 
    121  sSiteURL= sURL; 
    122  sFilePath = sPath; 
    123  sFileName = sName; 
    124  this.nSplitter = nSpiltter; 
    125  } 
    126  public String getSSiteURL() 
    127  { 
    128  return sSiteURL; 
    129  } 
    130  public void setSSiteURL(String value) 
    131  { 
    132  sSiteURL = value; 
    133  } 
    134  public String getSFilePath() 
    135  { 
    136  return sFilePath; 
    137  } 
    138  public void setSFilePath(String value) 
    139  { 
    140  sFilePath = value; 
    141  } 
    142  public String getSFileName() 
    143  { 
    144  return sFileName; 
    145  } 
    146  public void setSFileName(String value) 
    147  { 
    148  sFileName = value; 
    149  } 
    150  public int getNSplitter() 
    151  { 
    152  return nSplitter; 
    153  } 
    154  public void setNSplitter(int nCount) 
    155  { 
    156  nSplitter = nCount; 
    157  } 
    158  } 
    159  
    160  /* 
    161  **Utility.java 
    162  */ 
    163  package NetFox; 
    164  public class Utility { 
    165  public Utility() 
    166  { 
    167  } 
    168  public static void sleep(int nSecond) 
    169  { 
    170  try{ 
    171  Thread.sleep(nSecond); 
    172  } 
    173  catch(Exception e) 
    174  { 
    175  e.printStackTrace (); 
    176  } 
    177  } 
    178  public static void log(String sMsg) 
    179  { 
    180  System.err.println(sMsg); 
    181  } 
    182  public static void log(int sMsg) 
    183  { 
    184  System.err.println(sMsg); 
    185  } 
    186  } 
    187  
    188  /* 
    189  **TestMethod.java 
    190  */ 
    191  package NetFox; 
    192  public class TestMethod { 
    193  public TestMethod() 
    194  { ///xx/weblogic60b2_win.exe 
    195  try{ 
    196  SiteInfoBean bean = new SiteInfoBean("http://localhost/xx/weblogic60b2_win.exe",
    197      "L:\\temp","weblogic60b2_win.exe",5); 
    198  //SiteInfoBean bean = new SiteInfoBean("http://localhost:8080/down.zip","L:\\temp",
    199      "weblogic60b2_win.exe",5); 
    200  SiteFileFetch fileFetch = new SiteFileFetch(bean); 
    201  fileFetch.start(); 
    202  } 
    203  catch(Exception e){e.printStackTrace ();} 
    204  } 
    205  public static void main(String[] args) 
    206  { 
    207  new TestMethod(); 
    208  } 
    209  }

![复制代码][copycode.gif]

原文链接：[http://www.ibm.com/developerworks/cn/java/joy-down/][http_www.ibm.com_developerworks_cn_java_joy-down]

[ExpandedBlockStart.gif]: /images/20210511/9701ec74550c44629e51ef86309b947f.png
[copycode.gif]: /images/20210511/d4dc32d853cc4fe090f49efc42149607.png
[http_www.ibm.com_developerworks_cn_java_joy-down]: http://www.ibm.com/developerworks/cn/java/joy-down/