视音频数据处理入门：H.264视频码流解析

本是古典何须时尚 2022-07-17 04:43 251阅读 0赞

=====================================================

视音频数据处理入门系列文章：

[视音频数据处理入门：RGB、YUV像素数据处理][RGB_YUV]

[视音频数据处理入门：PCM音频采样数据处理][PCM]

[视音频数据处理入门：H.264视频码流解析][H.264]

[视音频数据处理入门：AAC音频码流解析][AAC]

[视音频数据处理入门：FLV封装格式解析][FLV]

[视音频数据处理入门：UDP-RTP协议解析][UDP-RTP]

=====================================================

前两篇文章介绍的YUV/RGB处理程序以及PCM处理程序都属于视音频原始数据的处理程序。从本文开始介绍视音频码流的处理程序。本文介绍的程序是视频码流处理程序。视频码流在视频播放器中的位置如下所示。

![20160118001528487][]

本文中的程序是一个H.264码流解析程序。该程序可以从H.264码流中分析得到它的基本单元NALU，并且可以简单解析NALU首部的字段。通过修改该程序可以实现不同的H.264码流处理功能。

## 原理 ##

H.264原始码流（又称为“裸流”）是由一个一个的NALU组成的。他们的结构如下图所示。

![20160118001549018][]

其中每个NALU之间通过startcode（起始码）进行分隔，起始码分成两种：0x000001（3Byte）或者0x00000001（4Byte）。如果NALU对应的Slice为一帧的开始就用0x00000001，否则就用0x000001。

H.264码流解析的步骤就是首先从码流中搜索0x000001和0x00000001，分离出NALU；然后再分析NALU的各个字段。本文的程序即实现了上述的两个步骤。

## 代码 ##

整个程序位于simplest\_h264\_parser()函数中，如下所示。

**\[cpp\]** [view plain][] [copy][view plain]

[![在CODE上查看代码片][CODE]][CODE_CODE] [![派生到我的代码片][ico_fork.svg]][ico_fork.svg 1]

1.  /\*\*
2.   \* 最简单的视音频数据处理示例
3.   \* Simplest MediaData Test
4.   \*
5.   \* 雷霄骅 Lei Xiaohua
6.   \* leixiaohua1020@126.com
7.   \* 中国传媒大学/数字电视技术
8.   \* Communication University of China / Digital TV Technology
9.   \* http://blog.csdn.net/leixiaohua1020
10.  \*
11.  \* 本项目包含如下几种视音频测试示例：
12.  \* (1)像素数据处理程序。包含RGB和YUV像素格式处理的函数。
13.  \* (2)音频采样数据处理程序。包含PCM音频采样格式处理的函数。
14.  \* (3)H.264码流分析程序。可以分离并解析NALU。
15.  \* (4)AAC码流分析程序。可以分离并解析ADTS帧。
16.  \* (5)FLV封装格式分析程序。可以将FLV中的MP3音频码流分离出来。
17.  \* (6)UDP-RTP协议分析程序。可以将分析UDP/RTP/MPEG-TS数据包。
18.  \*
19.  \* This project contains following samples to handling multimedia data:
20.  \* (1) Video pixel data handling program. It contains several examples to handle RGB and YUV data.
21.  \* (2) Audio sample data handling program. It contains several examples to handle PCM data.
22.  \* (3) H.264 stream analysis program. It can parse H.264 bitstream and analysis NALU of stream.
23.  \* (4) AAC stream analysis program. It can parse AAC bitstream and analysis ADTS frame of stream.
24.  \* (5) FLV format analysis program. It can analysis FLV file and extract MP3 audio stream.
25.  \* (6) UDP-RTP protocol analysis program. It can analysis UDP/RTP/MPEG-TS Packet.
26.  \*
27.  \*/
28. \#include <stdio.h>
29. \#include <stdlib.h>
30. \#include <string.h>
31. 
32. **typedef****enum** \{ 
33.  NALU\_TYPE\_SLICE = 1, 
34.  NALU\_TYPE\_DPA = 2, 
35.  NALU\_TYPE\_DPB = 3, 
36.  NALU\_TYPE\_DPC = 4, 
37.  NALU\_TYPE\_IDR = 5, 
38.  NALU\_TYPE\_SEI = 6, 
39.  NALU\_TYPE\_SPS = 7, 
40.  NALU\_TYPE\_PPS = 8, 
41.  NALU\_TYPE\_AUD = 9, 
42.  NALU\_TYPE\_EOSEQ = 10, 
43.  NALU\_TYPE\_EOSTREAM = 11, 
44.  NALU\_TYPE\_FILL = 12, 
45. \} NaluType; 
46. 
47. **typedef****enum** \{ 
48.  NALU\_PRIORITY\_DISPOSABLE = 0, 
49.  NALU\_PRIRITY\_LOW = 1, 
50.  NALU\_PRIORITY\_HIGH = 2, 
51.  NALU\_PRIORITY\_HIGHEST = 3 
52. \} NaluPriority; 
53. 
54. 
55. **typedef****struct**
56. \{ 
57. **int** startcodeprefix\_len; //! 4 for parameter sets and first slice in picture, 3 for everything else (suggested)
58.  unsigned len; //! Length of the NAL unit (Excluding the start code, which does not belong to the NALU)
59.  unsigned max\_size; //! Nal Unit Buffer size
60. **int** forbidden\_bit; //! should be always FALSE
61. **int** nal\_reference\_idc; //! NALU\_PRIORITY\_xxxx
62. **int** nal\_unit\_type; //! NALU\_TYPE\_xxxx 
63. **char** \*buf; //! contains the first byte followed by the EBSP
64. \} NALU\_t; 
65. 
66. **FILE** \*h264bitstream = NULL; //!< the bit stream file
67. 
68. **int** info2=0, info3=0; 
69. 
70. **static****int** FindStartCode2 (unsigned **char** \*Buf)\{ 
71. **if**(Buf\[0\]!=0 || Buf\[1\]!=0 || Buf\[2\] !=1) **return** 0; //0x000001?
72. **else****return** 1; 
73. \} 
74. 
75. **static****int** FindStartCode3 (unsigned **char** \*Buf)\{ 
76. **if**(Buf\[0\]!=0 || Buf\[1\]!=0 || Buf\[2\] !=0 || Buf\[3\] !=1) **return** 0;//0x00000001?
77. **else****return** 1; 
78. \} 
79. 
80. 
81. **int** GetAnnexbNALU (NALU\_t \*nalu)\{ 
82. **int** pos = 0; 
83. **int** StartCodeFound, rewind; 
84.  unsigned **char** \*Buf; 
85. 
86. **if** ((Buf = (unsigned **char**\*)calloc (nalu->max\_size , **sizeof**(**char**))) == NULL) 
87.  printf ("GetAnnexbNALU: Could not allocate Buf memory\\n"); 
88. 
89.  nalu->startcodeprefix\_len=3; 
90. 
91. **if** (3 != fread (Buf, 1, 3, h264bitstream))\{ 
92.  free(Buf); 
93. **return** 0; 
94.  \} 
95.  info2 = FindStartCode2 (Buf); 
96. **if**(info2 != 1) \{ 
97. **if**(1 != fread(Buf+3, 1, 1, h264bitstream))\{ 
98.  free(Buf); 
99. **return** 0; 
100.  \} 
101.  info3 = FindStartCode3 (Buf); 
102. **if** (info3 != 1)\{ 
103.  free(Buf); 
104. **return** \-1; 
105.  \} 
106. **else** \{ 
107.  pos = 4; 
108.  nalu->startcodeprefix\_len = 4; 
109.  \} 
110.  \} 
111. **else**\{ 
112.  nalu->startcodeprefix\_len = 3; 
113.  pos = 3; 
114.  \} 
115.  StartCodeFound = 0; 
116.  info2 = 0; 
117.  info3 = 0; 
118. 
119. **while** (!StartCodeFound)\{ 
120. **if** (feof (h264bitstream))\{ 
121.  nalu->len = (pos-1)-nalu->startcodeprefix\_len; 
122.  memcpy (nalu->buf, &Buf\[nalu->startcodeprefix\_len\], nalu->len); 
123.  nalu->forbidden\_bit = nalu->buf\[0\] & 0x80; //1 bit
124.  nalu->nal\_reference\_idc = nalu->buf\[0\] & 0x60; // 2 bit
125.  nalu->nal\_unit\_type = (nalu->buf\[0\]) & 0x1f;// 5 bit
126.  free(Buf); 
127. **return** pos-1; 
128.  \} 
129.  Buf\[pos++\] = fgetc (h264bitstream); 
130.  info3 = FindStartCode3(&Buf\[pos-4\]); 
131. **if**(info3 != 1) 
132.  info2 = FindStartCode2(&Buf\[pos-3\]); 
133.  StartCodeFound = (info2 == 1 || info3 == 1); 
134.  \} 
135. 
136. // Here, we have found another start code (and read length of startcode bytes more than we should
137. // have. Hence, go back in the file
138.  rewind = (info3 == 1)? -4 : -3; 
139. 
140. **if** (0 != fseek (h264bitstream, rewind, SEEK\_CUR))\{ 
141.  free(Buf); 
142.  printf("GetAnnexbNALU: Cannot fseek in the bit stream file"); 
143.  \} 
144. 
145. // Here the Start code, the complete NALU, and the next start code is in the Buf. 
146. // The size of Buf is pos, pos+rewind are the number of bytes excluding the next
147. // start code, and (pos+rewind)-startcodeprefix\_len is the size of the NALU excluding the start code
148. 
149.  nalu->len = (pos+rewind)-nalu->startcodeprefix\_len; 
150.  memcpy (nalu->buf, &Buf\[nalu->startcodeprefix\_len\], nalu->len);//
151.  nalu->forbidden\_bit = nalu->buf\[0\] & 0x80; //1 bit
152.  nalu->nal\_reference\_idc = nalu->buf\[0\] & 0x60; // 2 bit
153.  nalu->nal\_unit\_type = (nalu->buf\[0\]) & 0x1f;// 5 bit
154.  free(Buf); 
155. 
156. **return** (pos+rewind); 
157. \} 
158. 
159. /\*\*
160.  \* Analysis H.264 Bitstream
161.  \* @param url Location of input H.264 bitstream file.
162.  \*/
163. **int** simplest\_h264\_parser(**char** \*url)\{ 
164. 
165.  NALU\_t \*n; 
166. **int** buffersize=100000; 
167. 
168. //FILE \*myout=fopen("output\_log.txt","wb+");
169. **FILE** \*myout=stdout; 
170. 
171.  h264bitstream=fopen(url, "rb+"); 
172. **if** (h264bitstream==NULL)\{ 
173.  printf("Open file error\\n"); 
174. **return** 0; 
175.  \} 
176. 
177.  n = (NALU\_t\*)calloc (1, **sizeof** (NALU\_t)); 
178. **if** (n == NULL)\{ 
179.  printf("Alloc NALU Error\\n"); 
180. **return** 0; 
181.  \} 
182. 
183.  n->max\_size=buffersize; 
184.  n->buf = (**char**\*)calloc (buffersize, **sizeof** (**char**)); 
185. **if** (n->buf == NULL)\{ 
186.  free (n); 
187.  printf ("AllocNALU: n->buf"); 
188. **return** 0; 
189.  \} 
190. 
191. **int** data\_offset=0; 
192. **int** nal\_num=0; 
193.  printf("-----+-------- NALU Table ------+---------+\\n"); 
194.  printf(" NUM | POS | IDC | TYPE | LEN |\\n"); 
195.  printf("-----+---------+--------+-------+---------+\\n"); 
196. 
197. **while**(!feof(h264bitstream)) 
198.  \{ 
199. **int** data\_lenth; 
200.  data\_lenth=GetAnnexbNALU(n); 
201. 
202. **char** type\_str\[20\]=\{0\}; 
203. **switch**(n->nal\_unit\_type)\{ 
204. **case** NALU\_TYPE\_SLICE:sprintf(type\_str,"SLICE");**break**; 
205. **case** NALU\_TYPE\_DPA:sprintf(type\_str,"DPA");**break**; 
206. **case** NALU\_TYPE\_DPB:sprintf(type\_str,"DPB");**break**; 
207. **case** NALU\_TYPE\_DPC:sprintf(type\_str,"DPC");**break**; 
208. **case** NALU\_TYPE\_IDR:sprintf(type\_str,"IDR");**break**; 
209. **case** NALU\_TYPE\_SEI:sprintf(type\_str,"SEI");**break**; 
210. **case** NALU\_TYPE\_SPS:sprintf(type\_str,"SPS");**break**; 
211. **case** NALU\_TYPE\_PPS:sprintf(type\_str,"PPS");**break**; 
212. **case** NALU\_TYPE\_AUD:sprintf(type\_str,"AUD");**break**; 
213. **case** NALU\_TYPE\_EOSEQ:sprintf(type\_str,"EOSEQ");**break**; 
214. **case** NALU\_TYPE\_EOSTREAM:sprintf(type\_str,"EOSTREAM");**break**; 
215. **case** NALU\_TYPE\_FILL:sprintf(type\_str,"FILL");**break**; 
216.  \} 
217. **char** idc\_str\[20\]=\{0\}; 
218. **switch**(n->nal\_reference\_idc>>5)\{ 
219. **case** NALU\_PRIORITY\_DISPOSABLE:sprintf(idc\_str,"DISPOS");**break**; 
220. **case** NALU\_PRIRITY\_LOW:sprintf(idc\_str,"LOW");**break**; 
221. **case** NALU\_PRIORITY\_HIGH:sprintf(idc\_str,"HIGH");**break**; 
222. **case** NALU\_PRIORITY\_HIGHEST:sprintf(idc\_str,"HIGHEST");**break**; 
223.  \} 
224. 
225.  fprintf(myout,"%5d| %8d| %7s| %6s| %8d|\\n",nal\_num,data\_offset,idc\_str,type\_str,n->len); 
226. 
227.  data\_offset=data\_offset+data\_lenth; 
228. 
229.  nal\_num++; 
230.  \} 
231. 
232. //Free
233. **if** (n)\{ 
234. **if** (n->buf)\{ 
235.  free(n->buf); 
236.  n->buf=NULL; 
237.  \} 
238.  free (n); 
239.  \} 
240. **return** 0; 
241. \}

上文中的函数调用方法如下所示。

**\[cpp\]** [view plain][] [copy][view plain]

[![在CODE上查看代码片][CODE]][CODE_CODE] [![派生到我的代码片][ico_fork.svg]][ico_fork.svg 1]

1.  simplest\_h264\_parser("sintel.h264");

## 结果 ##

本程序的输入为一个H.264原始码流（裸流）的文件路径，输出为该码流的NALU统计数据，如下图所示。

![20160118003147122][]

## 下载 ##

**Simplest mediadata test**

项目主页

SourceForge：[https://sourceforge.net/projects/simplest-mediadata-test/][https_sourceforge.net_projects_simplest-mediadata-test]

Github：[https://github.com/leixiaohua1020/simplest\_mediadata\_test][https_github.com_leixiaohua1020_simplest_mediadata_test]

开源中国：[http://git.oschina.net/leixiaohua1020/simplest\_mediadata\_test][http_git.oschina.net_leixiaohua1020_simplest_mediadata_test]

CSDN下载地址：[http://download.csdn.net/detail/leixiaohua1020/9422409][http_download.csdn.net_detail_leixiaohua1020_9422409]

本项目包含如下几种视音频数据解析示例：

(1)像素数据处理程序。包含RGB和YUV像素格式处理的函数。  
 (2)音频采样数据处理程序。包含PCM音频采样格式处理的函数。  
 (3)H.264码流分析程序。可以分离并解析NALU。  
 (4)AAC码流分析程序。可以分离并解析ADTS帧。  
 (5)FLV封装格式分析程序。可以将FLV中的MP3音频码流分离出来。  
 (6)UDP-RTP协议分析程序。可以将分析UDP/RTP/MPEG-TS数据包。

**雷霄骅 (Lei Xiaohua)  
leixiaohua1020@126.com  
http://blog.csdn.net/leixiaohua1020**

[RGB_YUV]: http://blog.csdn.net/leixiaohua1020/article/details/50534150
[PCM]: http://blog.csdn.net/leixiaohua1020/article/details/50534316
[H.264]: http://blog.csdn.net/leixiaohua1020/article/details/50534369
[AAC]: http://blog.csdn.net/leixiaohua1020/article/details/50535042
[FLV]: http://blog.csdn.net/leixiaohua1020/article/details/50535082
[UDP-RTP]: http://blog.csdn.net/leixiaohua1020/article/details/50535230
[20160118001528487]: /images/20220717/94c90185d8b24fe7b1b2f04895c64125.png
[20160118001549018]: /images/20220717/19e9c55038974af4b7b3427698649f3d.png
[view plain]: http://blog.csdn.net/leixiaohua1020/article/details/50534369#
[CODE]: https://code.csdn.net/assets/CODE_ico.png
[CODE_CODE]: https://code.csdn.net/snippets/1570857
[ico_fork.svg]: https://code.csdn.net/assets/ico_fork.svg
[ico_fork.svg 1]: https://code.csdn.net/snippets/1570857/fork
[20160118003147122]: /images/20220717/634d079192f449729d47dc131f3122e9.png
[https_sourceforge.net_projects_simplest-mediadata-test]: https://sourceforge.net/projects/simplest-mediadata-test/
[https_github.com_leixiaohua1020_simplest_mediadata_test]: https://github.com/leixiaohua1020/simplest_mediadata_test
[http_git.oschina.net_leixiaohua1020_simplest_mediadata_test]: http://git.oschina.net/leixiaohua1020/simplest_mediadata_test
[http_download.csdn.net_detail_leixiaohua1020_9422409]: http://download.csdn.net/detail/leixiaohua1020/9422409