FLV文件中VideoPacket的详解

╰+哭是因爲堅強的太久メ 2022-02-03 15:04 220阅读 0赞

众所周知，FLV文件体主要有三种Tag组成：AUDIODATA、VIDEODATA和SCRIPTDATA，其中SCRIPTDATA中包含了MetaData，通过解析这些元数据，可以得到解码器的初始化信息，如音频解码元数据audiocodecid、audiodatarate、audiodelay、audiosamplerate、audiosamplesize、stereo，视频解码元数据videocodecid、framerate、height、width、videodatarate等信息。这些信息里面有的对于初始化解码器非常有用，例如解码器必须知道codecid，才能进行初始化从操作，视频解码器初始化时有的需要知道图像宽高信息，才能对解码重建Buffer进行初始化。

某些FLV文件，在SCRIPTDATA的MetaData中并未给出相应信息，这样就需要对AUDIODATA、VIDEODATA进行解析。Adobe Flash Video File Format Specification中只是简单交代了AVCVIDEOPACKET的格式。对其他的codecID对应的VideoPacket并未详细介绍。本文专门针对视频解码初始化过程中需要的width和height信息的深入解析为例，详细介绍一些VideoPacket。因为FLV的VideoPacket的格式和SWF文件离得VideoPacket格式基本一致，也可以参考SWF File Format Specification。详见http://www.adobe.com/go/swf\_file\_format。

1、VideoTagBody

IF FrameType == 5

UI8

ELSE (

IF CodecID == 2

H263VIDEOPACKET //Sorensen H263

IF CodecID == 3

SCREENVIDEOPACKET //Screen

IF CodecID == 4

VP6FLVVIDEOPACKET //On2 Vp6

IF CodecID == 5

VP6FLVALPHAVIDEOPACKET //On2 VP6 alpha

IF CodecID == 6

SCREENV2VIDEOPACKET //Screen V2

IF CodecID == 7

AVCVIDEOPACKET //AVC

)

2、H263VIDEOPACKET

这里的H263是Sorenson H263，对ITU-T的H.263进行了订制，所以在picture层和MB层的header的格式发生了改动，且GOB层被删除。其具体格式为：

表格 1 H263VIDEOPACKET

Field

Type

Comment

PictureStartCode

UB\[17\]

Similar to H.263 5.1.1  
0000 0000 0000 0000 1

Version

UB\[5\]

Video format version  
Flash Player 6 supports 0 and 1

TemporalReference

UB\[8\]

See H.263 5.1.2

PictureSize

UB\[3\]

000: custom, 1 byte  
001: custom, 2 bytes  
010: CIF (352x288)  
011: QCIF (176x144)  
100: SQCIF (128x96)  
101: 320x240  
110: 160x120  
111: reserved

CustomWidth

If PictureSize = 000, UB\[8\]  
If PictureSize = 001, UB\[16\]  
Otherwise absent  
Note: UB\[16\] is not the same  
as UI16; there is no byte  
swapping.

Width in pixels

CustomHeight

If PictureSize = 000, UB\[8\]  
If PictureSize = 001, UB\[16\]  
Otherwise absent  
Note: UB\[16\] is not the same  
as UI16; there is no byte  
swapping.

Height in pixels

PictureType

UB\[2\]

00: intra frame  
01: inter frame  
10: disposable inter frame  
11: reserved

DeblockingFlag

UB\[1\]

Requests use of deblocking  
filter (advisory only, Flash  
Player may ignore)

Quantizer

UB\[5\]

See H.263 5.1.4

ExtraInformationFlag

UB\[1\]

See H.263 5.1.9

ExtraInformation

If ExtraInformationFlag = 1,  
UB\[8\]  
Otherwise absent

See H.263 5.1.10

...

The ExtraInformationFlag-  
ExtraInformation sequence  
repeats until an  
ExtraInformationFlag of 0 is  
encountered

Macroblock

MACROBLOCK

See following

PictureStuffing

varies

See H.263 5.1.13

从上表可以得出，不用深入到macroblock层，只需解析头部，就可很简单的得到width和height信息。

2、SCREENVIDEOPACKET和SCREENV2VIDEOPACKET

Screen Video是一个简单的无损压缩视频格式，它利用帧间编码传递位图，其pixel\_froamt是BGR24，专用于电脑截屏视频传输。像素数据使用了Zlib开放标准进行了压缩。其具体格式为：

表格 2 SCREENVIDEOPACKET

Field

Type

Comment

BlockWidth

UB\[4\]

Pixel width of each block in the grid. This value is stored as (actualWidth / 16) - 1, so possible block sizes are a multiple of 16 and not more than 256.

ImageWidth

UB\[12\]

Pixel width of the full image.

BlockHeight

UB\[4\]

Pixel height of each block in the grid. This value is stored  
as (actualHeight / 16) - 1, so possible block sizes are a  
multiple of 16 and not more than 256.

ImageHeight

UB\[12\]

Pixel height of the full image.

ImageBlocks

IMAGEBLOCK\[n\]

Blocks of image data. See preceding for details of how  
to calculate n. Blocks are ordered from bottom left to  
top right, in rows.

表格 3 SCREENV2VIDEOPACKET

Field

Type

Comment

BlockWidth

UB\[4\]

Pixel width of each block in the grid. This value is stored as (actualWidth / 16) - 1, so possible block sizes are a multiple of 16 and not more than 256.

ImageWidth

UB\[12\]

Pixel width of the full image.

BlockHeight

UB\[4\]

Pixel height of each block in the grid. This value is stored as (actualHeight / 16) - 1, so possible block sizes are a multiple of 16 and not more than 256.

ImageHeight

UB\[12\]

Pixel height of the full image.

Reserved

UB\[6\]

Must be 0

HasIFrameImage

UB\[1\]

If 1, has IFrameImage

HasPaletteInfo

UB\[1\]

If 1, has PaletteInfo

PaletteInfo

If HasPaletteInfo,  
IMAGEBLOCK

One block of data to describe the palette.

ImageBlocks

IMAGEBLOCKV2\[n\]

Blocks of image data. See Block format for details of how to calculate n. Blocks are ordered from bottom left to top right in rows and can be a combination of keyblocks and interblocks.

IFrameImage

If HasIFrameImage,  
IMAGEBLOCKV2\[n\]

Blocks of image data representing interblocks that must be combined with the previous keyblocks to produce the image. See Block format for details of how to calculate n. Blocks are ordered from bottom left to top right in rows.

从上表可以得出，不用深入到macroblock层，只需解析头部，就可很简单的得到width和height信息。

3、VP6FLVVIDEOPACKET和VP6FLVALPHAVIDEOPACKET

VP6FLVVIDEOPACKET和VP6FLVALPHAVIDEOPACKET比较简单，只是简单的将VP6或VP6A的一个视频帧封装到VideoPacket即可。

表格 4 VP6FLVVIDEOPACKET

Field

Type

Comment

HorizontalAdjustment

UB\[4\]

Number of pixels to subtract from the total width. The resulting width is used on the stage, and the rightmost pixels of the video is cropped.

VerticalAdjustment

UB\[4\]

Number of pixels to subtract from the total height. The resulting height is used on the stage, and the rightmost pixels of the video is cropped.

Data

UI8\[n\]

Raw VP6 video stream data

表格 5 VP6FLVALPHAVIDEOPACKET

Field

Type

Comment

HorizontalAdjustment

UB\[4\]

Number of pixels to subtract from the total width. The resulting width is used on the stage, and the rightmost pixels of the video is cropped.

VerticalAdjustment

UB\[4\]

Number of pixels to subtract from the total height. The resulting height is used on the stage, and the rightmost pixels of the video is cropped.

OffsetToAlpha

UI\[24\]

Offset in bytes to the alpha channel video data.

Data

UI8\[n\]

Raw VP6 video stream data.

AlphaData

UI8\[n\]

Raw VP6 video stream data representing the alpha channel

这样如果MetaData中没有宽高信息的话，就需要深入到Data层进行header parse，解析得到宽高信息，过程比较麻烦，不过可以参考FFmpeg的avcodec中的vp6\_parse\_header()函数，亦或是参考VP6或VP6Alpha的codec文档吧，本文暂不做详解。

4、AVCVIDEOPACKET

FLV将AVC的SPS(序列参数集)和PPS(图像参数集)和其他的配置信息封装到AVCDecoderConfigurationRecord，其具体格式为：

AVCDecoderConfigurationRecord \{  
unsigned int(8) ConfigurationVersion = 1;  
unsigned int(8) AVCProfileIndication;  
unsigned int(8) ProfileCompatibility;  
unsigned int(8) AVCLevelIndication;  
bit(6) reserved = ‘111111’b;  
unsigned int(2) LengthSizeMinusOne;  
bit(3) reserved = ‘111’b;  
unsigned int(5) numOfSequenceParameterSets;  
for (i=0; i< numOfSequenceParameterSets; i++) \{  
unsigned int(16) sequenceParameterSetLength ;  
bit(8\*sequenceParameterSetLength) sequenceParameterSetNALUnit; //SPS  
\}  
unsigned int(8) numOfPictureParameterSets;  
for (i=0; i< numOfPictureParameterSets; i++) \{  
unsigned int(16) pictureParameterSetLength;  
bit(8\*pictureParameterSetLength) pictureParameterSetNALUnit; //PPS  
\}  
\}

图像的宽高信息在SPS中，所以需要得到宽高信息，需要解析H.264的SPS，详见H.264的文档，当然也可以参考FFmpeg的avcodec中的ff\_h264\_decode\_seq\_parameter\_set()函数。

5、一点点个人想法

流媒体播放中，多媒体音视频包经过网络到达终端，脱去一层层网络打包外衣，而后容器的外衣、再次是codec的header，经过艰辛的历程到达解码器。最终的目的就是节省网络带宽。随着互联网免费服务潮的到来，那些效果不错，但是收费高昂的标准或技术，遇到了很大的挑战。Google主推的VP8，较之ITU-T的H264，虽在编码效果上稍微逊色，但是其最大的杀手锏——免费，抓住了H264专利费高昂的七寸。Google强大的号召力或许真的会使VPx系列的codec最后胜出。一般公司玩技术、牛公司玩标准、超牛公司玩潮流，不得不佩服美帝的技术帝国。何种这些风潮才能Made in China?  
\---------------------  
作者：xietao\_live\_cn  
来源：CSDN  
原文：https://blog.csdn.net/xietao\_live\_cn/article/details/6539625  
版权声明：本文为博主原创文章，转载请附上博文链接！