VC-1 Technical Overview

masonchung

VC-1 Technical Overview

1 W; V9 x; d3 E2 ?( ZAbstract
) ~1 n! G$ |, o. ^5 ?This article provides an overview of VC-1 and details its status as an established open standard. Comparative information is given to demonstrate how well VC-1 compares to two other codecs in use today, H.264 and MPEG-2.
0 j+ a9 e' L2 G9 v6 U- t1 P6 s8 e
' F6 N3 X5 i7 s" g' DJay Loomis
Mike Wasson
Microsoft Corporation
October 2007


Applies to:


   Microsoft® Windows Media® Video 9
   Microsoft VC-1 Encoder
  

Introduction
VC-1 is a video codec specification that has been standardized by the Society of Motion Picture and Television Engineers (SMPTE) and implemented by Microsoft as Microsoft Windows Media Video (WMV) 9. Formal standardization of VC-1 represents the culmination of years of technical scrutiny by over 75 companies. SMPTE 421M details the complete bit stream syntax and is accompanied by two companion documents (SMPTE RP227 and SMPTE RP228) that describe VC-1 transport and conformance. These documents provide comprehensive guidance to ensure content delivery and interoperability. Standardizing the decoder bit stream facilitates independent implementation of interoperable VC-1 encoders and decoders. This, in turn, drives innovation around software and hardware encoding solutions.
. a+ o4 X# s% f1 {% }- z% s$ {
SMPTE Standardization Background
1 @# P4 V1 v7 K7 E5 v6 U  ?0 CSMPTE is the preeminent society of film and video experts, with members in 85 countries worldwide. The standards that SMPTE produces are widely used by professionals in the fields of video, motion pictures, and digital cinema.
) k' I, `; N/ Z- @$ w4 g
4 A' M# D0 S! ~4 B+ N1 wThe SMPTE standard for VC-1, SMPTE 421M, was originally based on Windows Media Video 9 codec. The Windows Media Video 9 codec is functionally equivalent to VC-1; it is the Microsoft implementation of the VC-1 standard. VC-1 includes the Simple, Main, and Advanced Profiles that are described in the Overview of VC-1 section of this article.
  E2 |. F- v2 k; Q+ d
/ x- l0 y6 N+ u+ Q" G2 A% B$ QThe standardization process was undertaken by the SMPTE Video Compression Technology Committee, also known as C24. This committee is responsible for technologies that encode, process, switch, and decode video signals to, in, and from the compressed domain.

Microsoft chose to standardize the Windows Media Video 9 codec for a number of reasons, including accessibility and interoperability. Standardizing enables independent implementations and ensures those implementations will be interoperable. Standardizing the bitstream syntax and decoding process gives hardware manufacturers the resources and stability required to invest in creating decoders on chips in a variety of hardware devices. Isolating the video codec standard from the other parts of a complete video system enables the use of the codec on many types of hardware and in many different systems.
3 N3 z# n8 A" E6 ^. f7 b- G3 u. a6 j
' {9 B! Y, S7 x& [; hIn addition to the other reasons for standardization, having an SMPTE standard that can be referenced by other format and system specifications makes the inclusion of VC-1 easier for independent companies and discourages the adoption of different versions of the technology by different organizations. Standardization also helps gain adoption of the technology by organizations that are committed to the use of open industry standards.
+ Y( l0 u7 ~; q9 O
There are three documents produced for SMPTE to describe VC-1. The first document, SMPTE 421M, is the VC-1 specification itself. This is the main document providing comprehensive details of the VC-1 bitstream syntax and decoder semantics. The second document, SMPTE RP228, is the VC-1 conformance specification. This document describes the test procedures and criteria for determining conformance to the SMPTE 421M specification and includes reference source code and bitstreams. The third document, SMPTE RP227, is the VC-1 transport specification. The transport document provides details about carrying VC-1 elementary streams in MPEG-2 Program and Transport Streams.
0 |, p% g" \4 M- e  w
- k) N! {5 T" W9 k/ a8 F, FOverview of VC-1
: r! F/ P. j5 q5 y/ F( |The VC-1 codec is designed to achieve state-of-the-art compressed video quality at bit rates that may range from very low to very high. The codec can easily handle 1920 pixel × 1080 pixel presentation at 6 to 30 megabits per second (Mbps) for high-definition video. VC-1 is capable of higher resolutions such as 2048 pixels × 1536 pixels for digital cinema, and of a maximum bit rate of 135 Mbps. An example of very low bit rate video would be 160 pixel × 120 pixel presentation at 10 kilobits per second (Kbps) for modem applications.

The basic functionality of VC-1 involves a block-based motion compensation and spatial transform scheme similar to that used in other video compression standards since MPEG-1 and H.261. However, VC-1 includes a number of innovations and optimizations that make it distinct from the basic compression scheme, resulting in excellent quality and efficiency. VC-1 Advanced Profile is also transport and container independent. This provides even greater flexibility for device manufacturers and content services.
" Q' a" O# Z4 H! w! J! U  b$ }
* P2 @0 c7 R# J; P0 K2 a
Innovations
/ O* \) ]" I; L1 D& M, h. ^, y6 sVC-1 includes a number of innovations that enable it to produce high quality content. This section provides brief descriptions of some of these features.

- z+ Y) ^! q5 S: S& i
# m0 ^+ M7 u6 z$ ?% b; J# ZAdaptive Block Size Transform
0 `$ l% R8 ~/ I3 j, l% P0 k) hTraditionally, 8 × 8 transforms have been used for image and video coding. However, there is evidence to suggest that 4 × 4 transforms can reduce ringing artifacts at edges and discontinuities. VC-1 is capable of coding an 8 × 8 block using either an 8 × 8 transform, two 8 × 4 transforms, two 4 × 8 transforms, or four 4 × 4 transforms. This feature enables coding that takes advantage of the different transform sizes as needed for optimal image quality.
4 S( [/ X/ i, u5 e" K1 h
# S% `7 e( |" R2 q/ n' P16-Bit Transforms
In order to minimize the computational complexity of the decoder, VC-1 uses 16-bit transforms. This also has the advantage of easy implementation on the large amount of digital signal processing (DSP) hardware built with 16-bit processors. Among the constraints put on VC-1 transforms is the requirement that the 16-bit values used produce results that can fit in 16 bits. The constraints on transforms ensure that decoding is as efficient as possible on a wide range of devices.
1 j' V: r9 L5 `7 k# p
5 H: W) c! }6 N. q  BMotion Compensation
Motion compensation is the process of generating a prediction of a video frame by displacing the reference frame. Typically, the prediction is formed for a block (an 8 × 8 pixel tile) or a macroblock (a 16 × 16 pixel tile) of data. The displacement of data due to motion is defined by a motion vector, which captures the shift along both the x- and y-axes.
; `1 s; s1 ]7 T) S/ b" R& G
' O/ u( X, a0 \7 Q* n: G, PThe efficiency of the codec is affected by the size of the predicted block, the granularity of sub-pixel data that can be captured, and the type of filter used for generating sub-pixel predictors. VC-1 uses 16 × 16 blocks for prediction, with the ability to generate mixed frames of 16 × 16 and 8 × 8 blocks. The finest granularity of sub-pixel information supported by VC-1 is 1/4 pixel. Two sets of filters are used by VC-1 for motion compensation. The first is an approximate bicubic filter with four taps. The second is a bilinear filter with two taps.

VC-1 combines the motion vector settings defined by the block size, sub-pixel granularity, and filter type into modes. The result is four motion compensation modes that suit a range of different situations. This classification of settings into modes also helps compact decoder implementations.

Loop Filtering
0 X6 M- ]+ m! B4 G# h+ _$ L7 x. FVC-1 uses an in-loop deblocking filter that attempts to remove block-boundary discontinuities introduced by quantization errors in interpolated frames. These discontinuities can cause visible artifacts in the decompressed video frames and can impact the quality of the frame as a predictor for future interpolated frames.

) q0 B( U, G( I4 v7 P- J. w# f, J* }) gThe loop filter takes into account the adaptive block size transforms. The filter is also optimized to reduce the number of operations required.

# c' d( X/ G' y' p9 `! IInterlace Coding
$ z3 g4 }7 R1 f' V/ WInterlaced video content is widely used in television broadcasting. When encoding interlaced content, the VC-1 codec can take advantage of the characteristics of interlaced frames to improve compression. This is achieved by using data from both fields to predict motion compensation in interpolated frames.

" b+ O. w; O/ K- G  \. ~1 eAdvanced B Frame Coding
A bi-directional or B frame is a frame that is interpolated from data both in previous and subsequent frames. B frames are distinct from I frames (also called key frames), which are encoded without reference to other frames. B frames are also distinct from P frames, which are interpolated from previous frames only. VC-1 includes several optimizations that make B frames more efficient.
; Z1 w4 q( U  i7 ^( _
( U7 I" q" l. p, D3 HFading Compensation
Due to the nature of compression that uses motion compensation, encoding of video frames that contain fades to or from black is very inefficient. With a uniform fade, every macroblock needs adjustments to luminance. VC-1 includes fading compensation, which detects fades and uses alternate methods to adjust luminance. This feature improves compression efficiency for sequences with fading and other global illumination changes.

Differential Quantization
Differential quantization, or dquant, is an encoding method in which multiple quantization steps are used within a single frame. Rather than quantize the entire frame with a single quantization level, macroblocks are identified within the frame that might benefit from lower quantization levels and greater number of preserved AC coefficients. Such macroblocks are then encoded at lower quantization levels than the one used for the remaining macroblocks in the frame. The simplest and typically most efficient form of differential quantization involves only two quantizer levels (bi-level dquant), but VC-1 supports multiple levels, too.

Profiles and Levels
VC-1 contains a number of profile and level combinations that support the encoding of many types of video. The profile determines the codec features that are available, and thereby determines the required decoder complexity (mathematical intensity). The following table lists VC-1 profiles and levels.

Profile Level Max Bit Rate Representative Resolutions by Frame Rate
Simple
Low
2 B% c: l  D% \# M) S/ m96 Kbps
8 H, X& ~. C3 k$ @176 × 144 @ 15 Hz (QCIF)

  H! O: z- p1 X$ w" g+ FMedium
4 L& U/ I7 g1 `' k; Z384 Kbps
240 × 176 @ 30 Hz
4 f# u+ k0 S. P4 |352 × 288 @ 15 Hz (CIF)
" d) v- u" u1 v9 z: v
Main
) s. T- {& k) U  {0 G  DLow
5 Y& o; b+ F& s9 P7 j# G9 e- u) k2 Mbps
& g' f; u! }3 N. D0 @  u320 × 240 @ 24 Hz (QVGA)
! n0 q- ?; `9 [' D: b5 p  a8 p
9 [. Q! j! p# ]% R/ _6 CMedium
4 c" ~* [: u5 u: v1 I( k10 Mbps
720 × 480 @ 30 Hz (480p)
720 × 576 @ 25 Hz (576p)

3 l, C* Y% x4 V1 K. e( VHigh
# W4 P$ G* Y" d: R2 E; c' |20 Mbps
; l5 ?; X  L! X2 X7 {9 P7 y- h5 C1920 × 1080 @ 30 Hz (1080p)

: J/ d2 j- @$ j5 aAdvanced
9 ^/ `" `$ D1 p# o3 ?6 h8 a+ \! O4 fL0
5 X7 s! h6 O7 S) ~2 Mbps
352 × 288 @ 30 Hz (CIF)
! k, v# L% n* Q
L1
' z9 H) E* r9 E( i1 R10 Mbps
0 ~! C1 C3 a. ^0 K+ v720 × 480 @ 30 Hz (NTSC-SD)
( d6 |. A, L" `5 v" G720 × 576 @ 25 Hz (PAL-SD)
! \' V+ h2 b, r/ I; a# C: l3 d
L2
8 O$ T9 U% _3 |" w/ [7 N- g: Y20 Mbps
1 ^( ^1 Z/ ^" |/ P: @1 n0 `720 × 480 @ 60 Hz (480p)
1280 × 720 @ 30 Hz (720p)
5 B3 j1 V1 }3 D' X  u. `% o
- G& u; ~' T8 bL3
45 Mbps
1920 × 1080 @ 24 Hz (1080p)
1920 × 1080 @ 30 Hz (1080i)
2 {2 [7 I6 f: e( D  n! k1280 × 720 @ 60 Hz (720p)
2 F. Z, I. w( b( Z. I
L4
135 Mbps
; I% L! `1 J7 g5 A0 O1920 × 1080 @ 60 Hz (1080p)
2048 × 1536 @ 24 Hz