Color Spaces in Frame Grabbers: RGB vs. YUV |
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
A Frame Grabber works with streaming video sources to capture individual frames. A quality frame grabber supports at least NTSC and PAL systems. However, the native stream or input to the frame grabber does not need to be analog. Newer frame grabbers also support HD video streams such as 1080i, 1080p or 720p. A capture board has hardware to retrieve individual frames and forward them to a capture source such as a PC or SBC.
There is some debate on what format is better for frame grabbers. YUV or RGB. The answer is that it depends on the application. |
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
RGB formats are usually straightforward, Red, Green, Blue with a given pixel size. RGB24 is the most common with 8bits per color component with an allowed value of 0-255. RGB frame grabbing is best if your image processing and/or storage requirements require it. RGB capture can be saved directly as a bitmap by adding a simple header. Windows XP GDI (Graphic Display Interface) and Linux GUIs work with and display RGB bitmaps.
The problem with RGB, however, is it is not the best mapping to represent visual perception. YUV color-spaces are a more efficient coding and reduce the bandwidth compared to RGB capture. Most video cards, therefore, render directly using YUV or Luminance/Chrominance images. The most important component for YUV capture is always the Luminance or Y component. For this reason, Y will have the highest (or same) sampling rate. Generally, 4:4:4 YUV format (equal sample per component) is a waste of bandwidth because the Chrominance can be adequately sampled at ½ the sample rate of the Luminance without the eye being able to notice the difference. YUV 4:2:2 and 4:2:0 formats are generally used with the preference on 4:2:2. 4:2:2 means the chrominance components are sampled horizontally at half the rate of the luminance. 4:2:0 means the Cb (blue chrominance) and Cr (red chrominance) components are sub-sampled at a factor of 2 in the vertical horizontal directions.
Capturing to YUV (or YCrCb) besides being more efficient is best when the images are to be rendered by a standard video display directly. Additionally, some image compression algorithms, such as JPEG directly support YUV so there is no need for RGB conversion. In the YCrCb model, 0x80 is the black level for Cr, Cb and 0x10 is the black level for Y.
Most all DirectX 8, 9, 10 hardware supports direct display of YUV images. With the correct format, you can render directly to the video card without transforming the image with MMX instructions or other (preferably optimized) code.
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
The most common video cards support packed YUV mode, YUY2 (4:2:2) and/or NV12 (4:2:0) natively. The byte ordering for YUY2 is as follows:
YUY2 format(4:2:2 = width*height Y values, width*height /2 Cr,Cb values = 2*W*H total bytes)
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
NV12 is a quasi-planar format and has all the Y components first in memory followed by at array of UV packed components. For a 640x480 NV12 image, the layout is as follows:
NV12 format( 640*480 Y values, 640 *480 /4 Cr and Cb values = 3/2 H*W total )
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Many hardware capture boards are most efficient using planar YUV mode. They capture YUV on a plane-by-plane basis. Sensorays 2255 for example can capture directly into 4:2:2 YUV422P format (It also supports YUY2 and RGB formats in the SDK). The planar format is advantageous when the final YUV format is not known because each component can be directly referenced as an array. A separate pointer can simply point to each of the component arrays. The packed formats do not allow this. Additionally, from the planar format, it is trivial to create the packed YUV formats such as NV12 or YUY2 with MMX optimized instructions for DirectX Video Accelerated (DirectX VA) or SDL display in Windows or Linux respectively.
YUV planar (4:2:2), 640x480 image
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
By Dean Anderson |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
