Color Spaces in Frame Grabbers: RGB vs. YUV
A frame grabber works with streaming video sources to capture individual frames. A quality frame grabber supports at least NTSC and PAL systems. However, the native stream or input to the frame grabber does not need to be analog. Newer frame grabbers also support HD video streams such as 1080i, 1080p or 720p. A capture board has hardware to retrieve individual frames and forward them to a capture source such as a PC or SBC. There is some debate over which format is better for frame grabbers: YUV or RGB. The answer depends on the application.

2246_photo.jpg

RGB formats are usually straightforward: red, green, and blue with a given pixel size. RGB24 is the most common, allowing 8 bits and a value of 0-255 per color component. RGB frame grabbing is best if your image processing and/or storage requirements call for it. RGB capture can be saved directly as a bitmap by adding a simple header. Windows XP GDI (Graphic Display Interface) and Linux GUIs work with and display RGB bitmaps.

The problem with RGB, however, is that it is not the best mapping for representing visual perception. YUV color-spaces are a more efficient coding and reduce the bandwidth more than RGB capture can. Most video cards, therefore, render directly using YUV or luminance/chrominance images. The most important component for YUV capture is always the luminance, or Y component. For this reason, Y should have the highest sampling rate, or the same rate as the other components. Generally, 4:4:4 YUV format (equal sample per component) is a waste of bandwidth because the chrominance can be adequately sampled at half the sample rate of the luminance without the eye being able to notice the difference. YUV 4:2:2 and 4:2:0 formats are generally used with the preference on 4:2:2. 4:2:2 means the chrominance components are sampled horizontally at half the rate of the luminance. 4:2:0 means the Cb (blue chrominance) and Cr (red chrominance) components are sub-sampled at a factor of 2 in the vertical horizontal directions.

Capturing to YUV (or YCrCb), besides being more efficient, is best when the images are to be rendered by a standard video display directly. Additionally, some image compression algorithms, such as JPEG, directly support YUV, so there is no need for RGB conversion. In the YCrCb model, 0x80 is the black level for Cr and Cb, and 0x10 is the black level for Y.

Most DirectX 8, 9, and 10 hardware supports direct display of YUV images. With the correct format, you can render directly to the video card without transforming the image with MMX instructions or other (preferably optimized) code.

The most common video cards support packed YUV mode, YUY2 (4:2:2) and/or NV12 (4:2:0) natively. The byte ordering for YUY2 is as follows:

Y0 U0 Y1 V0 Y2 U2 Y3 V2 Y4 U4 Y5 V4

YUY2 format (4:2:2 = width * height Y values, width * height / 2 Cr,Cb values = 2 * W * H total bytes)

NV12 is a quasi-planar format and has all the Y components first in the memory, followed by an array of UV packed components. For a 640x480 NV12 image, the layout is as follows:

Y0Y1Y3   
         
         
   Y301798Y301799
U0V0U1V1   
U76799V76799

NV12 format( 640 * 480 Y values, 640 * 480 / 4 Cr and Cb values = 3/2 H * W total )

Many hardware capture boards are most efficient using planar YUV mode. They capture YUV on a plane-by-plane basis. Sensoray's 2255, for example, can capture directly into 4:2:2 YUV422P format. (It also supports YUY2 and RGB formats in the SDK.) The planar format is advantageous when the final YUV format is not known, because each component can be directly referenced as an array. A separate pointer can simply point to each of the component arrays. The packed formats do not allow this. Additionally, from the planar format, it is trivial to create the packed YUV formats such as NV12 or YUY2 with MMX optimized instructions for DirectX VA (DirectX Video Accelerated) or SDL display in Windows or Linux respectively.

Y1Y3    
         
          
   Y301798Y301799
U0U1U2U3 
     U76798U76799
V0V1V2V3
V76798V76799

YUV planar (4:2:2), 640 x 480 image

SENSORAY | 7313 SW Tech Center Dr. | Tigard, OR 97223 | 503.684.8005 | Email Us

CONTACT SENSORAY

Employment | Privacy Policy | Press Releases | Copyright © 1982-2024 Sensoray ~ All Rights Reserved