ESTABLISHING A CONSISTENT BASELINE REFERENCE FOR HD CAMERAS
D. P. Adams and T. L. Scott
Abstract
With the rise of high-definition television, consistent picture quality is critical if the end-user is to enjoy to the full what the system has to offer. In maximising the potential of HDTV, the service provider must give camera line-up the attention it needs. Camera technology lightens the load on the vision engineer to use those craft skills to better advantage.
Camera line-up requires more than a simple black and white balance at the correct exposure. In order to achieve the most consistent pictures across several days on a multi-camera shoot several factors need attention. Recent advances in line-up charts make this easier than with traditional versions, however there are still limitations.
This paper showcases new camera line-up technology where many former limitations are overcome. New technology allows analysis of a camera’s colorimetric performance, identifying areas of deficiency and giving the user the tools necessary to make improvements.
Finally, what does the future hold? And what can we learn from today so that future standards change for the better?
WHY IS ESTABLISHING A BASELINE REFERENCE IMPORTANT?
It is a good question; surely there is a point when a break with the past is essential. Yet we always feel the need to have a measuring standard. To provide a better level of service requires that we should know where the borderline of acceptability lies and this is where past experience steps in to help.
WHAT STANDARD ARE WE WORKING IN?
There are many flavours of high-definition, before beginning it is worth highlighting the subtle differences between the different standards.
Figure 1 – Classic Diffraction Patterns
Sampling theory and filtered pulse responses have made these “Mexican Hat” curves familiar to a new generation of engineers – the concept of the Airy Disk is even older. In Figure 1 we see the classic diffraction patterns of light mapped into three- and two-dimensional domains. The main display is of an x-y plane given amplitude response in the z-axis. And shown at upper centre is the “bird’s eye view” which is also how the viewer perceives the scene with the undistorted x-y plane set to black level.
The preference for either interlace or progressive “scan” takes into account the mutual interference of these disk patterns. What the amplitude response can be and how finely the dot patterns are pitched together is at issue. Not surprisingly 720p and 1080i closely match in spatial expectation but not necessarily in amplitude and temporal dimensions.
Amplitude response is always an issue where a sensor supports all possible basic variants. There must be an aggregate of sub-pixels that can be conformed according to the preferred display format. For example the 1080 system demands an array of 1920H x 1080V pixels and a clock speed capable of transporting these. Clearly these numbers are greater than those for the rival 720 format but nevertheless they are the governing case affecting overall design. In practice sub-pixels are grouped into pixels because 1080 is not wholly divisible by 720.
The temporal domain cannot be ignored. Generally speaking a progressive format is deemed better at handling motion judder whereas an interlaced format handles fine detail well. It is a matter of adjusting the height and width of the Mexican Hat for the least annoying interference. However the nature of speed of movement within the picture is also critical, for example a football match with many camera whip pans differs totally from a motor racing event skillfully framed in tracking shots.
From the beginning of raster line scan television, interlace was preferred to reduce bandwidth just as the double-shutter technique of cinematography reduced the required film footage. If the future is with progressive formats then the penalty lies with costly bandwidth and increasing noise. Interlace remains attractive if the motion judder is not objectionable because horizontal resolution especially is higher and the finer “line” structure perceptibly improves the vertical dimension. Visual grammar of course changes as time passes and we can therefore expect viewers to become more discriminating with HD and 3D developments. Yet handle high symbol and bit rates without resort to excessive compression are problems not easily solved.
Once the issues of format dimensions have been decided, the practicalities of how to make it work with pictures invoke past techniques as well as future technology to create a new generation of charts and procedures. Despite the subtle differences the broad principles remain the same.
WHY 60% AND NOT 90% REFLECTANCE?
Northern European skin tones are 30% reflective. If a camera is lined up with a 90% reflectance chart then skin tones will have a signal brightness of ≈400 mV. The human eye is three-times more sensitive to the darker half of a scene than the brighter half. This has the side effect of the darker-half appearing to be noisier than the brighter half. Of course, noise is constant throughout.
By setting Northern European skin tones so close to the lower half of signal brightness they will appear to have a certain amount of noise. Darker skin tones, such as Mediterranean and afro-Caribbean, will appear to show an even higher level of noise.
Favouring Skin Tones
Using a line-up chart that is 60% reflectance favours skin tones by moving them further up the gamma curve. Northern European faces will sit at ≈560 mV, where noise is perceived to be considerably less than at 400 mV. Darker skin tones will also benefit, but not to as a great degree.
Although the noise level may be of a more than acceptable level to begin with in most applications, by using every bit intelligently at the front end this will allow the end transmission path to be more cost effective in terms of compression and cost savings may potentially be made
Considering Dynamic Range
Consideration must also be made to the nature of the content being captured. By using a lower reflectance line-up chart the dynamic range is also being limited. Dramas, movies and in particular natural history documentaries, for example, may require and demand the highest dynamic range possible, however there are circumstances where maximum dynamic range may not be desirable.
Consider an early morning breakfast show or an early evening news report: a presenter and guests sat in a studio with controlled lighting. The most important visual element in the studio is the presenter’s face; naturally this is what the human eye is drawn to. It must be well lit, sufficiently bright, and stand apart from the background enough so as to be easily discernable.
By using dynamic range wisely on programming that benefits from a lower range a more comfortable experience can be created for the viewer. If the viewer is not forced to frequently change the aperture of their iris due to differences in lighting level, possibly caused by a higher dynamic range, then less strain is placed upon the eye making a programme easier to watch
THE HARLEQUIN CHART
The Harlequin chart has been specifically designed to address the requirement of 60% reflectance for certain types of programming. However the chart is also more than that. It builds on existing technology to make it an all-round tool for camera line-up with sophisticated colour analysis. The Harlequin chart also allows the discerning engineer to maximise the potential from their camera in terms of colorimetry and signal-to-noise.
No crossed greyscales
Crossed greyscales appear on many charts. They are designed to check gamma and the evenness of light upon the chart, whereas in reality they just clutter the waveform. This is real estate that can be used to provide other information, especially when gamma and lighting levels can be checked in other ways.
Camera sawtooth instead of crossed-greyscales
Charts with crossed greyscales are at best only rough tools at checking gamma, even if the chart is lit perfectly and the camera is positioned exactly.
On a multi-camera shoot this is not possible without lining up each camera individually – which is time-consuming and costly. The chart is also restricted to the gamma curve that it has been designed for. Instead camera sawtooth is key. On a multi-camera shoot, the easiest way to check gamma is to turn on sawtooth and cut or wipe between the cameras. If the sawtooth is the same on all cameras, then gamma is the same.
If verification that the correct gamma curve is being used is required then a simple preprepared transparency with the desired gamma curve on it can be placed over the waveform monitor. Or for the purists with the time and patience a spreadsheet that calculates the signal amplitude at any given sample can be used in conjunction with individual sample readings from a scope.
Uniformity of light
Horizontal uniformity
Uniformity of light can be checked with ease using the Harlequin chart. If the camera is exposed correctly using the two exposure chips in the middle of the chart, then the two large white uniformity chips either side will indicate how skewed the chart is in relation to the camera. If all four chips are at exactly the same level then the chart is lit evenly horizontally.
Vertical uniformity
The two white uniformity chips, along with the 30% reflectance mid-tone grey background, are also designed to permit checking of vertical uniformity of light on a line-up chart. If the uniformity chips and/or mid-tone grey background when viewed at line-rate on a waveform monitor show clean horizontal lines with little evidence of noise then the chart is lit uniformly from top to bottom. The thicker and noisier the horizontal lines, the poorer the vertical uniformity of the light source.
THE IMPORTANCE OF COLOUR
Line-up shouldn’t just involve checking the greyscale; we transmit in colour too! So it makes sense that we should be checking the colorimetric performance and line-up of the camera as well. Detailed checking and analysis cannot be performed during line-up, however a detailed and recognisable pattern on the vectorscope can easily highlight any discrepancies between cameras.
With a distinctive cartwheel and spokes vectorscope pattern, differences in settings between cameras like the saturation level or matrix can be clearly identified. A simple greyscale chart does not permit this, which runs the risk of faithful matching being overlooked during line-up.
Why So Many Colours?
Over 170 colours may sound like a lot but it is peanuts when you consider that millions of colours are possible. It is not possible to represent every single colour on a line-up chart otherwise no discernable information could be gathered from the vectorscope. Instead a careful map has been developed.
Cartwheel spokes
24 calibrated colours at three different saturation levels provide the spokes of the cartwheel pattern. These can be used to check that the camera produces the same hues at the correct saturation and luminance levels. If the spokes are not straight or the rings that they produce are not uniform then there are problems with reproduction.
This in itself is nothing new in terms of camera line-up chart technology. Where the differences come into play is with the extra 96 colours at the most saturated level.
Matrixing
If a CDM chart is to be trusted then the colours it displays once viewed through a correctly white-balanced camera, should appear at certain pre-determined points on a waveform monitor and vectorscope under ITU-709 recommendations. Not all cameras do this however! As the aim is to reproduce scenes both faithfully and repeatability an engineer should pay attention here before building any custom looks.
Extra 96 colours
The extra 96 colour chips at the most saturated level are there to provide the maximum amount of information possible as to what is happening to the colour space whilst a custom matrix is being built. It is easy to think that the colour space is being improved whilst looking at only six or even 24 colours, but there is much going on between each of the hues, it is all too easy to introduce errors that are not immediately apparent. Greg Foad (1) talks about this briefly in his Tech Tip on the DSC website.
96 colours is the maximum amount of colours that can be used to make a complete ring at the most saturated level whilst clearly maintaining their individual definition when viewed Figure 4 – Harlequin chart as viewed on vectorscope on a vectorscope. Any more and it becomes a continuous ring and the spacing between them is lost.
Colour chips at line-rate
Colour chips arranged horizontally are easily viewed at line-rate in RGB on a waveform monitor. Chips arranged at frame rate become difficult to analyse on a waveform monitor. The more “linear” the amplitude of the video signal is for the colour chips, then the better the colorimetric performance. The topic of matrixing using a Harlequin chart is discussed in more detail in a paper presented to the SMPTE Annual Tech Conference in 2009; Corley et al (2).
Custom Looks and Post-Production
Having built a user matrix which makes the most of the camera in terms of colorimetry and signal-to-noise, it is now possible to build a custom look. There are two ways that this can be achieved; the first being in post-production. This is preferred because you can change your look as many times as you like without degrading the original image. However, it is expensive.
The alternative is to apply a custom look to the camera. This has the disadvantage of destroying your raw image, but it is far cheaper than in post-production and allows you to see the results immediately. Of course, on live productions this is the only option.
OTHER CHART FEATURES
The Harlequin chart has many other features too. In brief these are: Skin tone patches, neutral grey background at the same reflectance as Northern European skin tones; cavity black; and large horizontal dark grey swatches for checking flares.
WHY FRONT-LIT CHARTS INSTEAD OF BACK-LIT?
It is far better to illuminate the line-up chart with the lighting rig set and lit to the house standard of lux and colour temperature. Through this you are much less likely to suffer from the effects of metamerism, especially where fluorescent and LED sources are mixed with tungsten.
Front-lit charts are also more flexible. They take up considerably less room, are easy to position and present less of a problem in standardising colour temperature to be the same as that of the scene.
WHEN IS A HARLEQUIN CHART NOT ENOUGH?
Although the Harlequin chart has many features, there are some setups that still require a separate chart. The reason is two-fold: Firstly their inclusion would clutter the Harlequin chart unnecessarily; and secondly they can be setup better on separate charts.
The Harlequin chart is able to highlight when there may be a problem with shading, however the high amount of colour information on the chart can distort results. Instead a perfectly neutral greyscale should be used.
Resolution trumpets for checking detail clutter a chart and in particular the waveform. Instead a zone plate chart should be used.
Back focus is ideally checked on a back focus chart for best results
WHAT IS IN STORE FOR THE FUTURE?
Ultimately the biggest restriction to the best possible colorimetric performance and signalto-noise of a camera lies with the chip architecture in the camera head. This poses us with a few questions: What chip architecture do we use now? What is on the horizon? And what can we learn from now to make improvements in the future?
Developments do well to heed the past. A good design needs market study, high
production standards, a product that almost sells itself and gives lengthy trouble-free
service with effective manufacturer’s back-up when needed. Broadcast cameras do not
make much profit by themselves so future designs must look for ever-cheaper
manufacturing costs and broaden their appeal to other markets, mainly domestic.
Frame Transfer and Interline Transfer
The Frame Transfer [FT] CCD was the first successful solid-state sensor. Still in production, it has a simple architecture that belies the difficulty of attaining a high yield with large-scale wafers. It relies on like-sized but separate imaging and storage regions. This places reliance on a mechanical shutter when the image is transferred into storage for “line-by-line” readout otherwise incident light pollutes the picture. This disadvantage is offset by the lack of complex control structures in the substrate of the chip itself. Thus its HD resolution and colorimetry come very close to the designer’s dream but development is at its peak.
Interline Transfer [ILT] devices were developed to rival the FT approach and avoid manufacturing conflicts of interest. No mechanical shutter is necessary and the wafer size is halved because each pixel is 50% open to light for imaging and 50% masked off for storage. This immediately halves sensitivity and worsens optical aliasing because incident spatial frequencies outstrip the capacity of the pixel clocks. To overcome these weaknesses micro lenses gather all the light that an FT pixel would do in similar circumstances and greater care is given to the optical low-pass filters that keep annoying spatial frequencies within bounds. Initially the designs were prone to highlight overload, especially with infrared penetrating deep into the chip causing photoelectrons to “puncture” a vertical shift register. But the problem, once understood could be overcome by design, adding however to the complexity of manufacture. With HDTV the advantages of ILT begin to be outweighed by the penalties caused by its very architecture. Like FT, but for different reasons, ILT has reached its peak of economical development.
C-MOS
C-MOS technology is attractive first of all because its chip architecture is already in mass production for almost every type of digital chip. Manufacturing synergies and cost advantages are obvious. However image sensing is not a matter of clocking “ones” and “zeroes”. It is instead an analogue domain in which light sensitivity and colorimetry must be faithfully mapped from photons into electrons. At first sight this is only a matter of “scraping the black paint” off the top of each would-be photosite and allowing quantum physics to have its say. With some ten million or more phototransistors, gain variations must be overcome by negative feedback to reduce patterning noise. Reset jitter is going to be an issue because even the DC operating voltage is analogue and prone to noise.
Correlated double sampling can be adapted to deal with each photosite but this increases the transistor count. Viewing the picture it is unfortunate that the “slantidicular” verticals will return with movement because modern displays do not sit happily with camera line-array scanning. So ingenuity must be brought into play to minimise or remove such pendulumlike effects on verticals with motion or panning. Also the headroom of a C-MOS sensor with its fixed voltage rails is no match for a CCD that can sink potential wells to almost any necessary depth. Nor is its transfer mechanism as noiseless as a tunnel diode. Suddenly the contest for supremacy is not yet clear-cut but the smart money has to be on C-MOS in the long run.
What Can We Learn From Now For The Future?
Development over the years has passed from red-insensitive tube cameras to blueinsensitive solid-state devices. A tube used necessarily thin targets to accommodate an intense electric field constraining the mutually repulsive photoelectrons in “picture registration”. Target thinness was responsible for the red photons, which are the least energetic, to pass undetected straight through. Solid-state technology responded initially with deep CCD potential wells to gather the photoelectrons, which is rather inelegant but is remarkably tolerant. However the latest sensors tend to have a higher density of substrate circuitry than the initial offerings, especially so with ILT. These more highly doped regions trap the more energetic blue photons because one or other of the photo-carriers released in a photon-atom collision is relatively immobile. Undetected recombination inevitably reduces blue sensitivity and this is readily seen on the Harlequin chart where shortcomings are ruthlessly exposed. All solid-state sensors suffer this “blue deficiency” to a degree but ILT tends to suffer rather more than CCD. Thus whatever the sensor of the future may be, perhaps it is time to revisit the defining colorimetry equation and its coefficients so as to better match the forthcoming generation of sensors.
CONCLUSIONS
The Harlequin chart is designed to favour skin tones in scene composition by using the “less is more” adage in terms of dynamic range. Sophisticated analysis can be easily achieved of the camera’s colorimetric performance. Identification of areas of deficiency is possible and gives the user the tools necessary to make improvements.
A lot has been learnt over the past decade as high-definition has ascended. As we move forward and 3D services begin to launch, now is an ideal time to review our findings and establish exactly how we can continue to improve. With camera technology rapidly moving towards C-MOS architecture a colour difference equation that is formulated specifically for it is recommended.
REFERENCES
- Foad, G. 2007. Are Six Colors Enough? DSC Tech Tips – dsclabs.com/tech_tips.htm. September, 2007.
- Corley, D. C., Scott, T. L and Adams, D. P. 2009. Selecting Production Parameters to Ensure that Picture Quality Accommodates the Intended and Possible Future Imaging Systems. Proceedings of the SMPTE Annual Tech Conference. October, 2009. pp. 8 to 11, slides 27-36.
ACKNOWLEDGMENTS
The authors would like to thank David Corley of DSC Labs for his contributions to the development of this project and his continued support.
They would also like to thank Rien van Trotsenburg of Grass Valley Nederland B.V. and Nigel Arnott of Grass Valley (UK) Ltd for their contributions and involvement.
This paper is reproduced with kind permission of IBC.