Lights, camera... action! On the eve of the Oscars, we take a look at an ERC project in the field of film. ERC grantee Marcelo Bertalmío loves cinema so much that he made it the core of his research. A filmmaker himself – he has directed two movies – Dr. Bertalmío is developing a series of image processing algorithms that will create a better and cheaper way to shoot movies, whilst granting more artistic freedom to directors and cinematographers. He is the author of a book, Image Processing for Cinema (2014), which has already received plaudits from the film industry.
Lighting is an essential part of any film. Directors use it for artistic purposes – as in Amelie's soft, cream colours – but it also has a more prosaic function: allowing the camera to "see" the scenario and thus to register it. This may seem obvious, but it is not: cameras and humans process light in very different ways. One of those differences is the camera's dynamic range.
Dynamic range is the ratio between the darkest and the brightest values that a camera can capture. For most cameras – even the ones used in cinema – this range is lower than what our eyes are capable of seeing. To compensate for this difference, the film-set has to be artificially illuminated to enable the camera to record in a way that seems "natural" to us.
“If it is a sunny day and you are in a dark room with a window, you will see the bright world outside whilst still perceiving the shadowy objects in the room. Cameras will see only one or the other. So for them to register what's outside the window, the interior of the room has to be over-illuminated in order to reduce the contrast from bright to dark", explains Marcelo Bertalmío. This illumination process is often expensive and time consuming.
Imitating the human eye
It is not that cameras lack quality – in some respects, they are even better than our own eyes. It is just that we still do some things better than they do. What our eyes register is less distinct than a camera, but we process these images better thanks to the workings of our retina and visual cortex. Dr. Bertalmío's project attempts to mimic these neural processes in order to get better quality, more accurate images on camera.
Dr. Bertalmío is able to investigate these possibilities with the help of an ERC Starting Grant. He has put together a multi-disciplinary team made up of a mathematician, a psychologist with expertise in visual perception and a colour expert. “The ERC Grant really changed my life. I can hire my own team and also involve PhD students. We now have the resources to buy equipment, go to other research centres to establish collaborations and organise and attend conferences and events”, he acknowledges. He is now in the process of hiring a neurologist. This kind of expertise will support the project's research into the imitation of our neural processes by building software algorithms that can be applied to the images captured with a regular camera.
“Using these innovative methods, directors and cinematographers will be able to concentrate on the artistic lighting. Movies could be made in less time with less equipment and greater artistic freedom, because there would be more time and resources to dedicate to artistic lighting and they would be sure that what they see during the shooting is what they're actually getting. This research will also have a great impact on amateur videos, the majority of which are being shot without any extra lighting”.
Understanding our neuro-visual processes
The research will not only change the way films are made. It will also help us to understand the way our brain processes information about colour: “The way we perceive colour is far from being completely understood. We are able to adapt to light intensity so we still perceive the same colours no matter what kind of light we have –with either a candle or a fluorescent bulb a white shirt will always look white to us. But colour also depends on surroundings, the way other objects in the scene reflect light and many other things we don´t really understand”.
“When filming, we try to get similar results to those obtained by the human eye without knowing what's really going on in our brain. But if we succeed, we may help neuroscience to figure out these processes, in the same way developments in photography helped us to understand more about colour perception”.