Imagine having all the capability of a multi-thousand dollar DSLR camera in your pocket — maybe even as part of your smartphone. That’s the promise of startup Light’s innovative array camera architecture. Through the near-magic of computational imaging and a clever combination of optical, sensor, and mechanical design, Light believes it can put an 52MP-equivalent camera in a device about as thick as a cased smartphone by next year — with low noise, high quality image output, and even zoom capability. When we first covered their announcement two weeks ago, it was hard to sift through the possibilities. Since then, I’ve spent a morning with the founding team, and especially with CTO Rajiv Laoia, and am able to provide a lot deeper dive into what Light is planning, and why it thinks it can succeed where others haven’t.
Light’s camera module: An overview
Light is planning to use an array of small cameras — very similar to those you would find in a smartphone today — to mimic a larger camera. By precisely aligning and calibrating the individual cameras, the images from them can be combined to produce a single image of very high quality. By having some of the cameras feature a wide-angle lens, and some a telephoto lens, an optical zoom capability is also possible. By placing the telephoto lenses on their side, the module can stay nearly as thin as a typical smartphone camera module. Finally, by having actuator-driven mirrors on the telephoto lenses they can be directed to different portions of the wide-angle frame to enhance the zoom capability at intermediate focal lengths. All of this relies on a lot of serious after-the-fact computation, but since the design of the module includes traditional camera sensors, a typical-quality image is available instantly. Now that we have that out of the way, let’s take a deeper look at the technology.
Overcoming the impossible: Low noise with small sensors
If you ask nearly anyone involved with photography, they’ll explain that the big advantage of large-sensor cameras is big pixels, that in turn provide lower noise-images. That’s true because each pixel (or more correctly, photosite) is essentially a well that captures photons during a camera exposure. Due to the nature of light, not every photon goes exactly where you think it should, so a certain number wind up in the wrong well. The larger the wells, the better the ratio between photons that are where they belong (signal) and those that have strayed (noise). Smartphone cameras have almost impossibly small pixel sizes barely over 1 micron on a side. By comparison, a flagship full-frame DSLR might have pixels as large as 6 or 8 microns on a side — giving it as much as 50 times the light-gathering capacity per pixel.
From the back, a Light module is a jumble of flexible leads coming from many small cameras mounted onto a rigid chassis
Light’s CTO, Rajiv Laroia, explains that by treating the ten or more small sensors as pieces of a single, large sensor, Light’s camera module can add together all the photons that reach a particular location in each of the sensors to mimic larger photosites. By adding the signals, the ratio of signal to noise is greatly increased. As with many of Light’s solutions to difficult issues, this strategy makes perfect sense, but until Light shows that it can get sufficiently accurate alignment in a production module, and that other factors in its design don’t increase noise, we’ll have to wait and see.
The secret to Light’s array camera
Light is not the first company to attempt to replace smartphone cameras with an array of small sensors. Most notably, Pelican Imaging has been working for several years on array-based camera modules. Market acceptance has been delayed because of the processing power required to create the final image from its multiple monochrome sensors. That same issue also hampered market acceptance of alternative camera architecture innovators Foveon and Lytro. What makes Light’s architecture different, and gives them a potentially massive advantage, is that most or all of the small cameras it uses each take a perfectly usable conventional image. That means that the LCD preview can easily be shown in real time with no additional processing, and that Web-quality images and video are instantly accessible. Creation of high-quality images can be postponed until time and wall power are available, or even delegated to the cloud.
Like Pelican, Light is enthusiastic about the creative possibilities for combing different types of sensors to address particular imaging problems. Combining some “white” sensors for low-light sensitivity with color sensors is one possibility, as is having different camera use different exposures for improved high-dynamic-range imaging. Each of these approaches reduces the number of conventional images available, though, so I expect Light’s initial module designs to be fairly conservative, saving the experimentation for later.
A clever way to mimic a zoom lens
Light has also come up with a very clever way to mimic zoom capability using an array of fixed-focal-length (prime) lenses. Its arrays feature both wide-angle and telephoto lenses (it’s planning to have 35mm and 70mm lenses in its first smartphone-sized module) in an overlapping configuration. The telephoto lenses are mounted sideways in the module, to keep thickness to a minimum and allow for a small mirror that can be used to aim the telephoto lenses at different portions of the wide-angle view.
Light uses overlapping fields of view to emulate a single-lens optical zoom. Mirrors move the multiple 70mm lenses to create an overlapping field of view to provide effective focal lengths between 35mm and 70mm. The 35mm lenses provide a wide field of view image to assist in aligning the telephoto images.
So, for example, to take a 70mm image, all four telephoto lenses would be aimed at the center of the wide-angle image, providing four aligned images at 70mm. For an intermediate focal length, like 50mm, the mirrors would move the telephoto lenses about half way “out” towards the edge of the wide-angle field of view — like I’ve drawn somewhat crudely in the diagram above. That configuration yields a hybrid image, with all areas featuring some combination of images from the telephoto and the wide-angle lenses. The wide-angle images help provide a background for aligning the telephoto images. Not surprisingly, Light isn’t the only company aiming to bring zoom to smartphones. Another startup, Dynaoptics, plans to add the capability by laying a tiny zoom lens on its side, and changing the mechanism needed for zooming so that it is small enough to live within the thickness of a smartphone.
Here too, it is obvious Light has done its homework and come up with a clever solution. However, in practice Light will have to overcome technical issues that could make perfect alignment difficult. For example, smartphone imagers almost all use rolling shutters (where the entire image is not exposed at the same time). Merging multiple images taken with rolling shutters provides additional challenges. Also, each smartphone lens has slightly different physical properties, meaning that each one may have slightly different color shading defects. That can be adjusted for, but typically images aren’t laid end to end with each other and merged the way Light is planning, so Light’s application will push the alignment and calibration technology to its limits.
DARPA did one of these, but it cost a lot more
ARGUS-IS, surveying Quantico, Virginia. Click to see a much larger version.
The idea of combining a number of smartphone cameras to make a super camera isn’t new. DARPA constructed a surveillance camera from 368 5MP smartphone cameras that can cover a 4-mile area while at the same time being able to pick out objects as small as a bird in flight. Its ARGUS-IS, with a total resolution of 1.8 gigapixels, is designed to be carried under a helicopter. Once Light’s modules are in the market, it’ll be interesting to see what a similar large-scale project constructed out of its modules could do. Laoia explained to me that one reason solutions like this make sense is that the small, plastic lenses used in mobile devices can be molded into nearly arbitrary shapes, so it is possible to create a low-distortion lens with many less elements than can be done with the large glass components of a DSLR lens. That certainly goes counter to the conventional wisdom that smartphone lenses are inferior to their larger cousins, but my colleague and camera-array guru Harlyn Baker confirmed that they don’t have to be.
So what if I drop my phone?
As makers of 3D cameras have learned, precise alignment of the multiple imagers required is essential to creating a usable image. Light simplifies that issue a little by only using prime lenses, so it doesn’t have to keep zooms in sync, but it does have quite a few modules to keep under control. From the factory, Light modules will come calibrated, using a fairly standard checkerboard-pattern alignment process. The modules are designed to detect when they need re-calibration, so that the user can be guided through the process. It isn’t any more complex than adjusting the micro-focus adjustment is on a high-end DSLR now, so for serious shooters it will be a small price to pay. To minimize the need for re-calibration, Light mounts its cameras on a stiff metal plate.
Is it a superphone or a super camera?
Light is clearly targeting the high-end smartphone market with its product strategy. It has announced that manufacturing giant Foxconn will be using its camera module in some of its high-end smartphones. The team makes a compelling case that smartphone companies are running out of ways to differentiate on the high-end, so adding $60-$80 to the cost of a phone (which means about $150-$200 to the price) is entirely plausible. Qualcomm Executive Chairman Paul Jacobs is a board member and investor, so the company clearly has some supporters in the right places in the industry supply chain to help make adoption of its camera in smartphones a reality.
As a photographer, though, I’m actually looking to see if non-phone vendors start to get interested. The point-and-shoot market continues to tank, and the DSLR and mirrorless markets certainly aren’t doing all that well. But they still represent over 100 million cameras sold every year. What if someone built a camera that sold for $400 and could replace a $2,000 DSLR and $1,000 lens, but fit in your pocket? It could have real camera controls (something smartphones don’t), jack for a remote mic, a nice flash, and an Electronic Viewfinder. Light even has a slightly-larger prototype that can zoom to 150mm, enough for most photography needs. Recreating the depth-of-field effects and bokeh that come with larger sensors will be a challenge, but Light claims to have ways to address that as well. In other words, it could be an amazing camera for serious photographers, not just a feature on a high-end phone. Either way, I’m looking forward to getting a Light-enabled device to shoot with when they start shipping modules in 2016.
Source Article from http://www.extremetech.com/extreme/204427-how-light-plans-to-put-a-dslr-in-your-pocket http://www.extremetech.com/wp-content/uploads/2015/04/Light-348x196.jpg
How Light plans to put a DSLR in your pocket
No comments:
Post a Comment