Coded Aperture and Coded Mask
secondary research into the topic plenoptic and simple depth sensors - #2
Unlike microlenses which depend on a precisely manufactured array of microlenses which are an expensive piece of kit, coded aperture relays on a piece of thin plastic with holes cut in it.
A coded aperture or mask is the simplest and cheapest way of creating a light field camera by retrofitting a digital camera with a cutout mask. This technique is mostly used in radioactive materials for accurate localization as there are not that many ways of focusing high-powered radiation. This technique is a staple in x-ray and medical gamma-ray imaging for medical applications1. Coded aperture can also be used in lensless photography because each of the cutouts acts like a pinhole camera and produces overlapping images on the sensor.
If there are N pinholes in the aperture, the picture consists of N overlapping images of the object. The coded aperture technique (for a point source) can improve the signal-to-noise ratio by roughly √N when compared to the single pinhole camera.
If ease of construction is a big plus but the downside is that amount of light reaching the sensor is drastically reduced which can be mitigated to a point with the use of different style masks.
The recorded picture is not recognizable as the object because the many pinholes cause the picture to consist of many overlapping pictures. In order to be useful, the picture must be subjected to a reconstruction procedure that will compensate for the effects of the imaging system. The reconstruction procedure is designed to give the location and intensity of each source in the field of view. This is accomplished by detecting the location and strength of aperture patterns in the picture.
During the century of research different masks were developed with specific applications where they excel. There is also a mask that allows imaging without any lenses mostly for use in high power radiation imaging but its gaining traction in visual light capture as it reduced complexity with regards to lenses needed, makes the whole camera thinner, more compact, increases robustness, and lowers cost.
Mask can be created by cutting the stencil out of cardboard, by 3D printing or by printing on transparencies.
Different results can be achieved with the placement of the mask. If the mask is placed near the lens aperture it allows acquisition of full depth of field at full resolution and is called encoded blur mask. If the mask is placed near the sensor it captures a 4D light field without the need for microlenses and is called a Heterodyne light field mask. A term usually found in radio signal processing.
With the Heterodyne light field mask, we can treat the mask as a known signal noise which we can plug into a demodulating function to filter out the mask and get our image.
Heterodyne light field mask, depending on the style of the mask, creates spectral images or tiles. The spectral tiles of the light field are created by placing a high-frequency sinusoidal pattern between the sensor and the lens of the camera.
To recover the 4D light field, we take the Fourier transform of the 2D sensed signal, reassemble the 2D tiles into a 4D stack of planes, and take the inverse Fourier to transform. Unlike previous 4D cameras that rely on lens arrays, this hybrid imaging / light field design does not force resolution tradeoffs for in-focus parts of the scene. The mask does not bend rays as they travel from scene to sensor, but only attenuates them in a fine, shadow-like pattern. If we compensate for this shadowing, we retain a full-resolution 2D image of the parts of the scene that were in focus, as well as the lower-resolution 4D light field we recover by Fourier-domain decoding.2
The full documentation together with mathlab code for decoding images taken with a high-frequency sinusoidal pattern can be found on the MIT personal page of Ramesh Raskar. His website is full of interesting research so check out that too. One that I found interesting is the Bokode, out of focus pinhole QR code detection.
With all advances in aperture masking and microlenses photography discussed in an earlier article, it is still not understood why this kind of imaging technology has not found its way into more consumer products and is relegated to industrial and medical applications. There are certainly some disadvantages to coded aperture but much less than microlenses based light field. The complexity of computation means more powerful processors need to be accompanying the camera which increases cost but the reduction of lens components would have led to an overall reduction of manufacturing cost that could offset the need for the more powerful onboard processor. Then again that computational need could be offloaded to an external desktop pc that has sufficient computing power to process the image. The answer could just be that it’s just the case of legacy technology manufacturers holding off on the introduction of coded aperture or coded mask cameras until such time as they reach the limit of current manufacturing of cameras or outside fore pushes them.
The next step is to do some primary research into the coded aperture, coded mask camera, and microlens array cameras to find out just how complex would construction of such devices be and if it’s really possible to DIY the technology.
https://www.tandfonline.com/doi/full/10.1080/23746149.2021.1891001
https://www.ics.uci.edu/~majumder/COMPPC/papers/CodedApertureOpticalHeterodyning.pdf