Science —

CSI image enhancement: using evanescent waves to see the invisible

Researchers use an algorithm to reveal hidden image features.

Those of you who read my writing regularly know that I love pretty pictures. From my perspective, pretty much every camera and imaging system in the world is disappointing, because, when you "enhance the image," you end up with a pixelated mess. I want to be able to zoom in and have details magically appear, dammit.

The problem is that the information required to enhance the image isn't actually in the image itself. So, how can you possible recreate those details when you don't have them in the first place? This is precisely what a group of researchers from Israel and Germany has done using something called coherent diffractive imaging. And, while this is probably never going to turn up in a real crime scene investigation, the industrial and scientific applications are going to keep me happy for at least 20 minutes.

Where are the details?

In every article I write about imaging, I yammer on extensively about the diffraction limit, and how it limits the image to features above a certain size. Now, although I can (and will, if I don't get a drink) go on about this at length, the crux is that light carries the image information in its spatial frequencies. If we were to illuminate a scene with a green laser, the scene appears green, but the spatial details are still there, carried by the amplitudes and directions (the spatial frequencies) that the green light is traveling when it hits the lens of your eye.

Any imaging system has a limited aperture, which puts a cap the range of directions from which you can obtain light from the object you are viewing. That in turn limits the spatial details that we can perceive. But, even if we had a magic lens that could gather all the light scattered from the object, we would still miss details. Remember, light is a wave, and if we were to freeze it in time, we would find a spatial variation in the electric field. In other words, the wavelength of the light is itself a spatial frequency. This is, to within a factor of two or so, the highest spatial frequency that the light can carry.

To try to make this concept concrete, I have attempted to illustrate the situation. In the picture below, you see four images. The first is a picture of a girl, the second is a map of the spatial frequencies present in the image. Every pixel of that second image corresponds to a regular spatial variation in the image, while the intensity of the pixel corresponds to the amplitude of the contribution. The third picture gives an impression of how the image builds up as we add spatial frequencies (you need to click to load the gif).

Top left: original picture. Top right: spatial frequencies that make up the image. Bottom right: all spatial frequencies except the lowest removed. Bottom left: resulting copy of the original image.
Enlarge / Top left: original picture. Top right: spatial frequencies that make up the image. Bottom right: all spatial frequencies except the lowest removed. Bottom left: resulting copy of the original image.

All is not lost though, because the rest of the spatial information is still there. It is carried in waves (called evanescent waves) which fade away very fast. Their amplitude falls off by about one half for every wavelength travelled from the source, so for a 1mm focal length imaging system and green light, the evanescent waves are e2000 times weaker than their normal counterparts. For any reasonable and most unreasonable input powers, this corresponds to less than the energy carried by a single photon. Unfortunately, to see those hidden features in an image, you need to figure out the contribution of these waves.

Enhance that image anyway

So, the spatial information is still there, but it is encoded in waves that are so weak that they cannot be detected. Researchers have been asking themselves if the information encoded in these undetected waves might still have an indirect influence on the observed image. The idea is that only one combination of evanescent waves and normal waves leads to the observed image.

To put it slightly more concretely, you guess the form of the evanescent waves, and calculate the image. You compare the recorded image and the calculated image at its original resolution and see if they agree. If only one set of evanescent waves generates an image that agrees with the original, then the calculated image must correctly reconstruct the details not included in the original image. Hence, we can use this to get access to the evanescent waves, and use those to uncover details not included in the original image.

Unfortunately, this turns out to be completely wrong. Any combination of evanescent waves whatsoever will still provide the original image. This means that you can, essentially, make up the picture's hidden details without any noticeable modification to the original image. At this point it all seems to be a bit hopeless.

But, yell our intrepid researchers, don't anybody start singing yet. The key, it turns out, has to do with the density of the information in the image. If we look back at the map of spatial frequencies, we see that the image contains a significant contribution from all the possible spatial frequencies in that particular representation of the image. This makes it highly likely that any attempt to reconstruct hidden details will go wrong.

This is not the only way to represent the image, however. We could choose a different set of basic building blocks to represent the image. So, for instance, instead of using a set of wave-like functions as we did above, we could use a set of squares with different sizes and amplitudes. If we choose this set so that most of the values in the set are zero, then, for reasons I don't understand at all, a reconstruction algorithm will generate the hidden details correctly.

As I understand it, the researchers choose a representation that meets a threshold requirement for it being sparse—that is, a certain large fraction of the values in the representation must be zero. Then, their algorithm expands the set of spatial frequencies that represent the image so that it can contain many more spatial details. When they do this, the image features that were represented by a single value in the initial representation end up represented by many values in the new representation. In other words, the representation becomes less sparse. But it also has a whole lot of artifacts in it.

To refine the image, the algorithm modifies the amplitudes of each spatial frequency in this representation until, at the original resolution, they regain the original image. This new set of spatial frequencies is tested to see if it is as sparse as the original set of spatial frequencies. If not, it moves on to test a different expansion. In the end, the algorithm winds up with a correct image at higher resolution while maintaining the sparsest possible representation.

The researchers demonstrated the usefulness of coherent diffractive imaging by taking pictures of holes drilled in a chrome coated piece of glass. The groups of holes were spaced such that an ordinary optical microscope could distinguish them, but, within each group, the individual holes could not be seen. They showed that the image they recovered had a much higher resolution than the microscope image. Further, because they knew the layout of the holes, they could calculate the phase and amplitude spectrum of each spatial frequency and compare it to that from the reconstructed image. The two agree beautifully.

Does this mean we can expect to see even more beautiful images of tiny stuff in the near future? Yes... but, then again, maybe not. This demonstration used an object that was very high contrast. The holes transmitted light, while the rest did not. In more realistic imaging situations, say of nanoparticles or cells, then the contrast will be quite low. Now, the accuracy of the reconstruction depends on accurately retrieving the phase and amplitude of the high frequency spatial frequency components. My suspicion is that it is much more difficult to do that if the contrast is low.

Nature Materials, 2012, DOI: 10.1038/nmat3289

Listing image by Chris Lee

Channel Ars Technica