Images, illusions, and insights

In the lab with vision researcher Zhaoping Li

In a dimly lit laboratory, Zhaoping Li unrevels how we interpret — or misinterpret — what we see. The brain can be fooled by optical "fake news." How does this help the neuroscientist understand how we perceive the world? One subject reports.

 “Oh wow – this is unexpected!” Zhaoping Li excitedly points at the computer screen in front of her. The old-school tube monitor – “LCD just doesn’t come close yet for this kind of experiment,” she tells me later – shows an inconspicuous gray bar plot. What is so special about it escapes me at this point, but even in the dim light of the experimentation room the enthusiasm on her face is clearly visible.

We have just spent the past two hours together in the basement of the Max Planck Institute for Biological Cybernetics, where Zhaoping runs her visual research lab. For the better part of this time, I have been staring at dots on a screen through a set-up of mirrors: black and white dots, sometimes flickering, sometimes steady, but all of them arranged in the shape of a disk surrounded by a ring. My task: deciding whether the central disk appears to be in front of or behind the rest of the image.

Navigating the depths of illusions

This illusion of depth is what the mirror set-up is for: slightly different images are presented to the left and the right eye. The brain interprets a dot seen by the left eye and a dot seen by the right eye as one single speck. But if the position of the left-eye dot is shifted a bit to the right or left, the brain perceives the speck as closer or further away.

It sounds easy, but if you are Zhaoping’s guinea pig, it might not feel that way. For some of the pictures, I can only guess whether the disk appears in the fore or background. This is because some of the dot-pairs – sometimes many – consist of a black and a white dot, instead of two dots of the same shade. This reverses the depth signal the brain receives from the pair: if a pair of black dots signals “far”, changing the shade of just one of them will make it signal “near”, and vice versa.

For an uninitiated test subject like me, it is only apparent that the depth signals are rather disorienting, and Zhaoping is careful to not hint at what she is really interested in: the feedback process between different brain regions, according to her hypothesis the key to really seeing (rather than just looking). Sounds confusing? Let’s take things one step at a time. I am surprised at first that Zhaoping stays at my side for the entire duration of the experiment, calibrating the equipment and chatting with me during breaks; you might think that as a senior researcher she’d be too busy for hands-on experimentation. “I love working with my subjects!” she corrects me, “It gives me invaluable extra information.” Indeed, she takes notes of everything I report about my experience and asks background questions. Over three decades of brain research experience let her see connections other might miss. Sometimes the details she is not actively looking for are exactly what sparks an idea for a new hypothesis to explore.

Fake news for the visual cortex

While I am intently staring at the dot images, Zhaoping is tracking my eye movements: for some trials, I am instructed to look directly at the disk, while for others I am supposed to shift my gaze a little upwards. Vision in the peripheral field works differently from central vision, she explains me later: everything our eyes see arrives at the back of our brain, in the primary visual cortex. But only a tiny fraction of the input is forwarded to higher brain areas, which then try to make sense of it. When we are looking directly at something, the higher visual areas send feedback to the primary visual cortex, asking it for more information from the eyes. However, when an object is in our peripheral field of view, the brain just jumps to conclusions.

This is why our vision can rather easily be lied to. Case in point: in many of Zhaoping’s disk pictures, some of the dots in the disk appear in the fore and other in the background. “I am giving the visual cortex fake news,” she quips. “When you look directly at an ambiguous picture, your higher visual areas say ‘wait a minute, what is that?’ and ask the primary visual cortex to check again.” In that case, one sees the disk for what it is: a rather nonsensical cluster of spots. Not so if the disk is in the peripheral field of vision: the higher visual areas forego the fact check; they create a mental image from the sparse data available to them. The result: we perceive a disk in the front or the back – an illusion triggered by the fake news.

The charm of indirect observation

Zhaoping has been intrigued by the higher visual areas for a long time. Testing how they work without using invasive methods has a particular fascination for her: a physicist by training, she likes the challenges that come with indirect observation. “People ask me how I can research the brain without cutting it open,” she relates. “But think of electrons: has anyone ever seen one? We only know about them indirectly; for instance, we see sparks when rubbing a synthetic sweater in the dark.”

The indirect nature of her observations makes it so important for Zhaoping to keep collecting more evidence to harden her hypothesis. This time, she has added an extra twist to the series of experiments: some of the images flicker. This makes it harder or even impossible for the brain to do a fact check: by the time the brain tries to check, the image has already changed. The bar plots with my central vision trials showcase this: quickly jittering images can trick me into seeing the illusion, confirming that the feedback process between the different vision processing brain regions is indeed needed to properly look at something.   

But it is an incidental detail that has startled Zhaoping when first looking at my results: my peripheral vision cannot be misled as easily as expected. “You don’t believe the fake news!” she excitedly exclaims. Immediately, she starts thinking of different possible explanations and ways to test them. For someone like Zhaoping who is driven by enthusiastic curiosity, finding the unexpected is not a bump on the road but a thrilling experience: each answer Nature gives merely begs the next question in the endless journey of understanding.