Lab Manager | Run Your Lab Like a Business

The Science of Seeing Art and Color

How does a viewer see an artist’s brushstrokes of color as a cohesive image?

by University of Rochester
Register for free to listen to this article
Listen with Speechify

artThe Memorial Art Gallery’s exhibit "Monet’s Waterloo Bridge: Vision and Process" features eight paintings that Impressionist painter Claude Monet created at the turn of the 20th century. With each of the paintings in the series, Monet manipulates viewer perception in a way that scientists at the time did not completely understandCredit: University of Rochester photo / J. Adam Fenster

During three trips to London at the turn of the 20th century, Claude Monet painted more than 40 versions of a single scene: the Waterloo Bridge over the Thames River. Monet’s main subject was not the bridge itself, however; he was most captivated by the landscape and atmosphere of the scene, with its transitory light, fog, and mist.

Eight paintings from this series of London fogs are the centerpiece of the Memorial Art Gallery’s exhibition Monet’s Waterloo Bridge: Vision and Process. A recognized master of landscape painting, Monet was an integral founder of the Impressionist movement, which embraced the philosophy of expressing the fleeting sensory effects in a scene.

But how does Monet depict the same scene at different times of day and in various conditions? And how does a viewer see an artist’s brushstrokes of color as a cohesive image, and vastly different colors as the same bridge?

Get training in Risk Communication and Decision-making and earn CEUs.One of over 25 IACET-accredited courses in the Academy.
Risk Communication and Decision-making Course

With each of the paintings in the series, Monet manipulates viewer perception in a way that scientists at the time did not completely understand. Today, research such as that conducted at the University of Rochester’s Center for Visual Science, founded in 1963, provides insight into the complexity of the visual system, illuminating Monet’s processes and the intricacies of his work.

How do our eyes and brain work together to allow us to see color? 

The Memorial Art Gallery partnered with the Carnegie Museum of Art and the Worcester Art Museum to analyze the pigments of color Monet used in his Waterloo Bridges series. They found that Monet used a very limited color palette in his Waterloo Bridge series, but was still able to evoke a wide range of ambiances. How did he do this?

The answer involves how our eyes take in wavelengths of light, which our brains interpret, says David Williams, professor of optics at Rochester and the director of Rochester’s Center for Visual Science. In the retina of the eye, there are three types of cones: blue, which is sensitive to short wavelengths of light; green, which is medium-wavelength sensitive; and red, which is long-wavelength sensitive. These trichromatic signals “are very simple, yet the myriad shades of color that we experience are derived from just those three,” says Williams, whose laboratory, in the 1990s, was the first to image all three kinds of cones in a living human retina and identify how the cones are arranged.

From the retina, signals travel along the optic nerve to the visual cortex in the back of the brain. Signals are then transmitted back and forth between the visual cortex and other higher-level parts of the brain, including those involved in attention, memory, experience, and biases. The brain’s job is to integrate sensory information from the eyes into pieces—lines, shapes, and depth—and construct them into objects and scenes.

How did the visual system get so complicated?

In order to illustrate this complexity of the human visual system, Duje Tadin often begins his class on perception by asking students which is harder: math or vision?

Most people say math.

“Of course, this is a trick question,” says Tadin, a professor of brain and cognitive sciences at Rochester, who studies the neural mechanisms of visual perception. “Math is harder for us because so little of our brain is devoted to that whereas about half the brain is devoted to perception.” Take computers, for example. Computer vision programs still lag far behind what humans can do, yet even the smallest smartphones can perform complex calculations. “That’s because math is straightforward and there’s always a correct answer,” Tadin says. “Perception is heavily interconnected with other aspects of brain processing. Your prior experiences, your expectations, the way you pay attention, all of these other things that are not necessarily related to perception actually affect how you perceive things.”

Human vision, then, is “a massive reconstruction process,” says Woon Ju Park, a former postdoctoral researcher in Tadin’s lab, who helped put together the MAG’s companion exhibit Seeing in Color and Black-and-White. “This makes our perception sometimes different from the physical world existing outside of us.”

How do we perceive 3-D forms on a 2-D canvas?

One of the ways an artist like Monet exploits perception is in painting a three-dimensional scene on a two-dimensional canvas. The process is similar to what the eyes and brain do, Tadin says: our eyes are curved, but essentially a three-dimensional world gets projected—upside down—to a flat retina. The brain has to connect the dots, turn the image right side up, and extract this missing third dimension. Monet “tricks” a viewer’s brain by representing elements of light, shadow, and contrast to paint the “illusion” of a three-dimensional bridge.

“You may know it’s an illusion but your brain automatically groups things and lets you know that it is a three-dimensional scene,” Tadin says. Monet depicts things that are further away—such as the smokestacks in the Waterloo Bridges series—as smaller and blurrier to give a sense of depth. The brain’s grouping function also allows us to see the form of a bridge, river, and smokestacks before we see Monet’s individual brushstrokes of color.

“The goal of our visual perception is not to give us an accurate picture of the environment around us but to give us the most useful picture,” Tadin says. “And the most useful and the most accurate are not always the same.”

How do we perceive light in Monet’s paintings?

The illumination of an object, for example, can alter perception. That’s because what arrives at our eyes when viewing an object is a combination of both the illumination falling on the object and the intrinsic properties of the object itself, Williams says. “Your brain has a real challenge, which is to figure out what is true about this object even though what arrives at your eye is radically different depending on how it is illuminated.”

When you take an object like a white sheet of paper, it will almost always be interpreted as white—a phenomenon known as color constancy—even though the light arriving at your eye from the paper will be remarkedly different in color depending on how it is illuminated. For example, if you put the paper outside, it will still apear white in the morning light, in the middle of the day, and when the sun goes down, even thought “if we were to make objective measurements of the light entering your eye in those various circumstances, they would be very different,” he says.

The Waterloo Bridge itself never changes color, but Monet paints it by mixing pigments of color that differ in brightness, hue (a color’s relative lightness or darkness), and intensity (a color’s saturation) to depict sunrise, direct sunlight, and dusk. The brain is able to take in the illumination washing over the entire scene, integrate information, and make inferences. If all the objects have a bluish cast, for instance, the brain is able to infer that it is most likely daytime with a blue sky. If objects have a reddish cast, the brain infers that sunset is most likely approaching, Williams says. Ultimately, “Monet’s work emphasizes how different the same scene can be, depending on how it is illuminated. But any person with normal color vision looking at this series will know: the bridge is gray brick, no matter what time of day it is, because the brain has evolved clever tricks to estimate the true properties of objects despite the rich variety of illumination conditions we normally encounter.”

The phenomenon of color constancy, which vision scientists have studied for many years, received widespread attention several years ago in the infamous dress illusion, in which people viewing the same picture of a dress saw it as either blue and black or white and gold. While the dress itself was actually blue and black, people made differing assumptions about how the dress was illuminated, which, in turn, lead to differing perceptions of the color of the dress itself. “Many researchers up until that point had been assuming that everyone with normal color vision had more or less similar perceptions,” Williams says. “The remarkable differences in people’s interpretation of the dress was really an eye-opener, no pun intended, for many in the vision community.”

Another thing to know about the perception of color is that it is relative: a color changes as it interacts with other colors around it. Monet often applies starkly different colors side by side, without blending them, a technique that exploits simultaneous contrast: the same color will appear differently when placed next to different colors. The rough brushstrokes, then, are each “like spots of light that stimulate our eyes,” Park says. “Viewers can use their own reconstructive processes in the brain to integrate those patches into coherent objects that are meaningful to them.”

While our eyes and brain work to put together a cohesive view of the world, an Impressionist artist like Monet is able to do the opposite to deconstruct a scene into individual brushstrokes, she says. “Monet breaks down his perceptual experiences into different basic units of visual processing,” including color and shape, rather than focusing on the object of the bridge itself.

How does a colorblind person or person with vision or brain disorders view art?

One of the most important ways of understanding how human vision works to integrate parts into a whole is to understand what happens when it doesn’t work. Tadin, for instance, works to more accurately treat and diagnose atypical visual discrepancies associated with autismand schizophrenia.

“We know that in schizophrenia, the visual perception changes,” Tadin says. “A person with schizophrenia may experience visual hallucinations, but also their processing of incoming sensory information from the eyes is atypical, resulting in different visual experiences, affecting how one perceives and creates art.”

Williams and his colleagues are working on approaches to restore vision to the blind, either by replacing the cone photoreceptors sometimes destroyed by retinal disease or by converting other, undamaged cells in the retina—which are not normally sensitive to light—into light-sensitive cells that can substitute for the cones.  “Color blindness can be thought of as a very mild form of these diseases, in which a person is missing typically just one of the three cones responsible for daytime vision,” Williams says.

How the colorblind see art

There are three main types of colorblindness:

  • A protanope is red-green colorblind, lacks red cones, and has difficulty distinguishing between red, green, and yellow; researchers expect the color of their world to vary from blue through white to yellow with no perception of reds and greens.
  • A deuteranope is red-green colorblind, lacks green cones and, like a protanope, researchers expect the color of their world to vary from blue through white to yellow with no perception of reds and greens. Because they have cones sensitive to red light, red objects will look brighter to the deuteranope than the protanope.
  • A tritanope is blue-yellow colorblind, lacks blue cones, and has difficulty distinguishing between blue and yellow; researchers expect their world to appear various shades of red, white, and green.

Still, we can never know what another person sees, a conundrum that has also baffled philosophers and continues to occupy scientific research today. While most of us take vision for granted as an unconscious process allowing us to take in and interpret information, Tadin says, “it’s a lot harder for us to study things that are easy for us.”