Realism to Abstraction and Back Again in Art and Video Games

April 30, 2015

Forms of interactive computer simulations such as video games and virtual reality represent space in real time. The computer must make a new image every time an input is given, multiple times per second. In contrast, other computer graphics such as animated film or special Hollywood-type effects produce images that are pre-rendered and fixed, and each frame can sometimes take hours of computation to produce. 

The technical limitation of the real-time image causes video game creators to resort to abstraction in order to create space. Limited by vectors and pixels, the creators of these spaces must resort to distilling the worlds into simple geometries. 

Curiously enough, video game history starts in the mid-twentieth century with geometric abstraction representing its worlds. Now, in 2015, that history is at a point of near hyper-realism. Once realism in painting and sculpture was trumped by photography, other explorations of perception and representation of space were made and achieved within various media.  That is exactly what has been happening in video games, where pioneering developers and artists in the field are questioning how space and narrative are created in the medium of game design. Video games today are capable of creating narrative landscapes and environments experienced in real time.

Pixel

In 1972, fifty-seven years after the creation of Kazimir Malevich’s 1915 painting Black and White Suprematist Composition (Fig 1), its visual twin arose in the emerging medium of the video game. This was Pong, Atari’s flagship title, first seen in the arcade and arriving to homes everywhere in 1975 through the Magnavox Odyssey, the first successful video game console; in the dawn of the video game, a set of white squares on black backgrounds represented a table tennis match, with two large bars of squares arranged in a rectangle representing the playing humans or paddles, and a small square as the ball. The square was the pixel, the singular unit of representation available to achieve a real-time, controlled environment in which a “player” could control the tennis match. Color was limited to a binary black and white. 

The visual abstraction that had required the entire history of representation in painting to distill in the west, and had required photography to deny realism, unintentionally reached mass audiences in the form of Pong.

CSS_Castaneda_Videogames1.jpg

Fig 1. Left: Kazimir Malevich, Black and White, Suprematist Composition (1915), Oil on canvas. Right: Pong (1972), Arcade video game

Vector

Computational restrictions continued to forge unintentional parallels between art and gaming. The pixel continued to evolve past Pong into multi-colored representations. An additional development was the vector, a method of creating form purely through lines and points. Though sharper than the pixel, it eventually went into obscurity due to its lack of versatility. However, some of the first attempts at creating the illusion of 3D space came from the vector, as seen in the example of Atari’s 1980 game Battlezone (Fig 2). Early vector based games have an uncanny likeness to the work of geometric abstractionists of the 70s and 80s such as Gego (Gertrude Goldschmidt) or Sol Lewitt, whose works are just a few years apart from their game counterparts. In essence, both approaches create and distill form to point, plane, and line, and give a sensation of volumetric space. 

CSS_Castaneda_Videogames2.jpg

Fig 2. From the top, left to right: Gego, Untitled (1977), Wire. Battlezone (1980), Arcade video game. Asteroids (1979), Arcade video game. Sol Lewitt, Five Open Geometric Structures (1979), Mahogany

Pixel Scroll

After the vector came the scroll, an evolution of the pixel-based system where movement through space happened on layers of 2D planes, from left to right. In these games from the 80s and early 90s, narrative was constructed much as it was in ancient scrolls (Fig 3) where the landscape, scanned horizontally, also implies a voyage through time. Some games opted for an isometric perspective in order to simulate a sense of 3D space seen from a fixed vantage point, much like a Chinese scroll as well.

CSS_Castaneda_Videogames3.jpg

Fig 3. From the top, left to right: The Kangxi Emperor's Southern Inspection Tour, Scroll Three (1698), Handscroll. Super Mario Bros. 3 (1988), Video game. Sonic the Hedgehog 2 (1992), Video game. Energy Breaker (1996), Video game

Polygon 3D

Abstractions in the 20th century concerned with a flattening of the picture plane also resemble side scrollers or puzzle video games where events face the viewer frontally, denying angles and depth through perspective (Fig 4). The side view in games was complimented by the aerial perspective in titles such as Dwarf Fortress, Gauntlet, and in early exploration games such as Legend of Zelda: A Link to the Past where the world is depicted in a flat bird’s-eye-view representation of the space.

CSS_Castaneda_Videogames4.jpg

Fig 4. Left to right: Valerie Jaudon, Circa (2012), Oil on linen. Sonic the Hedgehog (1991), Video game. Tetris (1984), Video game. Stephen Ellis, Untitled or The Parlor Intellectual (1996), Oil and alkyd on linen. Dwarf Fortress (2006), Video game

Technological innovation and spatial storytelling grew together, and inevitably, illusionist three-dimensional space arose as soon as the computing hardware allowed. Though simple in form, early 3D polygon video games combined pixilation with vectors in order to create simple geometries. At first, in titles such as I, Robot (1983) and Starfox (1993) (Fig 5, top), flat, colored polygons were made by strategically placing pixel images to create the illusion of 3D space from a fixed vantage point. A few years down the line, in titles such as Super Mario 64, full 3D was achieved and one could rotate the camera to observe the geometric world (Fig 5, bottom). Meanwhile, around the same time as these hardware-limited video games were being made, artists like Sol Lewitt were erasing the individual hand of the artist and simplifying form as polygons. Even earlier, cubism had rendered space and figures in a manner that bears an uncanny resemblance to early 3D games.

CSS_Castaneda_Videogames5.jpg

Fig 5. From left to right: I, Robot (1983), Video game. Alpha Waves (1990), Video game. Star Fox (1993), Video game. Sol Lewitt, Wall Drawing #564 (2013, after 1988), Drawing. Super Mario 64 (1996), Video game. Metal Gear Solid (1998), Video game

Polygons Multiply Towards Realism

Polygonal games continue to grow in complexity toward increased realism and immersion.  Images are placed on the facets of the geometries to add texture. Polygons multiply over time to become organic illusions. Video games advance in computation, and parallel the Renaissance in their concern for showing how light works; how anatomy, skin, hair, and clothing work (Fig 6); and how particles flow in phenomena such as smoke and water, recalling Da Vinci’s studies recorded in his Codex Leicester (Fig 7).

CSS_Castaneda_Videogames6.jpg

Fig 6. From the top, left to right: Metal Gear Solid I and V (1998–2015), Video game. Uncharted 4 (2015), Videogame. Richard Estes, Oenophilia (1983), Oil on canvas. Chuck Close, Mark (1979), Acrylic on canvas. Grand Theft Auto V (2014), Video game

CSS_Castaneda_Videogames7.jpg

Fig 7. Clockwise: NVIDIA and Amtek studies of fluid dynamics (First five images). Leonardo da Vinci, Drawings of Water Flow and Siphon Effect (1500), Ink on paper

Digital Sublime

However, as seen in early video games, not all strive toward the illusion of reality. Deeply rooted in fantasy and science fiction, many have stylistic parallels to the nineteenth-century Romantic movement in art. In some games, sublime and gritty bloom effects abound, with the figure of the protagonist acting in an epic environment of moody vistas and monsters. Intentions and executions vary from game to game, be they a dominion of the game’s world through ultra-violence in titles such as Gears of War, or a serene partaking of the landscape in Journey (Fig 8). The visuals, however, are sometimes very reminiscent of paintings made in the 1800s by J.M.W. Turner, Caspar David Friedrich, or Frederic Edwin Church (Fig 8).

CSS_Castaneda_Videogames8.jpg

Fig 8: Gears of War 3 (2011), Video game. Caspar D. Friedrich, Wanderer Above the Sea... (1818), Oil on canvas. Frederic E. Church, Twilight (1865), Oil on canvas. JMW Turner, The Burning of the Houses...(1834), Oil on canvas. Journey (2012), Video game

Art/Video/Game/Art

The 2010s in video games parallel the nineteenth century in art, where realism and romanticism were pre-cursors of impressionism, subsequently giving rise to modern and contemporary art. With its compressed, flipped, sometimes inverted and sometimes linear lifetime, the video game finds itself face to face with Art with a capital A. Independent video game developers are looking to create meaningful artistic experiences such as Papers Please, a game where the player becomes a guard at an airport security checkpoint; or Kentucky Route Zero, a Tarkovsky-esque game set in a polygonal version of the southern United States. Gallery-based artists such as Tabor Robak create both gallery works relating fine art to video games and online virtual experiences like games.  Others creating such notable hybrids include Mason Lindroth, a game creator and visual artist who is making games in the style of clay animation; Jacolby Satterwhite, an artist working in performance and 3D animation merging his mother’s drawings with virtual environments that question identity; and La Turbo Avedon, an avatar-as-artist that makes virtual exhibition spaces, paintings, and environments, including Club Rothko, a virtual nightclub designed after the paintings of Mark Rothko (Fig 9).

CSS_Castaneda_Videogames9.jpg

Fig 9. Weird Egg...(2014), Vid game. Jacolby Satterwhite, En Plen Air...(2014), C Print. Tabor Robak, Carbon (2010). 3D Environ. Papers please (2013), Vid game. Tabor Robak, A*(2014), Real time 3D. Kentucky...(2014), Vid game. La Turbo...(2013), Virt Envi

An interweaving of media will continue to flourish as lines blur between artists who make games and virtual reality and those who are influenced by it. The horizons are broad for the contemporary artist whose works can cut across audiences and be accessed through live exhibition spaces, apps and computers. Also, thanks to the gaming community, virtual reality headsets such as the Oculus rift have resurfaced, and aim to bring a personal and total sensory virtual reality experience as a medium for entertainment and art  (Fig 10).

CSS_Castaneda_Videogames10.jpg

Fig 10. Clockwise: Image of 80s Arcade machine. Image of woman wearing the Oculus Rift. Leo Castaneda, Item 93201 (2014), Interactive sculpture for viewing virtual reality (Installation shot and interior view)

Extended Captions

Fig 1. Left: Kasmir Malevich, Black and White, Suprematist Composition, 1915, Oil on canvas. Right: Pong, Atari, 1972, Arcade video game.

Fig 2. Top Left: Gertrude Goldschmidt (GEGO), Untitled, 1977, Wire. Battlezone, Atari, 1980, Arcade video game. Asteroids, Atari inc, 1979, Arcade video game. Sol Lewitt, Five Open Geometric Structures, 1979, Mahogany

Fig 3.-The Kangxi Emperor's Southern Inspection Tour, Scroll Three: Ji'nan to Mount Tai, Qing dynasty (1644–1911), 1698, Wang Hui (Chinese, 1632–1717) and assistants, Handscroll; ink and color on silk; 26 3/4 x 548 1/2 in. Super Mario Bros. 3, Nintendo, 1988, Video game. Sonic the Hedgehog 2, Sega, Sonic Team, 1992 Video game. Energy Breaker, Tagita Corporation Japan, Nintendo, 1996. Video game

Fig 4. Valerie Jaudon, Circa, 2012, Oil on linen, 56”x64”. Sonic the Hedgehog 1, Sega, Sonic team, 1991, Video game. Tetris, Alexey Pajitnov, Infogrames, 1984, Video game. Stephen Ellis, Untitled or The Parlor Intellectual, 1996, Oil and Alkyd on linen, 60”x50”. Dwarf Fortress, Tarn Adams, Bay 12 games, 2006, Video game

Fig 5. I, Robot, Dave Theurer, Atari Inc, 1983, Video game. Alpha Waves, Christoph de Dinechin, Infogrames, 1990, Video game. Star Fox, Nintendo, 1993, Video game. Sol Lewitt, Wall Drawing #564, 2013, After 1988 drawing. Super Mario 64, Nintendo, 1996, Video game. Metal Gear Solid, Konami, PlayStation, 1998, Video game

Fig 6. Metal Gear Solid V Demonstration of Graphics evolution since MGS1 (1998-2015), Konami Entertainment, 2015. Uncharted 4 clothing study, Naughty Dog, Sony Entertainment, 2015. Richard Estes, Oenophilia, 1983, Oil on canvas. Chuck Close, Mark, 1979, Acrylic on canvas, 9' x 7'. Grand Theft Auto V, Rockstar Games, 2014, Videogame

Fig 7. NVIDIA and Amtek studies of fluid dynamics. Leonardo da Vinci, Drawings of Water Flow and Siphon Effect, Codex Leicester, 1500, Ink on paper

Fig 8. Gears of War 3, Epic Games, Microsoft Xbox 360, 2011, Video game. Caspar David Friedrich, Wanderer above the Sea of Fog, 1818, Oil on canvas, 37x29 inches. Frederic Edwin Church, Twilight: Mount Desert Island, Maine, 1865, Oil on canvas. JMW Turner, The Burning of the Houses of Parliament, 1834, Oil on canvas. Journey, That Game Company, Sony Computer entertainment, 2012, Video game

Fig 9. Weird Egg and Crushing Finger, Mason Lindroth, 2014, Video game. Jacolby Satterwhite, En Plen Air: Abduction, 2014, C Print of 3D Animation. Tabor Robak, Carbon, 2010, Real time 3D environment. Papers please, Lucas Pope, 2013, Video game. Tabor Robak, A*,  2014, 14 Channel HD Video, Real time 3D, 10 min, 14’x14’. Kentucky route zero Act III, Cardboard Computer, 2014, Video game. La Turbo Avedon, Club Rothko, 2013, Virtual environment

Fig 10. Image of 80’s Arcade machine. Image of woman wearing the Oculus Rift. Leo Castaneda, Item 93201, Interactive sculpture for viewing virtual reality, 2014. Leo Castaneda, Interior view of Item 93201, Still from virtual reality simulation, 2014