Digital puppetry

Digital puppetry is the manipulation and performance of digitally animated 2D or 3D figures and objects in a virtual environment that are rendered in real-time by computers. It is most commonly used in filmmaking and television production, but has also been utilized in interactive theme park attractions and live theatre.

The exact definition of what is and is not digital puppetry is subject to debate among puppeteers and computer graphics designers, but it is generally agreed that digital puppetry differs from conventional computer animation in that it involves performing characters in real time, rather than animating them frame by frame.

Digital puppetry is closely associated with motion capture technologies and 3D animation. Digital puppetry is also known as Virtual Puppetry, Performance Animation, Living Animation, Live Animation or Realtime Animation (although the latter also refers to the animation generated by computer game engines). Machinima is another form of digital puppetry and Machinima performers are increasingly being identified as puppeteers.

History and usage

Early experiments

One of the earliest pioneers of digital puppetry was Lee Harrison III. He conducted experiments in the early 1960s that animated figures using analog circuits and a cathode ray tube. Harrison rigged up a body suit with potentiometers and created the first working motion capture rig, animating 3D figures in real-time on his CRT screen. He made several short films with this system, which he called ANIMAC.^[1]

Waldo C. Graphic

Main article: Waldo C. Graphic

Perhaps the first truly commercially successful example of a digitally animated figure being performed and rendered in real-time is Waldo C. Graphic, a character created in 1988 by Jim Henson and Pacific Data Images for the Muppet television series The Jim Henson Hour. Henson had been trying to create computer generated puppets as early as 1985^[2] and Waldo grew out of experiments Henson conducted to create a computer generated version of his character Kermit the Frog.^[3]

Waldo's strength as a computer generated puppet was that he could be controlled by a single puppeteer (Steve Whitmire^[4]) in real-time in concert with conventional puppets. The computer image of Waldo was mixed with the video feed of the camera focused on physical puppets so that all of the puppeteers in a scene could perform together. (It was already standard Muppeteering practice to use monitors while performing, so the use of a virtual puppet did not significantly increase the complexity of the system.) Afterwards, in post production, PDI re-rendered Waldo in full resolution, adding a few dynamic elements on top of the performed motion.^[5]

Waldo C. Graphic can be seen today in Jim Henson's Muppet*Vision 3D at the Disney's Hollywood Studios and Disney's California Adventure theme parks.

Mike Normal

Another significant development in digital puppetry in 1988 was Mike Normal, which Brad DeGraf and partner Michael Wahrman developed to show off the real-time capabilities of Silicon Graphics' then-new 4D series workstations. Unveiled at the 1988 SIGGRAPH convention, it was the first live performance of a digital character. Mike was a sophisticated talking head driven by a specially built controller that allowed a single puppeteer to control many parameters of the character's face, including mouth, eyes, expression, and head position.^[6]

The system developed by deGraf/Wahrman to perform Mike Normal was later used to create a representation of the villain Cain in the motion picture RoboCop 2, which is believed to be the first example of digital puppetry being used to create a character in a full-length motion picture.

Trey Stokes was the puppeteer for both Mike Normal's SIGGRAPH debut and Robocop II.

Cave Troll and Gollum on "The Lord of the Rings, The Fellowship of The Ring" (2001)

In 2000, Ramon Rivero was the first person to perform a digital puppet using Optical Motion Capture against pre-recorded action footage of a feature film. The character was the Cave Troll in the first episode of the The Lord of the Rings trilogy. The motion capture technology was created by Biomechanics Inc in Atlanta (now Giant Studios), Ramon's ideas contributed to enhancements to the technology, directly related to markering systems; virtual feedback of footage and computerised versions of the film sets; as well as the retargeting software called CharMapper (short for Character Mapper). Although the final footage was made with keyframe animation, a few seconds of Ramon's original performance can still be appreciated in the film. The character Gollum, tested by Ramon but performed by Andy Serkis, was also made with the same technology and is still considered the epitome of a virtual character in the film industry. Contrary to the Cave Troll, most of the animation of Gollum made it to the final footage using the original motion captured performance.

Bugs Live

"Bugs Live" was a digital puppet of Bugs Bunny created by Phillip Reay for Warner Brothers Pictures. The puppet was created using hand drawn frames of animation that were puppeteered by Bruce Lanoil and David Barclay. The Bugs Live puppet was used to create nearly 900 minutes of live, fully interactive interviews of 2D animated Bugs character about his role in the movie Looney Tunes: Back in Action in English and Spanish. Bugs Live also appeared at the 2004 SIGGRAPH Digital Puppetry Special Session with the Muppet puppet Gonzo.

Disney theme parks

Walt Disney Imagineering has also been an important innovator in the field of digital puppetry, developing new technologies as part of its "Living Character Initiative" in Disney theme parks. In 2004 they used digital puppetry techniques to create the Turtle Talk with Crush attractions at Epcot and Disney's California Adventure. In the attraction, a hidden puppeteer performs and voices a digital puppet of Crush, the laid-back sea turtle from Finding Nemo, on a large rear-projection screen. To the audience Crush appears to be swimming inside an aquarium and engages in unscripted, real-time conversations with theme park guests.

Disney Imagineering continued its use of digital puppetry with the Monsters Inc. Laugh Floor, a new attraction in Tomorrowland at Walt Disney World's Magic Kingdom, which opened in the spring of 2007. Guests temporarily enter the "monster world" introduced in Disney and Pixar's 2001 film, Monsters, Inc., where they are entertained by Mike Wazowski and other monster comedians who are attempting to capture laughter, which they convert to energy. Much like Turtle Talk, the puppeteers interact with guests in real-time, just as a real-life comedian would interact with his/her audience.

Disney also uses digital puppetry techniques in Stitch Encounter, which opened in 2006 at the Hong Kong Disneyland park. Disney has another version of the same attraction in Disneyland Resort Paris called Stitch Live!

Types of digital puppetry

Waldo puppetry

A digital puppet is controlled onscreen by a puppeteer who uses a telemetric input device connected to the computer. The X-Y-Z axis movement of the input device causes the digital puppet to move correspondingly. A keyboard, mouse or joystick-like device is sometimes used in place of a telemetric control.

Motion capture puppetry (mocap puppetry) or Performance Animation

An object (puppet) or human body is used as a physical representation of a digital puppet and manipulated by a puppeteer. The movements of the object or body are matched correspondingly by the digital puppet in real-time.

Machinima

Main article: Machinima

A production technique that can be used to perform digital puppets. Machinima involves creating computer-generated imagery (CGI) using the low-end 3D engines in video games. Players act out scenes in real-time using characters and settings within a game and the resulting footage is recorded and later edited in to a finished film.

References

1. ^ A Critical History of Computer Graphics and Animation: Analog approaches, non-linear editing, and compositing, accessed April 28, 2007
2. ^ Sturman, David J. A Brief History of Motion Capture for Computer Character Animation, accessed February 9, 2007
3. ^ Finch, Christopher. Jim Henson: The Works (New York: Random House, 1993)
4. ^ Henson.com Featured Creature: Waldo C. Graphic (archive.org), accessed February 9, 2007
5. ^ Walters, Graham. The story of Waldo C. Graphic. Course Notes: 3D Character Animation by Computer, ACM SIGGRAPH '89, Boston, July 1989, pp. 65-79
6. ^ Barbara Robertson, Mike, the talking head Computer Graphics World, July 1988, pp. 15-17.

External links

• Machin-X: Digital Puppetry - Discussion of theories, tools and applications of digital puppetry as well as news from the digital puppetry community.
• The Henson Digital Puppetry Wiki - Wiki for Henson Digital Puppetry projects, people, characters, and technology.
• Animata - Free, open source real-time animation software commonly used to create digital puppets.
• Mike the talking head - Web page about Mike Normal, one of the earliest examples of digital puppetry.
• Digital Puppet - Taiwanese blog about puppetry applications of interactive technologies.