When household objects come to life onscreen (think candlesticks and teacups in Beauty and the Beast, books and brooms in Harry Potter, the cast of practically every Pixar film), it’s the result of careful animation or painstaking CGI. Done well, it’s magic. But computer-generated imagery doesn’t translate well into live theater, so researchers in Zurich have concocted an enchanting alternative: dancing drones. For the past five years, a team with ETH Zurich (the Swiss Federal Institute of Technology) has studied the possibilities of “athletic quadcopters,” algorithmically-powered drones that can solve problems like a human might. Their work caught the attention of Cirque du Soleil, which you could say also is in the business of solving physical problems. The unlikely collaborators created SPARKED, a special-effects-free video starring a repairman and a troupe of dancing lampshades.

The lampshades are, of course, drones in costume. Verity Studios, an ETH offshoot, shot the four-minute film at the Flying Machine Arena at ETH Zurich, which researcher Markus Hehn calls a “sophisticated test bed for autonomous flight … used for development, testing, and demonstration of flying machines.” Established at ETH Zurich in 2007, the Flying Machine Arena serves as a testbed for research in aerial vehicles. ETH Zurich researchers have explored quadrocopter control and state estimation, trajectory generation, increased autonomy, adaptation and learning, high-precision flight maneuvers, aerial construction, and cooperation between multiple vehicles.

Many of these research results have been fundamental to the realization of SPARKED. The seamless coordination of multiple vehicles, the design of suitable trajectories and choreographies, and the high reliability and robustness of the Flying Machine Arena infrastructure were key ingredients for the success of the film shoot. In the clip, they blow a fuse. His shop goes dark, then lights up as the lampshades come alive. When he waves his arms, the lights spin and twirl overhead.

The choreography is the result of algorithms that capture data from the robots and a motion capture system that acts like an indoor GPS. It’s all processed through a couple of standard desktop PCs loaded with software that can rapidly prototype the drones movement. Creating the illusion of a finely-tuned ballet meant the ETH team had to make the positioning algorithms even more precise than usual: “Operating several of them in close proximity makes their control more difficult, because the air pushed by one robot’s propellers will affect the motion of other robots nearby," says Henn. Algorithms used to control many robots must be robust to these effects.”

SPARKED showcases a fairly simple dance number. But you can imagine that as those algorithms become more robust, the robots will become more athletic, like the performers of Cirque du Soleil.

For three days, the Flying Machine Arena became a film set: Three hardwood work benches, 50 lamps, and other props were standing in. During the shoot the space also hosted the researchers, creative team, producers, technicians, and film crew as well as 11 actors – one human and 10 machines. The Flying Machine Arena (FMA) is a portable space devoted to autonomous flight. Measuring up to 10 x 10 x 10 meters, it consists of a high-precision motion capture system, a wireless communication network, and custom software executing sophisticated algorithms for estimation and control.

The motion capture system can locate multiple objects in the space at rates exceeding 200 frames per second. While this may seem extremely fast, the objects in the space can move at speeds in excess of 10 m/s, resulting in displacements of over 5 cm between successive snapshots. This information is fused with other data and models of the system dynamics to predict the state of the objects into the future. The system uses this knowledge to determine what commands the vehicles should execute next to achieve their desired behavior, such as performing high-speed flips, balancing objects, building structures, or engaging in a game of paddle-ball. Then, via wireless links, the system sends the commands to the vehicles, which execute them with the aid of on-board computers and sensors such as rate gyros and accelerometers.

Although various objects can fly in the FMA, the machine of choice is the quadrocopter due to its agility, its mechanical simplicity and robustness, and its ability to hover. Furthermore, the quadrocopter is a great platform for research in adaptation and learning: it has well understood, low order first- principle models near hover, but is difficult to characterize when performing high-speed maneuvers due to complex aerodynamic effects. We cope with the difficult to model effects with algorithms that use first-principle models to roughly determine what a vehicle should do to perform a given task, and then learn and adapt based on flight data.

{ Behind the Technology Video Transcript }

Many people from ETH Zurich, Verity Studios, and Cirque du Soleil were involved in the realization of SPARKED: Welby Altidor, Federico Augugliaro, Simon Berger, Jean-Francois Bouchard, Dario Brescianini, Marc-Andre Corzillius, Rino Côté, Raffaello D'Andrea, Benjamin Dupont, Michael Egli, Bernard Fouché, Luca Gherardi, Martin Giguère, Daryl Hefti, Markus Hen, Bill Keays, Nicolas Leresche, Carlos Larrea, Martin Luchsinger, Sylvie McLaughlin, Mark W. Mueller, Francis Ouellet, Luce Pellerin, Philip Petersson, Robin Ritz, Neilson Vignola, Alain Vinet, Markus Waibel, Matthew Whelan, Alex Wilkinson, Evan Wilson.

Film Production Company:
Who's McQueen Picture Gmbh

Music by: Danny Elfman
"The Broom" from IRIS