3D animation breakthrough by Disney enables realistic 3D eye reconstruction from a single photo

Jul 24, 2016 | By Alec

Over the past few months, Disney has been working very hard to make 3D printing suitable for the production of toys and merchandise – with the help of breakthroughs in high-res 3D printing processes and in the replicating of reflective properties onto 3D printed surfaces, among others. But they haven’t forgotten about their core product either: animation. And thanks to a new parametric model by Disney Research, they can now easily and accurately design realistic 3D eyes using nothing more than a single photograph.

This is a very significant breakthrough, which could provide a huge boost to 3D scanning everywhere. As you might know, facial scanning has become an industry standard in recent years, and is especially used for the development of models for video games, visual effects in films, medical applications and a whole lot more. Just over the last decade or so, the quality of facial scanning technology has improved so much that it is becoming difficult to distinguish between digital and real faces. But one hurdle continues to frustrate animators, and that is the human eye. “The eyes are arguably the most important part of the face,” said Markus Gross, vice president at Disney Research. “That's where we focus first when we look at someone and it's the eyes that convey emotions and foretell actions that are critical to storytelling.”

Of course, eyes can already be accurately captured by scanning, but it’s a cumbersome, time-consuming and error-prone process that few actors enjoy. As senior research scientist at Disney Research Thabo Beeler explained, it requires actors to lie horizontally, heads immobilized by neck braces, while they manually hold their eyes open for dozens of photos over a 20-minute period. “The physical burden of that approach is quite far from the single-shot face scanners that exist today, which are as easy as taking a single photo,” he said.

That’s why Beeler, along with fellow scientists Pascal Berard, Derek Bradley and Markus Gross from Disney Research Zurich developed a new parametric model that requires far less cumbersome input. In fact, their new model can build a stunningly realistic 3D animation by relying on a database of pre-captured eyes, and only need a single facial scan or even a photo to recreate an actor’s unique eyes. Their new method will be presented at the ACM International Conference on Computer Graphics & Interactive Techniques (SIGGRAPH) in Anaheim, California on July 24, but they have also already described it in their paper Lightweight Eye Capture Using a Parametric Model.

In a nutshell, they have built a new parametric model for eyes that relies on a novel image-based model fitting algorithm. This should enable them to generate automatic reconstructions of real eyes, as well as provide them with full artistic control over the parameters that are used to generate specific eyes for animation purposes. “This new method of eye capture enables us to create highly realistic animations for films, games and medical applications and to do so with as little fuss as possible,” Gross said.

What’s more, this new method is much quicker than the existing eye capturing techniques. By employing a parametric model of the eye, the researchers explain in their paper, they can easily reproduce the variations in the size and shape of the eyeball, as well as all the other factors that make it so human – from the spots, craters and banding of the colored iris and even the red veins on your white sclera. This also means that it can easily manipulate an actor’s eyes to suit a scene-specific look, such as tired or blood-shot eyes. The pupils can even be adjusted to match lighting conditions.

This parametric model would not be possible without the database of 30 eyes, which were previously captured in high resolution using the exhaustive existing processes. This provided them with all the necessary data on all aspects of the human eye, which was used to set up a separate model to represent variations in size, shape and all the other characteristics. Interestingly enough, the model only builds left eyes -and mirrors them for the right ones.

While that sounds simple enough, the model actually consists of three separate components, to exhaustively cover all the different regions and scales of the eye. “A single all-encompassing parametric model is not practical. For this reason we compose a model built from three separate components, namely an eyeball model that represents the low-frequency variability of the entire eyeball shape, an iris model that represents the high-resolution shape, color and pupillary deformation of the iris, and a sclera vein model,” the researchers explain.

That is a lot of data that needs to be mined for each and every eye, but the researchers argue that this is simply unavoidable – especially when it comes to the iris. “Much of the individuality of an eye can be found in the iris. A large variety of irises exist in the human population, and the dominant hues are brown, green and blue. In addition to the hue, irises also vary greatly in the number and distribution of smaller features like spots, craters, banding, and other fibrous structures,” they say.

But despite that incredible amount of data, a single facial scan is enough to capture all the details of an actor’s eye, with the algorithm taking care of the rest and transferring all the details to the 3D animation. According to Beeler and his team, this approach is very flexible – giving animators a lot of creative freedom. “The results are very plausible eye reconstructions with realistic details,” said Pascal Bérard. Disney’s (animated) films are thus about to become a whole lot more realistic. While this new model probably arrived too late to be used for the upcoming Star Wars movie, we can’t wait to see the results.

Posted in 3D Printing Application

Maybe you also like:

www.3ders.org

3D printer and 3D printing news

3D animation breakthrough by Disney enables realistic 3D eye reconstruction from a single photo

Subscribe us to

About 3Ders.org

Resources

News Archive

Categories