Aug. 7, 2014
It is common that editors of photos routinely resize objects, or move them up, down or sideways, but researchers at Carnegie Mellon University and the University of California are adding an extra dimension to photo editing by enabling editors to turn or flip objects any way they want, even exposing surfaces not visible in the original photograph, using freely available stock 3D models of everyday objects.
A chair in a photograph of a living room, for instance, can be turned around or even upside down in the photo, displaying sides of the chair that would have been hidden from the camera, yet appearing to be realistic.
This three-dimensional manipulation of objects in a single, two-dimensional photograph is possible because 3-D numerical models of many everyday objects — furniture, cookware, automobiles, clothes, appliances — are readily available online. The research team led by Yaser Sheikh, associate research professor of robotics, found they could create realistic edits by fitting these models into the geometry of the photo and then applying colors, textures and lighting consistent with the photo.
"In the real world, we're used to handling objects — lifting them, turning them around or knocking them over," said Natasha Kholgade, a Ph.D. student in the Robotics Institute and lead author of the study. "We've created an environment that gives you that same freedom when editing a photo."
Originally the system was designed to work using digital imagery, but the researchers have found that it can also be used to manipulate images in paintings and in historical photos. Objects that can be manipulated in photos also can be animated; the researchers demonstrated that an origami bird held in a hand can be made to flap its wings and fly away, or a taxi cab shown in a street scene can levitate, flip over to reveal its undercarriage and zip off into the heavens.
"Instead of simply editing 'what we see' in the photograph, our goal is to manipulate 'what we know' about the scene behind the photograph," Kholgade said.
The main weakness of using publicly available 3-D models is that the models seldom fit a photo exactly. Variations occur between the models and the physical objects; real-life objects such as seat cushions and backpacks are sometimes deformed as they are used, and appearances may change because of aging, weathering or lighting.
To fix these variations, the researchers developed a technique to semi-automatically align the model to the geometry of the object in the photo while preserving the symmetries in the object. The system then automatically estimates the environmental illumination and appearance of the hidden parts of the object — the visible side of a seat cushion or of a banana is used to create a plausible appearance for the opposite side. If the photo doesn't contain pertinent appearance information — such as the underside of a taxicab — the system uses the appearance of the stock 3-D model.
Though a wide variety of stock models is available online, models are not available for every object in a photo. But thanks to the widespread of 3D scanning and printing technologies, the selection is getting better and better. "The more pressing question will soon be, not whether a particular model exists online, but whether the user can find it," Sheikh said. One thrust of future research will thus need to be automating the search for 3-D models in a database of millions.
Kholgade will present the team's findings Aug. 13 at the SIGGRAPH 2014 Conference on Computer Graphics and Interactive Techniques in Vancouver, Canada.
If you're interested in the details behind the software, you might like to read the paper the researchers just published, and you can also download the software to try out the effect yourself. The source code is also available here in case you feel like doing some programming.
Posted in 3D Software
Maybe you also like:
- Siemens donates $440 million to prepare students at YSU for advanced manufacturing
- Free Cubify Draw app turns your fingers into an instant 3D drawing tool
- Tinkercad reopens, Autodesk agrees to acquire Tinkercad
- Shape-It-Up lets designers manipulate 3D shapes using hand gestures
- 3D Systems can turn you into a Star Trek figure for $70
- Mecube: easy-to-learn 3D design app for iPhone and iPad
- Blender 2.67 RC1 and DVD Training Blender for 3D Printing Released
- Printcraft latest version released, teaming up with Makerbot
- Try glasses online using 3D Modeling and an iPad
@DJH How said something about 3D Printing?
Anders wrote at 8/14/2014 2:33:25 PM:
And it seems to be only a MAC exe.
DJH wrote at 8/11/2014 4:02:38 PM:
Interesting, but has nothing to do with 3d printing.
3Ders.org provides the latest news about 3D printing technology and 3D printers. We are now seven years old and have around 1.5 million unique visitors per month.
