Tech video: Adobe’s #3DPhotoMagic demo

While commercial products like the new Premiere Clip app and the Photoshop CC 2014.2 update got most of the attention at the recent Adobe MAX show, the firm also showed a couple of cool technology demos.

Drag 3D models into photos and match lighting automatically
One of them, identified only by the hashtag #3DPhotoMagic, is a pretty nifty consumer technology for integrating 3D models into photos, simply by dragging and dropping them into the image.

From a single photo, the system reconstructs the geometry and light sources within the scene. When a 3D model is dragged into the image, it automatically aligns itself to surfaces along their normals.

The tool, which is built on top of open-source physically based renderer LuxRender, can then render the model to match the lighting and shadows in the photo. It even supports simple animations.

Read the original research paper
The demo is based on work done by Adobe intern Kevin Karsch, set out in the Siggraph research paper Automatic Scene Inference for 3D Object Compositing.

From a single source image, the system generates depth and reflectance maps, estimates which bright pixels represent light sources, and matches ‘out-of-view’ lights to a library of panoramic images to set up IBL.

You can see the original demo video via the link below. The results aren’t perfect – but given that literally everything is done automatically from a single, standard RGB image, it’s pretty impressive.

Read Automatic Scene Inference for 3D Object Compositing on Kevin Karsch’s website