3D Object Manipulation in a Single Photograph using Stock 3D Models
Natasha Kholgade1 Tomas Simon1 Alexei Efros2 Yaser Sheikh1
1Carnegie Mellon University 2University of California, Berkeley
Original Photograph Object Manipulated in 3D
>
Hidden Right Side
Appearance from Left Side
Hidden Under Side
Appearance from Stock Model
Before After Before After
3D Copy-PasteEstimated Illumination
z
y
x
Figure 1: Using our approach, a user manipulates the taxi cab in a photograph to do a backflip, and copy-pastes the cabs to create a traffic
jam (right) by aligning a stock 3D model (inset) obtained from an online repository. Such 3D manipulations often reveal hidden parts of the
object. Our approach completes the hidden parts using symmetries and the stock model appearance, while accounting for illumination in 3D.
Photo Credits (leftmost photograph): Flickr user © Lucas Maystre.
Abstract
Photo-editing software restricts the control of objects in a photo-
graph to the 2D image plane. We present a method that enables
users to perform the full range of 3D manipulations, including scal-
ing, rotation, translation, and nonrigid deformations, to an object in
a photograph. As 3D manipulations often reveal parts of the object
that are hidden in the original photograph, our approach uses pub-
licly available 3D models to guide the completion of the geometry
and appearance of the revealed areas of the object. The completion
process leverages the structure and symmetry in the stock 3D model
to factor out the effects of illumination, and to complete the appear-
ance of the object. We demonstrate our system by producing object
manipulations that would be impossible in traditional 2D photo-
editing programs, such as turning a car over, making a paper-crane
flap its wings, or manipulating airplanes in a historical photograph
to change its story.
CR Categories: I.3.7 [Computer Graphics]: Three-Dimensional
Graphics and Realism—Virtual Reality;
Keywords: three-dimensional, photo-editing, 3D models
Links: DL PDF
1 Introduction
One of the c