Farm 3D

Farm3D introduces a groundbreaking approach to learning articulated 3D animals through the distillation of 2D diffusion. Developed by researchers Tomas Jakab, Ruining Li, Shangzhe Wu, Christian Rupprecht, and Andrea Vedaldi at the University of Oxford, Farm3D leverages "free" virtual supervision from a 2D diffusion-based image generator, such as Stable Diffusion, to teach an articulated object category entirely.

The framework not only utilizes the image generator for training data but also incorporates the diffusion model as a scoring mechanism, enhancing the learning process. The result is a monocular reconstruction network capable of swiftly generating controllable 3D assets from a single input image, whether real or generated. This innovative method signifies a significant advancement in monocular 3D reconstruction, enabling functionalities like textured controllable synthesis, animation, relighting, and texture swapping in a matter of seconds.