GenCAD: Image-conditioned Computer-Aided Design Generation with Transformer-based Contrastive Representation and Diffusion Priors

We present GenCAD, an image-conditional CAD generation model. Our model not only generates the 3D CAD but also the
entire parameterized CAD command history, CAD program, as output.

The complexity of CAD data structures such as boundary representation (B-rep) makes it difficult to train efficient
AI models. Due to the ease of data availability, common approaches often resort to representations
like meshes, voxels, or point clouds, which sacrifice the accuracy and modifiability of true CAD
models that are critical for engineering tasks, manufacturing and design space exploration. Here we
propose GenCAD, an image conditional generative model that generates parametric CAD command
sequences, also known as CAD programs, that can be converted to a 3D solid model using a geometry
kernel. At the core of GenCAD, we develop a strong representation learning framework for multiple
modalities of computational engineering designs.

Our proposed GenCAD architecture is a combination of four critical steps; 1) an autoregressive transformer encoder is used for learning the latent representation
of the CAD command sequences, 2) a contrastive learning-based model is used to learn the joint
representations of the latent spaces between CAD command sequences and CAD-images, 3) a latent
diffusion model that can generate the latent representation of CAD command sequences conditioned
on CAD-images, and 4) finally, a decoder model that can convert cad latents into a sequence of
parametric CAD commands. Most importantly, GenCAD does not merely generate a 3D solid but also the entire CAD program. Our work represents a step forward in CAD, offering more precise and
modifiable 3D modeling from images, potentially enhancing automated design processes.