Why Physically Accurate AI Image Composition Is Harder Than Your Vendor Admits

Most enterprise teams evaluating visual AI for product imagery discover the same thing at roughly the same point in their pilot: the generative step is not the bottleneck. Producing a plausible-looking image of a product in a new scene is straightforward. Producing one where the lighting wraps correctly around the object's geometry, where shadows fall at the right angle, and where the scene itself responds to the object's presence, is a different class of problem. Vendors rarely frame it that way during procurement. This article explains why physical plausibility is architecturally hard, what the research tells us about the current state of solutions, and how to stress-test vendor claims before you commit to a production pipeline.

The Generative Step Is Not the Hard Part

Diffusion models have made high-resolution image synthesis accessible to the point where most enterprise vendors can demonstrate convincing demos within minutes. The demos tend to share a common characteristic: the inserted object is visually coherent in isolation, and the scene looks plausible at a glance.

What the demos rarely show is the same pipeline applied to a product with complex geometry, photographed from an unfamiliar angle, placed into a scene with directional lighting from a source that was never in the training distribution. That is the production case for most retail and e-commerce workflows, and it is where the failure modes become commercially significant.

The core issue is that diffusion models trained on 2D image pairs learn appearance correlation, not physical causality. They can replicate the look of a shadow without understanding where it should fall. That distinction matters when you are producing thousands of product images that will be reviewed by buyers and merchandisers who notice when a sneaker's sole casts a shadow in the wrong direction.

Why Geometry Is the Structural Constraint

Object insertion that looks physically plausible requires the model to reason about three-dimensional structure, not just pixel distribution. Perspective consistency, surface normal alignment, and contact geometry all depend on an underlying representation of shape that standard 2D pipelines do not maintain.

Pose and Viewpoint Adaptation

When a reference image of a product is captured from one angle and the target scene implies a different viewpoint, the model must infer what the object looks like from that new angle. Without a geometry-aware representation, models typically copy appearance features from the reference and hope the result is coherent. The result is often a product that looks pasted rather than placed.

Recent work addresses this directly by coupling geometry estimation into the adaptation pipeline. PhysInsert, for example, integrates a VGGT geometry stream alongside image and text streams, using target-scene feature supervision to constrain spatially consistent insertion (Gong et al., HuggingFace 2026). The practical implication is that geometry-aware architectures require geometry-aware training data, which raises its own set of infrastructure requirements.

Contact and Occlusion Plausibility

A second geometric failure mode is contact. When a product is placed on a surface, the model must infer where the object meets the scene, how that contact deforms the scene geometry if at all, and what occlusion relationships result. Models that treat insertion as a masked inpainting task frequently get this wrong because the mask boundary does not encode physical contact information.

The Illumination Problem Is Bidirectional

Lighting adaptation is where most production pipelines quietly fail at scale. The standard framing is unidirectional: adapt the object's appearance to match the scene's lighting. That framing is incomplete, and the incompleteness is commercially consequential.

Object-to-Scene Light Transfer

A physically accurate insertion also requires the object to affect the scene. A white product placed near a coloured wall will cast a colour bleed. A glossy object will reflect its surroundings. A lamp placed in a room will illuminate it. These bidirectional interactions are what human reviewers use, often unconsciously, to judge whether an image is real.

Existing mask-and-fill methods can adapt object illumination but typically fail to model how the inserted object changes its surroundings (Gong et al., HuggingFace 2026). The architectural reason is that supervision is applied only to the object region, so the model has no gradient signal pushing it to modify the scene in response to the object's presence.

Training Data Requirements

Solving bidirectional illumination adaptation requires training pairs where both the object and the scene vary in a controlled, physically grounded way. This is not achievable through web-scraped image pairs. It requires synthetic rendering pipelines, typically Blender-based, that can produce counterfactual pairs: the same scene with and without the object, across multiple lighting configurations and viewpoints. The Insert World dataset used in PhysInsert was constructed precisely this way (Gong et al., HuggingFace 2026). The data infrastructure cost is non-trivial, and vendors who cannot describe their training data provenance in this level of detail are unlikely to have solved the problem.

What Vendor Evaluation Should Actually Cover

Most vendor evaluations focus on visual quality metrics applied to the vendor's own benchmark images. That is a weak test. The images are typically selected to show the pipeline at its best, and the metrics used, FID score or CLIP similarity, do not measure geometric or illumination plausibility directly.

A more informative evaluation protocol should include cross-environment cases, where the same product reference is inserted into scenes with meaningfully different lighting conditions. It should include cross-view cases, where the reference image viewpoint differs substantially from the implied viewpoint in the target scene. And it should include cases where the inserted object is large enough or reflective enough to visibly affect its surroundings.

We have written about the broader architectural requirements for geometry-native pipelines in enterprise contexts, including how geometry encoders and 3D-coherent latent representations change the evaluation criteria for visual AI tooling.

Companion piece to our broader work on geometry-aware visual AI. See 3D Consistency in Enterprise Visual AI Pipelines for a technical guide to geometry encoders, 3D coherence, and vendor evaluation criteria.

The Dataset and Architecture Requirements That Separate Credible Systems

A vendor's ability to solve physical plausibility at production scale is largely determined by two decisions made before any model training begins: the architecture and the dataset construction methodology.

On the architecture side, the minimum requirement for geometry-consistent insertion is a model that maintains a 3D-aware or geometry-conditioned representation throughout the generation process, not just as a post-processing correction. Systems that bolt geometry checks onto a 2D diffusion backbone will produce inconsistencies under distribution shift, which is exactly what production workloads generate continuously.

On the data side, the requirement is physically rendered counterfactual pairs with controlled variation across viewpoints and lighting environments. Web-scraped composites cannot provide this because they lack the ground-truth physical consistency needed to supervise bidirectional interaction. The cost of building this data infrastructure is high enough that it functions as a meaningful signal of a vendor's genuine commitment to the problem. Asking a vendor to describe their training data construction process in detail will quickly separate those who have invested in solving physical plausibility from those who have approximated it.