Snapshot Polarimetric Display Inverse Rendering

Seokjun Choi1*, Yunseong Moon1*, Kaizhang Kang1, Hoon-Gyu Chung1, Jin-Nyeong Kim1, Giljoo Nam2, Seung-Hwan Baek1
1POSTECH   2Meta
*Equal contribution
ACM Transactions on Graphics (Proc. SIGGRAPH Asia 2026), Kuala Lumpur, Malaysia
Paper arXiv Code
TL;DR

We present a feed-forward inverse rendering method that works from a snapshot input. On a display setup it combines color multiplexing with polarization imaging built on measured reflectance data.

Teaser

Inverse rendering remains a core challenge in graphics and vision, especially in the snapshot configurations required for lightweight desktop workflows, where the per-frame information budget is highly constrained. Previous inverse rendering work explores various available dimensions for enriching the per-shot information, including temporal modulation, spectral encoding, and polarization. In this work, we introduce polarimetric display inverse rendering, using an LCD to project a linearly polarized RGB binary pattern and an RGB polarization camera augmented with a quarter-wave plate to acquire color-polarimetric measurements in a single shot. A feed-forward transformer maps these measurements to per-pixel normal, albedo, roughness, and metallicity. To overcome training data scarcity, we expand a limited set of measured polarimetric bidirectional reflectance distribution functions via a generative manifold. Evaluations on a real desktop setup demonstrate accurate inverse rendering across diverse scenes, outperforming existing approaches.

Results

Estimated PBR maps

Estimated PBR maps

Relighting under a held-out display pattern

Relighting comparison

Move the light

Environment lighting

Objects
Lighting
Environment relighting grid

Dynamic scene

Capture
Relit, frame by frame

Imaging system

  • An LCD is already a polarized emitter — no polarizing optics on the illumination side.
  • RGB binary pattern: right half → R, left half → G, upper half → B. Three lighting directions in one shot.
  • A quarter-wave plate on the lens trades S2 for S3, making circular polarization measurable.
  • Per channel: unpolarized = S0−S1, LP = −S1, CP = S3/S0.
Display-camera imaging system

Feed-forward inverse rendering

  • The nine measurements enter an encoder–decoder transformer as a token sequence.
  • Attention alternates over space, spectral channel, polarization state and frames.
  • One forward pass → normal, albedo, roughness, metallicity.
Network overview

Expanded pBRDF dataset

  • Measured pBRDFs are compressed to a PCA basis.
  • A generator samples PCA weights conditioned on PBR parameters → 2,000 physically valid pBRDFs.
  • Assigned to Objaverse geometry and rendered in Mitsuba 3 → 12k training scenes.
pBRDF expansion pipeline

Generated pBRDF samples

Drag anywhere in the frame to move the split. All three maps are registered, so the crosshair reads one sphere across every attribute at once.

Citation

@article{choi2026snapshot,
  title     = {Snapshot Polarimetric Display Inverse Rendering},
  author    = {Choi, Seokjun and Moon, Yunseong and Kang, Kaizhang and Chung, Hoon-Gyu
               and Kim, Jin-Nyeong and Nam, Giljoo and Baek, Seung-Hwan},
  journal   = {ACM Transactions on Graphics},
  volume    = {45},
  number    = {6},
  articleno = {201},
  year      = {2026},
  doi       = {10.1145/3842531}
}