Synthetic Light for Real Sensors

Robots perceive the world through sensors. Camera, infrared, LiDAR, radar, sonar — each reads the world through a different physics: light, heat, radio waves, sound. Creating these sensors in simulation means computing that physics for real to synthesize a response. The common tool is ray tracing.

One Scene, Five Readings

The same scene is read by five sensors, each through its own physics.

Rendering diagram…

• EO (visible-band camera) — RTX path tracing renders physically accurate reflections, shadows, and material response.
• IR (infrared) — Models thermal emission as a function of object temperature and material emissivity. Humans and engine exhaust glow on an otherwise dark scene.
• LiDAR — Casts laser beams and computes round-trip range via ray-cast to build a point cloud. Weak-reflectance materials cause realistic drop-out.
• Radar — Emits radio waves and computes reflections based on each target's RCS (radar cross-section).
• Sonar (underwater) — Instead of light, sound. Instead of light rays, acoustic rays.

Tracing Sound Instead of Light — Underwater Acoustics

Underwater, light barely penetrates, so sonar replaces the camera. Sound speed varies with water temperature, salinity, and depth, causing acoustic rays to refract. Tracking that refraction and the resulting transmission loss (TL) is what acoustic models like BELLHOP do.

💡 The sonar equation — Sonar detection performance is computed via an acoustic budget such as SNR = SL − 2·TL + TS − NL − DT (active). Transmission loss is the critical output of the ray-tracing step. Just as the rendering equation builds an EO camera, the sonar equation builds a sonar.

⚠️ Do not mix modality equations — Active sonar (two-way propagation) and acoustic communication (one-way) use different equations. Applying a two-way formula to a one-way path makes the signal appear unrealistically weak. Even within "acoustics," the right equation depends on the propagation path structure.

Why Physics-Based?

Building sensors that are "roughly plausible" causes perception models trained on that data to fail in the real world. Physics-based synthesis that is consistent with environmental variables (weather, illumination, water temperature) is what enables models trained in simulation to generalize to real sensors. The goal of multi-sensor simulation is for multiple sensors to return consistent responses to the same event.

In a nutshell

📌 EO/IR/LiDAR/Radar/Sonar are each synthesized by tracing rays or acoustic paths through the true physics of light, heat, laser, radio, and sound. EO uses the rendering equation; sonar uses the acoustic equation — physics-based synthesis is what makes models trained on the data transfer to real sensors.