Attaching Sensors

For a robot to perceive its environment, it needs sensors. In Isaac Sim, sensors are created as prims in the scene and flow out to ROS2 topics via OmniGraph. But even the same sensor type — LiDAR — comes in two fundamentally different flavors, and not knowing the difference will cost you time.

Two Kinds of LiDAR — PhysX vs RTX

Rendering diagram…

• PhysX LiDAR — Pure geometric ray-casting for range measurement. Lightweight and fast.
• RTX LiDAR — Goes through the RTX render pipeline to reproduce real sensor imperfections: weak returns from low-reflectance materials, drop-outs, and multipath ghosts. Requires rendering.

RTX LiDAR reads the render output through an annotator and converts it into a point cloud. Choose RTX when physical fidelity matters; choose PhysX when you just need range data.

Camera, IMU, and Contact

Other sensors follow the same pattern:

• Camera — Receives a render product and publishes it as an image topic.
• IMU — Reads linear acceleration and angular velocity from a rigid body and publishes inertial data.
• Contact — Contact sensor for collision/ground-contact detection.

💡 Sensors travel with the robot USD — Attach sensor prims and their publishing graphs to the robot USD, and those sensors come along every time the robot is spawned into any environment. This is the same philosophy as adding a robot with a single manifest.

A Declared Sensor Is Not an Attached Sensor

Sensors are declared in the manifest with a name, a type and a mounting pose. Between declaration and attachment sits one gate: does the runtime recognise that type name? Here is the actual record from a run using our Husky manifest.

[RdpManifestLoader] Loaded manifest ... 3 sensors, drive_type=differential
[Isaac] Created RTX LiDAR: config=OS1, pos=[0.0, 0.0, 0.35], rate_hz=10.0
[Isaac] IMUSensor created+initialized: /World/Robots/HO_A/base_link/imu_sensor
[Isaac] WARNING: Unknown sensor type 'gps' ... — skipped.
          Supported: lidar, lidar_3d, camera_rgb, camera_depth, imu, gnss, gnss_rtk, radar, thermal
[Isaac] Configured 2 sensors for ROBOT_CLEARPATH_HUSKY_A200: HO_A

Three sensors declared in the manifest, two attached at runtime. The gps entry is skipped with a single warning line because the runtime knows that sensor as gnss. The run does not fail — it completes normally, reporting 240 LiDAR scans totalling 11,805,070 points.

The frightening outcome here is not failure but success. The run finishes, a verdict is produced, a report is issued — the verdict simply has no GPS in it. "I mounted a sensor and no data came out" is a much cheaper problem than "I read a result as normal without knowing a sensor was missing."

⚠️ Count sensors by attachments, not declarations. Writing "three instrumented channels" on the strength of three manifest entries is wrong. Configured N sensors in the log is the real number, and any gap between the two is something that got skipped.

Pitfall — RTX Sensors Require Rendering Even in Headless Mode

⚠️ "It's headless, so I can turn off rendering" — this is wrong — RTX LiDAR, depth annotators, and semantic annotators all consume render output even with no display window. Rendering must remain active. This trap — "headless = no-render" — is covered in more depth in a separate post.

In a nutshell

📌 Sensors are created as scene prims. PhysX LiDAR (fast ranging) and RTX LiDAR (physically accurate, point cloud via annotator) are fundamentally different. Camera, IMU, and contact sensors follow the same pattern, and RTX sensors require rendering even in headless mode.