Fresh Mode vs Daemon Mode

Isaac Sim takes time to boot. That naturally leads to a tempting thought — "If I keep it running and reuse it across runs, isn't that faster?" That temptation has a name: Daemon mode. Why we insist on Fresh mode instead is the subject of this post.

The Two Modes

Rendering diagram…

• Fresh mode — The simulator is booted fresh for every run and shut down when it finishes. Every run starts from a clean slate.
• Daemon mode — The simulator is booted once and multiple runs reuse the same process. Boot time is amortized.

The Daemon Trap — State Contamination

Daemon mode really is faster. But it has a fatal weakness: state persists between runs.

⚠️ Leftovers from the previous run contaminate the next one — When the same process is reused, prims, physics state, caches, and memory created by run N may not be fully cleaned up before run N+1 begins. That means run N+1's results are not purely run N+1's results — they are mixed with residue from run N. In validation, this is catastrophic: "Is this KPI actually caused by this configuration, or by leftover state from the prior run?" becomes unanswerable.

The lifeblood of a validation platform is reproducibility: the same inputs must always produce the same outputs (idempotency). Daemon's state contamination breaks that guarantee.

Weighing What Is Saved Against What Is Given Up

What daemon mode saves is boot time. Measured on one of our runs, across the whole span from queued to artifacts synced, loading was 9% (see run lifecycle). The other 91% is simulation plus scoring and output.

What that 9% costs is isolation — and isolation is the product. Run the same conditions twice and the same answer must come out. Reuse the process and one leftover prim, one leftover PhysicsScene, one leftover setting from the previous run can carry over, and that contamination does not raise an error. It just produces a different answer.

⚠️ The core problem is that there is no way to notice when reproducibility breaks. A crash is visible; "this answer shifted slightly because of residue from the previous run" is not. That 9% of boot time is what we pay to not buy that risk.

Why We Stick with Fresh

💡 Isolation > speed (in validation) — Shaving a few seconds off boot time at the cost of run-to-run isolation means trading all runs' trustworthiness for marginal throughput. Each run must execute in a fully independent, clean process so that its KPI can be attributed solely to its configuration. That is why the validation path uses Fresh mode exclusively — Daemon mode is not used even where its code remains.

This is the same principle as "stateless is better" in general software engineering. Stateful optimizations are fast but invite debugging nightmares. In a domain where correctness is everything — like validation — clean isolation is the right tradeoff even at the cost of speed.

In a nutshell

📌 Fresh mode (new boot per run, full isolation) vs Daemon mode (keep-alive reuse, faster). Daemon is faster but cross-run state contamination breaks reproducibility, so validation uses only Fresh mode despite the boot overhead — isolation matters more than speed.