DOC 04PERCEPTION

Perception ground truth

A simulator knows what was really there. LEVIATHAN turns that knowledge into ground truth with stated definitions: per-pixel geometry and labels, how each visible surface point moved between frames, which object it belongs to over time, and which surfaces a sonar return came from.

All products on this page are computed from immutable snapshots of the canonical World (see Time, frames and determinism). They are content-addressed and declare their units and frames. They come from the simulation's own geometry and identity, never from matching images.

Surface identity

Motion, track and cross-modal ground truth all rest on one definition of "the same physical surface point". A visible sample is identified by:

• its owner, the persistent scene-entity identifier;
• its geometry, the owner's content hash, which pins triangle and vertex order;
• its element, the node, mesh and primitive within that geometry;
• its triangle, in the primitive's own source order;
• its barycentric coordinates (b₀, b₁, b₂) on that triangle, in float64, unclamped.

Frame-local instance ranks, world-space proximity, rendered colours and anything the Viewport displays are never used. Identity is stable across repeated loads, cache eviction, scene ordering, and static versus dynamic placement of the same asset. Only scanned-asset and procedural geometry has this identity. Surfaces without it are reported as unsupported geometry.

Surface correspondence

For every pixel of camera A, correspondence identifies the surface sample the pixel sees and where that same sample is at snapshot B. That means its world point, its projection into camera B with distortion applied, its range and z-depth from B, and whether camera B actually sees it. The two snapshots must come from the same sensor. Poses come only from the snapshots, so no velocity is needed. The forward product requires t_B > t_A. A backward product is computed separately, never by negating the forward one.

Optical flow and scene displacement

Optical flow and scene displacement are derived directly from correspondence, the single geometric authority. Each direction is its own product on its source pixel grid.

• flow_px is in pixels over the exact interval between the snapshots, ordered (du, dv). It is not divided by time and it is not a motion field.
• scene_displacement_world_m is the same material point's displacement in the NED world frame, in metres. It is exactly zero for static geometry whatever the camera does. For a moving rigid body it is that body's own motion only.
• valid marks pixels whose point is visible at B, which are valid flow supervision. occluded comes from the exact re-cast, never from photometry. disoccluded exists only on the backward product and marks points the earlier camera did not see.

The legacy optical-flow camera, an ego-motion approximation in the analytic sensor set, is superseded by this product and is not used by it.

Instantaneous motion field

The motion field is the instantaneous image velocity of the material point each pixel sees, at one snapshot:

It is computed from over-ground twists recorded in the snapshot, never from finite differences of poses. The inputs are the camera owner's twist, carried through the lever arm to the camera, and each moving rigid body's twist. Static points have zero velocity. As the interval shrinks, flow divided by the interval tends to the motion field. The product also gives each point's world velocity, its velocity relative to the camera, and its z-depth.

Object and track ground truth

Track ground truth follows objects through time for one camera, using the simulation's own identity. No overlap, appearance, centroid or proximity rule is involved.

• A track is one life of one scene entity: the run of consecutive samples in which it is present with one geometry. Track identifiers are <entity id>#<life>. The per-frame instance channel is a rank within that frame; it is linked to tracks through the entity identifier.
• Each observation records the track's world facts (presence, pose, twist) separately from the camera facts: visible, occluded, beyond range, out of frame or behind camera.
• Each track has a representation state: imaged, excluded, unrepresented, unsupported or unavailable.
• Events mark lifecycle and visibility changes: born, died, replaced, entered or exited the frame, became occluded or visible, left or entered range, crossed the camera plane, and representation changed.
• Boxes are inclusive pixel bounds [u_min, v_min, u_max, v_max], with pixel centres at integers.
• Authored components of annotated assets are tracked as components. An unknown semantic class is kept as 65535 and means ignore, never a guess.

Camera–sonar contributor ground truth

A sonar cell is a sum over the elevation samples that land in it, so it is not a pixel, and there is no honest pixel-to-cell map. Instead, LEVIATHAN provides the complete contributor table of one sonar measurement. Each row is one sampled sonar ray's hit, carrying:

• its surface identity (element, triangle, barycentrics) and its object track;
• its sonar coordinates (cell, range, elevation, incidence) and relative return;
• the camera's projection of the same material point, with the correspondence visibility status at the camera.

Summing the rows reproduces the measurement bit for bit: each cell's linear_return, its primary and top-K contributors, its contributor count, its truncated fraction and its validity. A calibration record holds the camera and sonar poses. Per-track aggregates summarise how much of the return each object produced.

Sonar observability

For every object track present at the camera's sample, observability states whether the sonar observes it acoustically. This is a physical fact about the sonar, independent of the camera's visibility.

Queries go from camera samples to sonar support. Same object asks whether the sonar sees the object a pixel belongs to. Same triangle asks whether it sees the very surface element. Acoustic coordinates asks where in the sonar image the pixel's surface point falls.

Asynchronous camera–sonar ground truth

Under review Real cameras and sonars rarely sample at the same instant. A separate set of products handles that case. They are implemented and exposed by the simulation service, but are still under internal review, so their contracts may change.

• Temporal pairing. Records which actual camera measurement and which actual sonar measurement of a capture set were selected for comparison, by what policy, and their exact time offset. Pairing is selection only: it asserts no correspondence and interpolates nothing.
• Sonar to camera. Each sampled sonar contributor's material point is carried to the camera's time using the recorded poses of its rigid owner, then checked against the camera's actual view at that time.
• Camera to sonar. A camera pixel's material point is carried to the sonar's time and projected into acoustic coordinates. That projection is kept separate from which sonar samples actually hit the same object or triangle.
• Temporal datasets. These package a capture set, its pairing and the selected records, with exact-time records embedded as exact-time products and asynchronous ones per direction.

Transport is defined for static and rigid owners. Deforming or articulated geometry, changed geometry and changed tracks are reported with explicit statuses. No "corresponding sonar pixel" is ever claimed.