Adds new depth AOV to camera classes

[kellyguo11]

Description

Please include a summary of the change and which issue is fixed. Please also include relevant motivation and context.
List any dependencies that are required for this change.

Fixes # (issue)

Type of change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (existing functionality will not work without user modification)
• Documentation update

Screenshots

Please attach before and after screenshots of the change if applicable.

Checklist

• I have read and understood the contribution guidelines
• I have run the pre-commit checks with ./isaaclab.sh --format
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• I have updated the changelog and the corresponding version in the extension's config/extension.toml file
• I have added my name to the CONTRIBUTORS.md or my name already exists there

[greptile-apps]

Greptile Summary

Adds support for a new depth_new data type that uses the Depth AOV (Arbitrary Output Variable) from the rendering pipeline. This provides an alternative depth representation alongside the existing distance_to_image_plane and distance_to_camera options.

Key Changes:

• Registered new Depth AOV annotator in both Camera and TiledCamera classes (temporary solution until officially exposed by replicator)
• Applied depth clipping behavior consistently with other depth types
• Added comprehensive tests for both camera types
• Created example environment configuration in cartpole tasks

Issues Found:

• Inconsistent clipping logic in tiled_camera.py - missing the initial clip-to-infinity step for depth_new that exists for distance_to_camera
• Potential duplicate annotator registration when multiple camera prims use depth_new (nested loop calls registration without checking if already registered)

Confidence Score: 3/5

• Safe to merge after addressing the clipping logic inconsistency
• Core functionality is well-tested and follows existing patterns, but has two logic issues: missing clipping logic in tiled camera creates inconsistent behavior, and potential duplicate registrations in multi-camera scenarios
• source/isaaclab/isaaclab/sensors/camera/tiled_camera.py needs the clipping logic fix, source/isaaclab/isaaclab/sensors/camera/camera.py should prevent duplicate registrations

Important Files Changed

Filename	Overview
source/isaaclab/isaaclab/sensors/camera/camera.py	Added depth_new AOV support with temporary annotator registration, consistent clipping logic applied
source/isaaclab/isaaclab/sensors/camera/tiled_camera.py	Added depth_new support but missing clipping logic that sets values > max to infinity, inconsistent with regular camera
source/isaaclab/test/sensors/test_camera.py	Added test for depth_new data type, validates shape and dtype correctly
source/isaaclab/test/sensors/test_tiled_camera.py	Added comprehensive test for depth_new in tiled camera, validates all expected outputs
source/isaaclab_tasks/isaaclab_tasks/direct/cartpole/init.py	Registered new environment variant for depth_new camera testing
source/isaaclab_tasks/isaaclab_tasks/direct/cartpole/cartpole_camera_env.py	Added configuration and observation handling for depth_new data type

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Camera Configuration] -->|data_types includes depth_new| B{Camera Type?}
    B -->|Camera| C[Register Depth AOV Annotator]
    B -->|TiledCamera| D[Register Depth AOV Annotator]
    C --> E[Create Annotator: Depth]
    D --> F[Create Annotator: Depth]
    E --> G[Attach to Render Product]
    F --> H[Attach to Render Products]
    G --> I[Update Buffers]
    H --> J[Update Buffers]
    I --> K[Clip values > max to inf]
    J --> L{Apply Clipping?}
    L -->|distance_to_camera| M[Clip to inf]
    L -->|depth_new| N[Missing Clip Step]
    K --> O[Apply depth_clipping_behavior]
    M --> O
    N --> O
    O -->|none| P[Keep inf values]
    O -->|zero| Q[Replace inf with 0.0]
    O -->|clamp| R[Replace inf with max]
    P --> S[Output depth_new data]
    Q --> S
    R --> S

Loading

_{Last reviewed commit: 5d3575c}

[greptile-apps]

_{6 files reviewed, 2 comments}

_{Edit Code Review Agent Settings | Greptile}

[greptile-apps]

annotator registration called inside nested loop (line 428: for cam_prim, line 459: for name) - if multiple cameras use depth_new, this registers the same annotator multiple times - wrap in a check or move outside loop

# Before the outer loop (around line 427), add:
_registered_aovs = set()

# Then in the inner loop:
if name == "depth_new" and "Depth" not in _registered_aovs:
    rep.AnnotatorRegistry.register_annotator_from_aov(
        aov="Depth", output_data_type=np.float32, output_channels=1
    )
    _registered_aovs.add("Depth")

[greptile-apps]

Additional Comments (1)

source/isaaclab/isaaclab/sensors/camera/tiled_camera.py
missing clipping logic for depth_new - in camera.py both distance_to_camera and depth_new get clipped to infinity, but here only distance_to_camera is handled

            if data_type == "distance_to_camera" or data_type == "depth_new":
                self._data.output[data_type][self._data.output[data_type] > self.cfg.spawn.clipping_range[1]] = (
                    torch.inf
                )