Assessment Methods

Evaluation focused on generalization, not leaderboard gaming

The iMED Challenge uses conventional but clinically meaningful metrics: geometric accuracy for pose estimation and photometric fidelity for deformable novel view synthesis, all computed on hidden test scenes.

Primary ranking metrics

iMED PE

Primary ranking metric: Absolute Trajectory Error (ATE)

Also reported: relative rotational error (ROT), registered tracking fraction (RTF), runtime transparency

iMED NVS

Primary ranking metric: PSNR

Also reported: SSIM, training FPS, rendering FPS

Ranking procedure

Task Per-sequence computation Aggregate rank Fallback for missing outputs
iMED PE ATE, ROT, and valid-frame fraction computed sequence-by-sequence Mean ATE over all hidden test sequences Missing outputs count as failure cases with capped error and zero tracking fraction
iMED NVS PSNR and SSIM computed per frame, then averaged per sequence Mean PSNR over all hidden test sequences Missing outputs contribute zero-quality frames in the aggregate

Data usage policy

External data

Publicly available external datasets and pretrained models are allowed, provided their use is documented clearly in the submitted method description.

Private data

Private clinical data and other non-public institutional datasets are not allowed for training, tuning, or evaluation.

Validation and hidden test philosophy

  • Participants are expected to self-evaluate on public development data using official scripts.
  • Intermediate validation submissions are planned before the final test phase.
  • Final rankings are computed only on hidden test scenes that remain unseen to participants.

Prizes and awards

Each task will recognize first-, second-, and third-place teams with monetary awards. Final prize amounts will be announced on the challenge website.

Publication policy

Teams with valid submissions are eligible for inclusion in the joint challenge paper, with up to three named contributors per team depending on total participation. Teams must submit a structured method report and may later publish separate papers that cite the official challenge publication.

Suggested public wording to keep

Public leaderboard

Display the official metrics and the best valid submission per team, with organizer-affiliated teams clearly labeled.

Top methods announcement

Publicly announce the top three external teams for each task during EndoVis / MICCAI and on the challenge website.