Evaluation focused on generalization, not leaderboard gaming
The iMED Challenge uses conventional but clinically meaningful metrics: geometric accuracy for pose estimation and photometric fidelity for deformable novel view synthesis, all computed on hidden test scenes.
Primary ranking metrics
iMED PE
Primary ranking metric: Absolute Trajectory Error (ATE)
Also reported: relative rotational error (ROT), registered tracking fraction (RTF), runtime transparency
iMED NVS
Primary ranking metric: PSNR
Also reported: SSIM, training FPS, rendering FPS
Ranking procedure
| Task | Per-sequence computation | Aggregate rank | Fallback for missing outputs |
|---|---|---|---|
| iMED PE | ATE, ROT, and valid-frame fraction computed sequence-by-sequence | Mean ATE over all hidden test sequences | Missing outputs count as failure cases with capped error and zero tracking fraction |
| iMED NVS | PSNR and SSIM computed per frame, then averaged per sequence | Mean PSNR over all hidden test sequences | Missing outputs contribute zero-quality frames in the aggregate |
Data usage policy
External data
Publicly available external datasets and pretrained models are allowed, provided their use is documented clearly in the submitted method description.
Private data
Private clinical data and other non-public institutional datasets are not allowed for training, tuning, or evaluation.
Validation and hidden test philosophy
- Participants are expected to self-evaluate on public development data using official scripts.
- Intermediate validation submissions are planned before the final test phase.
- Final rankings are computed only on hidden test scenes that remain unseen to participants.
Prizes and awards
Each task will recognize first-, second-, and third-place teams with monetary awards. Final prize amounts will be announced on the challenge website.
Publication policy
Teams with valid submissions are eligible for inclusion in the joint challenge paper, with up to three named contributors per team depending on total participation. Teams must submit a structured method report and may later publish separate papers that cite the official challenge publication.
Suggested public wording to keep
Public leaderboard
Display the official metrics and the best valid submission per team, with organizer-affiliated teams clearly labeled.
Top methods announcement
Publicly announce the top three external teams for each task during EndoVis / MICCAI and on the challenge website.