Design: Retrospective validation study

Setting: Thirteen surgeon raters with ≥3 years of experience in abdominopelvic surgery and without red/green color blindness were trained to use the Likert scale.

Patients or Participants: Seventy-two color images (36 pairs) were obtained from the phase 2 study (NCT04238481) in which adults undergoing abdominopelvic surgery received single doses of ASP5354. Image pairs showed the same anatomical area containing the ureters viewed with NIR-F illumination (“signal”) and white light (“no signal”). Images were taken at the beginning (0–30 min after administering ASP5354, n=18 pairs), middle (between 30 min post-ASP5354 and end of surgery, n=13 pairs), and end of surgery (n=5 pairs). Images were displayed for 30 seconds and ratings were collected.

Interventions: N/A

Measurements and Main Results: Intraclass correlation coefficients (ICC) were used to evaluate inter-rater reliability (based on surgeons’ ratings at baseline) and intra-rater reliability (based on consistency between baseline and retesting 1 week later). ICC cut-off values were ≥0.70 for inter-rater and ≥0.80 for intra-rater reliability. Our study found good inter-rater (ICC=0.7; 95% confidence interval [CI]=0.7, 0.8) and intra-rater reliability (0.8; 95% CI=0.8, 0.8).

Conclusion: Surgeons using the 5-point Likert scale for assessing ureter conspicuity in patients who received ASP5354 during abdominopelvic surgery reached statistically significant levels of inter- and intra-rater reliability as required by FDA standards.