In real clinical workflows, doctors rarely provide explicit prompts like “segment the left kidney.” Instead, they raise implicit queries such as “What can be inferred from this shadow?” Existing MLLMs, though capable of vision-language interaction, still produce image-level outputs and rely heavily on handcrafted spatial prompts for grounding—inputs that are rarely available in practice.

Current datasets reflect this disconnect: VQA datasets lack spatial supervision, while segmentation datasets lack language. No existing dataset aligns implicit clinical queries with chain-of-thought reasoning and pixel-level localization, making it impossible to evaluate whether a model can truly reason and ground under realistic conditions.

To address the limitations of existing medical grounding systems, we define the Unified Medical Reasoning Grounding (UMRG) task, which challenges models to interpret implicit clinical queries, reason over visual and anatomical cues, and produce accurate pixel-level grounding—mirroring how clinicians observe, reflect, and pinpoint regions of interest in medical images. We tackle this task with a two-fold approach: (1) we construct U-MRG-14K, a dataset that pairs implicit queries with interpretable reasoning traces and pixel-level masks; and (2) we introduce MedReasoner, a reinforcement-learning framework that decouples reasoning from segmentation and grounds vague clinical language without relying on handcrafted spatial prompts.

We envision MedReasoner as a step toward trustworthy and generalizable medical grounding systems, enabling future clinical applications that demand both interpretability and spatial precision.

To support reasoning-based grounding under implicit clinical queries, we construct U-MRG-14K through a structured three-stage pipeline. This pipeline combines standardized medical data with GPT-4o–generated question–answer pairs and chain-of-thought reasoning traces, ensuring both semantic richness and spatial accuracy.

Our construction process emphasizes realism and interpretability: we simulate implicit clinical queries using GPT-4o, align them with precise pixel-level masks, and enrich each sample with structured reasoning traces. This design enables both language understanding and spatial evaluation in a unified setting. To our knowledge, U-MRG-14K is the first dataset to bridge implicit medical questions, chain-of-thought reasoning, and pixel-level grounding at scale.

While existing medical datasets either offer pixel-level masks or clinical question–answering pairs, none integrate implicit queries with chain-of-thought (CoT) reasoning and fine-grained spatial grounding. U-MRG-14K uniquely combines all three: it supports reasoning-aware evaluation with high-quality QA pairs grounded in pixel-level masks across diverse anatomical regions. It is the first dataset to bridge segmentation and medical VQA under realistic, implicit clinical language.

Our MedReasoner framework decouples language reasoning from visual segmentation, consisting of two modular components: a trainable Clinical Reasoning Module (CRM) that interprets implicit queries and predicts spatial prompts (a bounding box and two key points), and a frozen Anatomical Segmentation Module (ASM) that converts these prompts into high-resolution masks using MedSAM2. This design enables authentic reasoning without handcrafted spatial cues, avoids phrase overfitting, and supports plug-and-play compatibility with strong segmentation backbones.

To optimize the CRM, we design three categories of reward functions tailored to the UMRG task: (1) format rewards to enforce structured output, (2) box and point rewards to evaluate grounding accuracy, and (3) smoothing and penalization terms to ensure training stability and output plausibility. Together, these components guide the model toward reasoning-aligned spatial grounding. Extensive experiments confirm that MedReasoner achieves state-of-the-art performance on U-MRG-14K and generalizes well to unseen clinical queries.