Research
Two DOI-archived papers, with a real control group
The featured study tests a real question with a proper baseline and significance testing, not one flattering number.
MedInsight: Evaluating Retrieval-Augmented Vision-Language Models for Evidence-Grounded Medical Image Understanding
Does giving a medical vision-language model retrieved similar-case evidence actually make it more accurate — or does it just look like it should?
Abstract
MedInsight tests whether retrieval-augmented prompting improves a vision-language model's accuracy on medical visual question answering, using only openly licensed, non-credentialed data. A CLIP-embedded FAISS index over ~90,000 ROCOv2 radiology image-caption pairs supplies retrieved evidence captions to a pretrained BLIP-2 model, and that retrieval-augmented condition is compared against the same model answering with no retrieved evidence at all — same checkpoint, same decoding settings, same scoring code, on the official VQA-RAD test split (451 clinician-authored questions), with paired bootstrap significance testing rather than a single reported delta.
Contributions
- A fully reproducible retrieval-augmented medical VQA pipeline built entirely on open, non-credentialed datasets — no data-use agreement required to run it end to end
- A controlled baseline-vs-RAG comparison that shares one generation code path between both conditions, so any measured difference can be attributed to retrieval itself rather than an implementation quirk
- Statistical rigor beyond a raw accuracy delta: paired bootstrap significance testing (10,000 resamples) plus a qualitative improved/regressed/unchanged error breakdown
Methodology
CLIP-embedded FAISS flat inner-product index over the full ROCOv2 corpus (~90K radiology image-caption pairs)
Shared generation code path (BaselineVLM.generate_from_prompt) for both the baseline and retrieval-augmented BLIP-2 conditions — identical tokenization, decoding, and scoring
Evaluation on VQA-RAD's official 451-example held-out test split, scored by exact match (closed questions) and BLEU-4/ROUGE-L (open questions)
Paired bootstrap significance testing (10,000 resamples) plus a sweep over retrieval depth k ∈ {1, 3, 5, 10}
Key results
Closed-question accuracy
Overall exact match
Significance (closed EM)
Best retrieval depth
Datasets
- ROCOv2 (retrieval corpus)
- VQA-RAD (evaluation benchmark)
Stack
Findings
- Retrieval augmentation improved every reported metric at the default depth (k = 5), but paired bootstrap testing found neither the closed-question nor open-question improvement statistically significant — an honest result more useful than an overstated one
- Accuracy peaked at the smallest retrieval depth (k = 1) and declined monotonically through k = 10, suggesting excess retrieved evidence can dilute rather than reinforce a base model's answer
- Error analysis showed both conditions failed on the majority of questions (58%) — the underlying VLM's zero-shot capability on radiology imagery, not retrieval, is the primary bottleneck
Limitations, stated plainly
- VQA-RAD's 451-example test split is small, which is why significance testing was treated as a requirement rather than an optional check
- The retriever matches only on image similarity — the question text never participates in retrieval — a plausible confound with the k-depth findings
- A single BLIP-2 checkpoint was evaluated at a single scale; no clinician review of answer quality was performed, so results describe this pipeline, not a validated clinical tool
Earlier work
Other published research
AI-Based Depression Detection from Social Media Using LSTM Networks
This research investigates whether Long Short-Term Memory (LSTM) networks can pick up linguistic and behavioral indicators of depression in social media text more reliably than simple bag-of-words baselines. NLP preprocessing (tokenization, cleaning, stopword removal, embedding generation) feeds a sequence model trained to classify posts as depression-indicative or not, with class-imbalance handling and standard classification metrics.