97% accuracy on fungal contamination detection with SAM

In plant pathology screening, the bottleneck isn't growing the samples, it's reading them. A single image holds around twenty petri dishes, each with a leaf, and every leaf needs a verdict: infected or clean, and how bad. Trained eyes on that many plates is slow, subjective, and doesn't scale. We built a computer-vision pipeline that goes from that crowded image to per-leaf infection maps at 97% accuracy. Here's the actual pipeline, stage by stage.

Dishes first, leaves second, fungus third

The input is one wide image containing roughly twenty petri dishes. The pipeline decomposes it in stages, and that decomposition is most of the engineering:

Dish extraction. First, find and crop each dish out of the full image. One messy, information-dense image becomes twenty clean, uniform inputs.
Leaf extraction. From each dish, segment the leaf away from agar, dish edges, and background. From here on, the model only ever sees leaves.
Batched inference. All the extracted leaves go to the ML model together. The model identifies fungal regions on each leaf.
Overlay. The detected fungal regions are drawn back onto the leaf image in a contrasting color. This is the difference between a black-box score and something a pathologist can verify at a glance: the model's claim is visible, right on the evidence.
Area quantification. Finally, compute the fungus-affected area per leaf. Detection answers "is it infected"; area answers "how bad", and severity is what actually drives decisions: keep or discard, treatment priority, resistance scoring.

Each stage shrinks and standardizes the problem for the next one. The fungus detector never has to reason about dish rims, labels, or agar texture. That staging is a large part of why a modest amount of labeled data went as far as it did.

Why SAM

The segmentation backbone is SAM, the Segment Anything Model. The classic alternative, training a segmentation model from scratch on contamination imagery, dies on data: every label needs a trained eye, and there are never enough of them. SAM already understands "what is an object" from massive pretraining, so it delineates colony boundaries and morphology out of the box. Our learning problem shrank from "learn to see" to "learn what fungal contamination looks like": smaller, cheaper, and far more tractable. For niche scientific imaging, that leverage is the whole game: foundation models let small teams punch far above their dataset size.

Getting to 97%

There was no single breakthrough; it was a grind of small wins, and the biggest jumps came from data work, not architecture work. Cleaning label noise. Making sure edge cases were represented: early-stage growth that's barely visible, atypical colony morphology, imaging artifacts that look like growth but aren't. And building an evaluation set that reflected production conditions (twenty dishes per image, variable leaf placement, imperfect lighting) instead of a curated gallery of easy cases. A test set of clean examples will lie to you; ours wasn't allowed to.

Deployed on Azure ML

A model that only runs in a notebook helps nobody. We deployed on Azure ML for training and experiment tracking, with serving on Azure Container Apps, containerized inference that scales with screening volume. Around it, real MLOps: versioned models, reproducible runs, and a retraining path as new labeled data comes in. That last part matters: strains and imaging setups drift, and a deployed model without a retraining story is a model with an expiration date. For inference performance, we evaluated NVIDIA Triton Inference Server, which paid off on throughput when batching many leaves per request. The right serving stack depends on whether your bottleneck is latency or volume, and we measured both.

We kept a human in the loop for low-confidence predictions. The cost of a missed infection dwarfs the cost of a false alarm, so borderline leaves get flagged for human review rather than auto-cleared. Designing that threshold explicitly, instead of defaulting to the model's argmax, was one of the most important product decisions in the project. Monitoring closes the loop: we track prediction confidence distributions and flag drift, so if incoming image characteristics shift (a new camera, a different dish supplier), the team hears about it before accuracy silently degrades.

Under the hood

The stack behind the pipeline is deliberately boring in the best way: proven pieces, composed carefully:

Segmentation → SAM. The Segment Anything Model does the heavy visual lifting: delineating dish boundaries, leaf contours, and fungal colony morphology. Pretrained on massive data, it gives us object boundaries out of the box, so our labeled data goes toward the domain problem (what contamination looks like) instead of teaching a model to see.
Pipeline → staged Python CV. Dish extraction, leaf segmentation, batched inference, overlay rendering, area computation. Each stage is a discrete, testable step. If the overlay looks wrong, we know exactly which stage to blame. Monolithic end-to-end models don't give you that.
Deployment → Azure ML + Container Apps, with MLOps. Training tracked and versioned in Azure ML; serving on Azure Container Apps for scalable, containerized inference. Full MLOps around it: versioned models, reproducible runs, automated retraining as new labeled data lands, and prediction-confidence monitoring that catches drift (a new camera, a different dish supplier) before accuracy silently degrades. We also evaluated NVIDIA Triton Inference Server for higher-throughput serving, squeezing more inferences per GPU when batching dense workloads.

The pattern worth stealing: foundation model for perception, classical pipeline for structure, managed ML platform for operations, served on containers, with Triton on the table when throughput matters. Each layer does what it's good at, and no layer is asked to do another's job.

Practical lessons

Decompose the image before you classify it. Dish → leaf → fungus. Each stage hands the next a cleaner problem.
Make outputs verifiable. An overlay a pathologist can check beats a score they have to trust.
Quantify, don't just detect. Severity (area) is the decision variable; presence/absence is just the headline.
Data beats architecture. Nearly every accuracy gain came from better labels and harder evaluation.
Design for drift. Ship the retraining pipeline alongside the model, not after.

Vision in bioprocess isn't about chasing state-of-the-art on a benchmark; it's about building something reliable enough that a screening team trusts it with real samples. Staged decomposition got us leverage, SAM got us there faster than training from scratch ever could, Azure ML keeps it running reproducibly, and the unglamorous data work is what made 97% real instead of theoretical.