Sharven Rane

ML engineer: GPU inference, model compilers, computer vision

I put computer vision models for medical radiographs into production and make them faster without changing their answers.

  • 1,000+

    production computer vision models deployed end to end

  • 30%

    lower production inference latency

  • 8M+

    radiographs used to train and validate vision models

Projects

Public repositories with measured results. Each one links to its code.

  • TensorRT C++ and CUDA(code on GitHub)

    Custom CUDA preprocessing kernels bring the C++ pipeline to 0.579 ms per image, 13.4× one CPU thread, and reproduce Pillow's antialiased resize exactly: 0 of 2,107,392 pixels differ. On the Python side, TensorRT FP16 runs 34.7× CPU FP32 at batch 8 with unchanged accuracy.

    • C++
    • CUDA
    • TensorRT
    • Nsight

    1,726 img/s

    end to end, C++ TensorRT FP16 with CUDA preprocessing

  • Kidney tumor CT segmentation(code on GitHub)

    3D CT segmentation built with MONAI and deployed with TensorRT FP16, evaluated at the validated Dice.

    • MONAI
    • 3D CT
    • TensorRT

    8.7×

    TensorRT FP16 over FP32, at equal Dice

  • DeepStream video analytics(code on GitHub)

    Multi stream detection and tracking on DeepStream 9.1, 3.3× a plain TensorRT Python loop on the same streams.

    • DeepStream
    • TensorRT

    1,516 FPS

    across 16 streams at 1080p

  • Compiler correctness(code on GitHub)

    Differential fuzzing of constant folding in TensorRT's ONNX GraphSurgeon against ONNX Runtime and TensorRT 10.16. Fixes and tests are in review upstream.

    • Differential fuzzing
    • ONNX
    • TensorRT

    4

    classes of silent wrong output bugs found, fixes in review

  • Radiology report triage with an LLM(code on GitHub)

    LoRA SFT on Qwen2.5 1.5B for triage of public chest X-ray reports, against 0.918 for logistic regression on sentence embeddings. DPO added no gain.

    • LoRA
    • SFT
    • DPO
    • Qwen2.5 1.5B

    0.959

    balanced accuracy, mean of 3 seeds

  • Whole slide pathology(code on GitHub)

    Max pooling multiple instance learning on whole slide images.

    • MIL
    • CAMELYON16

    0.909

    test AUC on CAMELYON16

  • Surgical phase recognition(code on GitHub)

    A causal MS-TCN, so every prediction uses only frames already seen.

    • MS-TCN
    • Cholec80

    0.888

    test accuracy on Cholec80

  • Distributed ML and MLOps(code on GitHub)

    Ray Data batch inference on 104k CAMELYON16 patches, where block size decided throughput more than worker count did. Plus Airflow 3 pipelines, Terraform and Kubernetes.

    • Ray
    • Airflow 3
    • Terraform
    • Kubernetes

    3.2 to 3.6×

    faster at 32 MB blocks than Ray Data's default, same 16 decode workers

GPU inference and compilers

Making models fast on NVIDIA GPUs, and checking that the fast version still gives the same answer.

  • C++ TensorRT engines and custom CUDA kernels, profiled with Nsight
  • Cut production inference latency 30%
  • Differential fuzzing found 4 classes of silent wrong output bugs in TensorRT's ONNX GraphSurgeon constant folding; fixes are in review

Upstream: 15 of the 19 open source pull requests are in compilers and exporters (TensorRT, PyTorch, Torch-TensorRT, ONNX Script, LiteRT Torch, coremltools), all in review.

Medical imaging

Production computer vision for radiographs, plus public work on CT, pathology slides and surgical video.

  • Production computer vision for medical radiographs
  • 1,000+ production models deployed end to end
  • Vision models trained and validated on 8M+ radiographs

Open source

Bug fixes upstream in the compilers and exporters that production models depend on.

19 bug fix pull requests upstream (18 in review, 1 merged) across 10 projects: silent wrong outputs, invalid exports and crashes in the compilers and libraries that production models run on.

Scratchpad

My notebook: comparisons, engineering notes, opinions and whatever I am geeking out on.

Open Scratchpad

No posts published yet. Code and fixes are on GitHub in the meantime.

Contact

Email is the fastest way to reach me.

Now
ML engineer in production computer vision for medical radiographs.
Research interests
  • Reliability of models after deployment optimization: precision, compilers and runtimes
  • GPU inference
  • Medical imaging
Education
MS Data Science, Stevens Institute of Technology. Ranked first in cohort.
Location
United States

Sharven Rane

ML engineer: GPU inference, model compilers, computer vision

Click anything on the desk, or use the menu above.