Sharven Rane
ML engineer: GPU inference, model compilers, computer vision
I put computer vision models for medical radiographs into production and make them faster without changing their answers.

1,000+
production computer vision models deployed end to end
30%
lower production inference latency
8M+
radiographs used to train and validate vision models
Projects
Public repositories with measured results. Each one links to its code.
TensorRT C++ and CUDA(code on GitHub)
Custom CUDA preprocessing kernels bring the C++ pipeline to 0.579 ms per image, 13.4× one CPU thread, and reproduce Pillow's antialiased resize exactly: 0 of 2,107,392 pixels differ. On the Python side, TensorRT FP16 runs 34.7× CPU FP32 at batch 8 with unchanged accuracy.
- C++
- CUDA
- TensorRT
- Nsight
1,726 img/s
end to end, C++ TensorRT FP16 with CUDA preprocessing
Kidney tumor CT segmentation(code on GitHub)
3D CT segmentation built with MONAI and deployed with TensorRT FP16, evaluated at the validated Dice.
- MONAI
- 3D CT
- TensorRT
8.7×
TensorRT FP16 over FP32, at equal Dice
DeepStream video analytics(code on GitHub)
Multi stream detection and tracking on DeepStream 9.1, 3.3× a plain TensorRT Python loop on the same streams.
- DeepStream
- TensorRT
1,516 FPS
across 16 streams at 1080p
Compiler correctness(code on GitHub)
Differential fuzzing of constant folding in TensorRT's ONNX GraphSurgeon against ONNX Runtime and TensorRT 10.16. Fixes and tests are in review upstream.
- Differential fuzzing
- ONNX
- TensorRT
4
classes of silent wrong output bugs found, fixes in review
Radiology report triage with an LLM(code on GitHub)
LoRA SFT on Qwen2.5 1.5B for triage of public chest X-ray reports, against 0.918 for logistic regression on sentence embeddings. DPO added no gain.
- LoRA
- SFT
- DPO
- Qwen2.5 1.5B
0.959
balanced accuracy, mean of 3 seeds
Whole slide pathology(code on GitHub)
Max pooling multiple instance learning on whole slide images.
- MIL
- CAMELYON16
0.909
test AUC on CAMELYON16
Surgical phase recognition(code on GitHub)
A causal MS-TCN, so every prediction uses only frames already seen.
- MS-TCN
- Cholec80
0.888
test accuracy on Cholec80
Distributed ML and MLOps(code on GitHub)
Ray Data batch inference on 104k CAMELYON16 patches, where block size decided throughput more than worker count did. Plus Airflow 3 pipelines, Terraform and Kubernetes.
- Ray
- Airflow 3
- Terraform
- Kubernetes
3.2 to 3.6×
faster at 32 MB blocks than Ray Data's default, same 16 decode workers
GPU inference and compilers
Making models fast on NVIDIA GPUs, and checking that the fast version still gives the same answer.
- C++ TensorRT engines and custom CUDA kernels, profiled with Nsight
- Cut production inference latency 30%
- Differential fuzzing found 4 classes of silent wrong output bugs in TensorRT's ONNX GraphSurgeon constant folding; fixes are in review
Related projects
Upstream: 15 of the 19 open source pull requests are in compilers and exporters (TensorRT, PyTorch, Torch-TensorRT, ONNX Script, LiteRT Torch, coremltools), all in review.
Medical imaging
Production computer vision for radiographs, plus public work on CT, pathology slides and surgical video.
- Production computer vision for medical radiographs
- 1,000+ production models deployed end to end
- Vision models trained and validated on 8M+ radiographs
Open source
Bug fixes upstream in the compilers and exporters that production models depend on.
19 bug fix pull requests upstream (18 in review, 1 merged) across 10 projects: silent wrong outputs, invalid exports and crashes in the compilers and libraries that production models run on.
PyTorchpytorch/pytorch
Torch-TensorRTpytorch/TensorRT
Microsoft ONNX Scriptmicrosoft/onnxscript
Google LiteRT Torchgoogle-ai-edge/litert-torch
Apple coremltoolsapple/coremltools
torchvisionpytorch/vision
Amazon Chronosamazon-science/chronos-forecasting
AutoGluonautogluon/autogluon
Project MONAI MONAILabelProject-MONAI/MONAILabel
Scratchpad
My notebook: comparisons, engineering notes, opinions and whatever I am geeking out on.
Open ScratchpadNo posts published yet. Code and fixes are on GitHub in the meantime.
Contact
Email is the fastest way to reach me.
- Now
- ML engineer in production computer vision for medical radiographs.
- Research interests
- Reliability of models after deployment optimization: precision, compilers and runtimes
- GPU inference
- Medical imaging
- Education
- MS Data Science, Stevens Institute of Technology. Ranked first in cohort.
- Location
- United States