Debasish Mohanty

GPU & LLM Inference Performance & Infrastructure Engineer

I make LLM inference faster, cheaper and more reliable.

CUDA → vLLM → Kubernetes → AWS

I benchmark, profile and tune LLM inference on real GPUs: tensor parallelism, KV cache, quantization and CUDA kernels. Then I make it run reliably on Kubernetes and AWS.

Contributor to vLLM, Kubernetes and Falco.

Performance results

Multi-GPU LLM InferencevLLM · 2× T4

Tensor parallelism and queue tuning, measured end to end.

2.3× higher throughput (478 → 1,091 tok/s) with tensor parallelism on 2× T4.

300×+ lower p99 TTFT (37.1 s → 113 ms) by tuning max-num-seqs to eliminate queueing.

~93% higher goodput under SLO for only 4% higher cost/token, with 0 failures across 81 runs.

View on GitHub

CUDA Attention KernelTesla T4

A faster attention kernel, profiled with Nsight Compute.

1.81× faster attention (4.66 → 2.57 ms) via shared-memory tiling.

51% faster QK and 43% faster PV kernels, validated against a CPU reference.

Registers per thread cut 64 → 35. Nsight Compute isolates QK as the remaining bottleneck.

View on GitHub

Quantization BenchmarkGPTQ INT4 · Qwen2.5-3B · vLLM

Cheaper tokens without giving up accuracy.

1.6× higher throughput (448 → 711 tok/s) and 37% lower cost/token.

66% smaller weights, 53% more KV-cache capacity, 0 failures across 1,500 requests.

Accuracy within noise (GSM8K 72-78%, p > 0.3). Flagged an INT4 format regression (11/100).

View on GitHub

Open source

vLLM

KV offloading failed for models with mixed KV-cache groups, such as DeepSeek-V4-Flash and Gemma-4.

Added blocks_per_chunk for heterogeneous KV-cache groups, backward compatible with the existing block_size setting.

PR #48878

Distributed inference needed smarter request routing.

Co-authored cache-aware routing using KV-cache signals, with Nick Hill. Merged July 27.

PR #49204

Kubernetes Autoscaler

An Exoscale instance pool configured with minSize 3 was silently read as 1, causing under-provisioning.

Carried minSize and maxSize through. Backported to releases 1.32 and 1.33.

PR #9506

AWS scale-up from zero did not work with CSI volumes.

Fixed it by populating the CSI node topology.

PR #10133

OCI template nodes did not use the registered ephemeral-storage.

Used the node’s registered ephemeral-storage for the template.

PR #9993

Auto Scaling calls ignored the AWS SDK endpoint environment variables.

Honored those variables for the AWS Auto Scaling client.

PR #9992

Documentation links in the cluster-autoscaler README broke after the cloud provider migration.

Reported the issue.

Issue #10105

Kubernetes VPA

The admission controller failed on custom resources in Helm installs.

Restored the missing */scale RBAC permissions.

PR #9515

Falco

Memory metrics silently overflowed for processes using more than 4GB.

Fixed with 64-bit arithmetic.

PR #2930

File descriptors leaked on error paths in libscap.

Closed the directory handle on every failure path. Verified leak-free with Heaptrack.

PR #2926

Cluster API · Kubespray · Metrics Server · K8s website

Cluster API: stable BuildKit cache mount IDs.

PR #13930

Kubespray: updated the CSI sidecar image versions.

PR #13395

Metrics Server: clarified the kube-apiserver version skew policy.

PR #1769

Kubernetes website: clarified the control plane node description.

PR #56522

Experience

AI Systems InternAirDawg Labs (Caudal AI)
Aug 2026 - Sep 2026 · Remote

Built structured test suites to evaluate LLM systems on accuracy, reliability and safety, surfacing failure modes.

AWS Cloud InternTechEazy Consulting
Jul 2025 - Dec 2025 · Remote

Automated AI application infrastructure using Terraform and AWS, including VPC, EC2, IAM and S3.

AI Infrastructure InternElevate Labs · Best Performer
Apr 2025 - Jun 2025 · Remote

Built Kubernetes CI/CD and GitOps workflows for AI applications with Prometheus and Grafana observability.

Projects

LLM Inference Platform

LLM serving on AWS that costs nothing until you turn it on.

Terraform-built EKS with GPU Spot nodes scaled to zero, vLLM behind a FastAPI gateway, S3 model storage with IRSA, and GitHub Actions CI/CD.

View on GitHub

LLM Serving Platform

GPU LLM serving where cost and latency are measured, not guessed.

Production-grade vLLM on Kubernetes with full-stack observability and cost and latency benchmarking.

View on GitHub

GPU Compute Management

Learn to operate a GPU without paying for one.

Hardware health, driver and CUDA audit, VRAM budgeting, KV cache, quantization and batching. Runs mostly on zero GPU cost via simulation.

View on GitHub

AI Platform Infrastructure

A reusable AWS base for AI workloads.

Terraform, EKS and OIDC-based CI/CD for deploying containerized apps.

View on GitHub

Notes

Measure first

p99 TTFT dropped 300× from one queue setting. Tune what the profile shows.

API assumptions

The bug was in the API’s assumption, not the runtime.

PR #48878

Silent failures

Silently wrong is worse than crashing.

PR #9506

Low-level bugs

A bug that never crashes can still break security.

PR #2930

More on LinkedIn

Now

  • Working on cache-aware routing in vLLM.
  • Profiling and optimizing CUDA kernels and multi-GPU inference.
  • Building LLM serving and GPU tooling on AWS, in the open.

Updated October 2026

Terminal

Type "help" or tap a command below.

Let's talk.

debasishm8765@gmail.com

GitHub LinkedIn