Projects · Nabin Prasad Dev

projects

Work That
Matters

Each project: Problem → Stack → Solution → Result. Real impact, measurable outcomes.

⭐ Featured · Interpretability · Research2026

FM Difficulty Probe — One Pipeline, Two Foundation Models: A Universal Predictive Null and an LLM-Specific Causal Signal

"Run the same experiment twice, in two worlds that share no architecture. What replicates is a property of SAEs. What doesn't tells you where the signal actually lives."

The Problem

Two separate studies, one on an autoregressive LM (Pythia), one on an encoder-based time-series FM (Chronos-T5) each asked whether TopK-SAE features encode a self-difficulty signal beyond a cheap baseline, and each found a negative result. But a single negative result invites the reviewer's reflex: "you just did it wrong." The unclaimed question in the 2026 literature isn't "does it fail on model X?" it's does the phenomenon generalize? That can only be answered by running the identical pipeline, with the identical metrics, across two unrelated modalities then seeing which findings survive the crossing and which are artifacts of one architecture.

The Stack

Backbones: Pythia-410M/2.8B (LLM) and Chronos-T5-small/base (TSFM), behind one interface.
Shared core (modality-agnostic, zero model code): TopK SAE (configurable expansion/k, aux dead-feature revival) · five-rung probe ladder (cheap → cheap+raw → cheap+SAE, +2 diagnostic isolations) · paired bootstrap (B=2000) ΔAUROC · label-permutation test · reconstruction-patch ablation with a single-vs-all-position coverage knob · risk-coverage/AURC · Platt + isotonic 5-fold OOF recalibration · cheap↔expensive cascade. Unit-tested on synthetic arrays.
Two thin adapters implementing one Modality Protocol: lexical stats vs classical TS stats; residual-stream vs encoder-block hooks (mid + late each); StratifiedKFold vs TimeSeriesSplit CV; Δnats vs ΔCRPS causal metric; binary-correctness vs continuous-CRPS error scale.
One config-driven runner: --modality {llm,tsfm} --experiment {probe,causal,selective,cascade,calibrate} → uniform json + parquet + PNG artifacts. Guardrails (refuses missing/random-init SAE and single-class labels), MPS threading, one unattended reproduce.sh.

The Solution

A merge-not-staple refactor: extract everything genuinely modality-independent into one tested core, make each model a ~150-line adapter, and gate the whole thing behind a regression check no new experiment was trusted until both legacy headline numbers reproduced exactly through the shared code (LLM SQuAD/L18 raw AUROC 0.716; TSFM Δ(SAE−cheap) −0.228). Only then were the three reconciliation experiments run: (1) promote the TSFM raw-activation middle rung to a first-class comparator so both modalities report the same three-rung ladder at both layers; (2) port the LLM's single-vs-all-position patching distinction to Chronos with the metric held fixed; (3) a SAE-expansion robustness sweep (4×↔8×). The entire results table every number, figure, and parquet regenerates from one command.

PythonPyTorchPythiaChronos-T5TopK SAEscikit-learnTransformersstatsmodelssafetensorsParquetMatplotlibLaTeX

Merged Repo (GitHub) →LLM Deep Dive (Pythia) ↗TSFM Deep Dive (Chronos) ↗

Impact & Academic Rigor

Modalities (autoregressive LM & encoder TSFM cross-modal replication, not two stapled reports)

Δ≤0 ×3

Predictive null replicates everywhere SAE adds no power over the strongest cheap rung in all 3 datasets, both layers (SQuAD −0.079 [−0.118,−0.041]; ETTh1 −0.228 [−0.366,−0.091])

5/5 vs 0/5

Causal signal is LLM-specific, Pythia features causally active under all-position patching (5/5), Chronos features silent under either coverage (0/5, at full 50-sample parity)

30–41%

Deployable artifact replicates a Platt-recalibrated cheap-baseline selective predictor captures 30% (TSFM) / 41% (LLM) of oracle AURC

4× ≡ 8×

Null is not an expansion artifact — Δ(SAE−raw) stays significantly negative at swapped SAE widths; the larger LLM SAE still doesn't beat raw

The contribution is the dissociation. Three legs of the original thesis were tested across modalities; two replicated and one split and the split is the result. The predictive null and the deployable selective predictor are universal; the causal contribution is specific to the autoregressive LM (where coverage-not-fidelity holds: single-position patching undershoots, all-position surfaces 5/5 features) and absent in the TSFM (predictively redundant and causally quiet, reproducing the legacy Chronos null exactly at full sample budget). A forced two-way replication would invite "you tuned it"; a universal null + a modality-specific positive is harder to dismiss and pinpoints where the signal lives. Engineered as a regression-gated, leakage-controlled, one-command-reproducible repo core/ carries zero model code and every function is unit-testable on synthetic arrays, so the cross-modal claim rests on the same code running in both worlds, by construction.

Not a third SAE benchmark, the experiment that turns two negative results into one falsifiable claim about where, and in which kind of foundation model, sparse features carry a causal difficulty signal.

⭐ Featured · LLM Systems · Research2026

Cascade Engine — Multi-Tier LLM Inference Router

"Not every query needs GPT-4o. Knowing which ones do is the hard part."

The Problem

Serving generative AI in production is incredibly expensive — frontier models like GPT-4o cost nearly 100× more per token than small or local models. But blindly routing all traffic to a cheap model destroys quality. Deciding which queries actually need premium reasoning is a hard routing problem, and in production, it's a moving target: cloud-model latency and reliability drift hour-by-hour with load, deprecations, and silent re-snapshots.

The Stack

Routing: Frugal cascade logic · Contextual Discounted Thompson Sampling (CD-TS) · Adaptive CD-TS · V_T-free drift tracking via two-sided CUSUM change-detection · Linear Thompson Sampling (LinTS) contextual bandit over prompt features/embeddings
Intelligent Layers: Microsoft Presidio (PII) · FAISS (semantic cache) · DistilBERT + VADER (gatekeeper)
Engines: Ollama — local llama3.2:3b · OpenAI — gpt-4o-mini / gpt-4o
Gateway: TypeScript + Express API gateway (API-key auth · rate limiting · batch · core failover)
Ops: pytest · vitest + supertest · GitHub Actions CI/CD (Python core and TS gateway) · LaTeX manuscript

The Solution

A deterministic + learned routing framework that triages every query through an intelligent pipeline before any expensive API call. PrivacyFilter (Presidio) redacts PII before it leaves the local network; SemanticCache (FAISS) returns highly-rated past answers via vector similarity, bypassing inference entirely; Gatekeeper (DistilBERT + VADER) scores syntactic complexity and intent to preemptively assign the cheapest capable tier; then Multi-Tier Routing dispatches to Tier 1 (local llama3.2:3b), Tier 2 (gpt-4o-mini), or Tier 3 (gpt-4o) only when warranted.

The learned router doesn't just split traffic, it adapts online as engine reliability drifts: CD-TS discounts stale evidence under a known variation budget, and Adaptive CD-TS detects abrupt breaks (outages, deprecations) via two-sided CUSUM and restarts the affected arm with no prior knowledge of the drift rate. A hardened TypeScript gateway fronts the Python core with auth, rate-limiting, and transparent failover.

PythonPyTorchDistilBERTFAISSPresidioOllamaOpenAIVADERTypeScriptExpresspytestvitestGitHub ActionsLaTeX

View on GitHub →Manuscript (LaTeX) →

Impact & Academic Rigor

60%

Cost Reduction

95%+

Quality Parity

100×

Frontier Cost Delta

Model Tiers

CD-TS

No-Regret Learned Router

Full evaluation pipeline on Alpaca-Eval generating statistically rigorous Pareto frontiers (Cost vs. Quality), benchmarked against RouteLLM and FrugalGPT.

The learned core is provably no-regret under non-stationary drift: CD-TS attains a sublinear pseudo-regret rate that matches the Besbes–Gur–Zeevi lower bound up to log factors, with a measured cross-horizon regret exponent of 0.20–0.55 versus a frozen offline-optimal baseline that degrades 25–33× under drift. The Adaptive CD-TS variant matches or beats the oracle-tuned discount without ever observing the drift budget V_T, and LinTS generalizes routing decisions across semantically similar prompts rather than fixed complexity bins. All formalized into a LaTeX academic manuscript with dual-language CI prepared for submission to top-tier ML/Systems venues (e.g. MLSys).

Not an LLM wrapper — the infrastructure that makes LLMs cheap enough, and reliable enough under real-world drift, for enterprises to actually deploy.

⭐ Featured · MLOps · Hospitality AI2025 / 2026

Hotel Revenue ML Platform

"7 years working in hotels showed me exactly what data was being wasted."

The Problem

Hotel management relies heavily on static, backward-looking reports. Critical revenue decisions — like overbooking limits and dynamic pricing — are often made via intuition rather than predictive data, resulting in lost revenue and operational inefficiencies.

The Stack

Backend: FastAPI · Python 3.11 · GZip + CORS middleware
ML Models: XGBoost (classification) · N-BEATS & Prophet (time-series forecasting) · KNN (recommender) · SHAP (explainability)
NLP: 3-tier sentiment pipeline — HuggingFace Inference API / Anthropic Claude / TextBlob
Ops & UI: Docker Compose · Streamlit · MLflow · GitHub Actions CI/CD (with automated metric-floor tests)

The Solution

A production-grade MLOps system featuring robust FastAPI endpoints. The core forecasting engine pits Prophet against a deep-learning N-BEATS baseline to predict ADR, Occupancy, and Revenue.

A rigorously evaluated XGBoost cancellation classifier drives an LP-based overbooking optimizer and a dynamic pricing engine. To ensure production viability, the model is evaluated exclusively on a temporal holdout set using walk-forward TimeSeriesSplit, and notorious target-leakage features (like booking_changes) from the public dataset were strictly removed.

The entire platform is containerized via Docker and heavily monitored with MLflow and nightly CI/CD API health checks. Built entirely from operational domain expertise earned at Nobu Hotel Toronto and Marriott.

Python 3.11FastAPIXGBoostNeuralForecast (N-BEATS)ProphetSHAPPuLPStreamlitMLflowDockerGitHub ActionsHuggingFace

View on GitHub →Live Demo ↗

Measurable Outcomes

81.4%

Honest AUC — leak-free, temporal walk-forward evaluation

15.0%

Occupancy MAPE

7.4%

ADR Forecast MAPE

18%

Scheduling Savings (projected)

3×

Faster Reporting

179,390 booking records ingested · Calibrated Brier Score 0.163 · Data quality grade B — models trained on real, messy operational data (not a clean toy set).

Hospitality domain expertise + ML engineering = solving problems most AI engineers don't even know to ask about.

Computer Vision · Group2024

Sign Language Translation

"Communication should never be a barrier."

Real-time ASL gesture classification using CNN + MediaPipe landmark extraction. Bridges communication gaps for the Deaf and hard-of-hearing community with multi-sign vocabulary support.

Real-time gesture classification · Multi-sign vocabulary support

PythonOpenCVCNNMediaPipeTensorFlow

Group Repo →

AI / GANs · Research2021

Earthquake Early Warning System

"National safety infrastructure, powered by ML."

GAN-based ML models developed at Nepal's national science institute to improve seismic infrastructure response modelling. Technical reports translated for government stakeholders to inform national safety protocols.

Presented to government officials · Informed national safety policy

TensorFlowGANsScikit-LearnNumPy

Smart City · IoT2024

IoT Traffic Signal System

"Vision Zero starts with smarter roads."

Computer Vision system predicting traffic signals and road hazards in real-time. Directly applicable to Toronto's Smart City initiatives, Vision Zero program, and congestion management infrastructure.

Real-time hazard detection · Vision Zero applicable

OpenCVYOLOIoTPredictive ML

Urban Policy · Research2024

Urban Green Open Space Policy

"Governance gaps cost cities their lungs."

Analyzed unplanned urbanization in Kathmandu Valley, mapping governance gaps between national and municipal authorities. Proposed a unified framework with direct applicability to Toronto's open space strategies.

Policy framework with Toronto comparative analysis

Spatial MappingPolicy AnalysisComparative Gov

Data Engineering2022

Infrastructure Data Platform

"Legacy systems were costing 30% of operational capacity."

Built end-to-end data analysis systems for a large investment conglomerate. Legacy database modernization and integrated cross-departmental data storage achieved a measurable 30% efficiency gain.

30% efficiency gain · Cross-departmental data integration

PandasNumPySQLPostgreSQL

Nabin Prasad Dev

AI/ML Engineer.

Engineer.
Researcher.
Urban Thinker.

What I Bring
to the Table

End-to-End ML Lifecycle

Where I've Been,
What I've Built

Work That
Matters

Thinking Out
Loud

Academic
Foundation

Through
My Lens

Got an Opportunity?
Let's Talk.

Nabin Prasad Dev

AI/ML Engineer.

Engineer.Researcher.Urban Thinker.

What I Bringto the Table

End-to-End ML Lifecycle

Where I've Been,What I've Built

Work ThatMatters

Thinking OutLoud

AcademicFoundation

ThroughMy Lens

Got an Opportunity?Let's Talk.

Engineer.
Researcher.
Urban Thinker.

What I Bring
to the Table

Where I've Been,
What I've Built

Work That
Matters

Thinking Out
Loud

Academic
Foundation

Through
My Lens

Got an Opportunity?
Let's Talk.