portfolio

A driver-safety training and risk-scoring perception platform, built and deployed end to end: a hardware-in-the-loop driving simulator with a custom secondary physics layer, and a Jetson NX in-vehicle device running the real-time perception stack (DeepStream/TensorRT) for live driver coaching. In beta with a regional transit agency; demonstrated live at three industry conferences.

Several fragmented CV models became one multi-camera pipeline with sensor fusion (camera, IMU, GPS), producing distance, collision, lane, and driver-behavior signals on constrained hardware. Real-time perception latency dropped from 300ms to under 40ms p99. The scoring layer extends NHTSA and insurance-industry risk models with a predictive tier in beta and a Safe-RL (CMDP) policy layer in development.

SORT, implemented from the filter up. A seven-dimensional constant-velocity state over box center, area and aspect ratio. Association by IoU cost, resolved with Hungarian assignment. Lifecycle is probation, coasting, death. No filterpy, no torch.

Writing it by hand was about the covariance. A coasting track's inflated P is what widens the region it will accept a match from, and that stops being an abstraction the moment you have to pick the initial values yourself.

Scored against Ultralytics ByteTrack and BoT-SORT over the same 841-frame clip, same YOLO26s detections, same machine. 34.2 FPS end to end against ByteTrack's 33.5. There is no human ground truth for the clip, so ByteTrack stands in as pseudo-reference: three identity divergences across the full sequence. BoT-SORT surrenders about a third of its throughput to camera-motion compensation, which buys nothing on a static camera and everything on a dash cam.

One deliberate departure from the 2016 paper. Confirmation latches here. In the original, a confirmed track that misses a few frames must re-serve full probation before it is emitted again, so it goes silent precisely when trajectory continuity matters most.

Camera calibration done by hand in CARLA, where every parameter is analytically known and the exercise is verification rather than recovery. Intrinsics from field of view, principal point at image center, distortion identically zero because the RGB sensor is a clean pinhole. A checkerboard fit that returns non-trivial distortion coefficients is reporting a bug in the fit.

The frame convention is what bites. Unreal is body-centric, x forward and z up. OpenCV is image-centric, z down the optical axis and y pointing down. Skip the swap and nothing throws. Points project and land inside the frame. They are also wrong.

Two tests come free. Vertical disparity across a rectified pair has a known true value of zero, so any drift means a sign error in the baseline or an unremoved rotation in the extrinsics. Predicted disparity from fx·B/Z can be checked against ground truth, provided Z is read off the transformed camera-frame coordinate rather than Euclidean range. That substitution is the standard trap and it fails quietly.

Simulation also supplies MOT annotation at zero cost. Actor IDs with projected boxes are pixel-perfect labels, which is thousands of hours that nobody has to spend.

A standalone recognition device (Jetson Orin Nano) serving 210+ enrolled users at a live client site, integrated with the client's workforce-management platform and running as a monitored production service at 99.9% logged uptime. No identifying data is held on the edge device.

Multi-subject and real-time: 30ms median detection, up to 14 simultaneous faces. When layered temporal validation cannot confirm an identity, the system withholds the match. An 18-day production evaluation recorded zero false identifications, including under adversarial enrollment tests.

Architect and lead of a learned decision layer for intra-day transit dispatch — the flagship research program of a transit technology company and a ground-up replacement of the dispatch engine behind its primary platform. Reports directly to the CEO, with hiring authority for the research team. First components went live in September 2026.

The problem is not scheduling. A manifest is built the night before and begins decaying within the first hours of service — riders cancel, traffic moves, vehicles go down. The question a dispatcher faces is not what the optimal plan was; it is what to do now, given what actually happened, with no way to test a decision before committing to it. That is a different problem from the one an overnight optimizer solves, and it does not transfer.

What it is built from. The formulation is a semi-Markov decision process — decisions arrive when a projected vehicle breaches a lateness trigger, not on a fixed tick, so actions are variable-duration options and discounting runs over elapsed time rather than step count. A PUCT-guided tree search plays the remainder of the service day forward under a learned world model, with a policy prior replacing UCT's uninformed exploration term. What it returns is a distribution over candidate actions rather than a score: the visit share of the recommended action is exported as a first-class confidence figure, which is what makes graduated autonomy — shadow mode, then advisory, then bounded automation above a threshold — definable in advance rather than negotiated after deployment.

Distributions, not point forecasts. Two trips both predicted at twenty minutes are indistinguishable to a point forecaster even when one is routine and the other has historically run from twelve minutes to fifty. The correct action differs between them. Every stochastic quantity in the system — travel time, dwell, no-show, cancellation — is therefore a calibrated conditional distribution, trained on the pinball loss and gated on interval coverage rather than mean error, and sampled by inverse-quantile so each simulated future inherits that specific trip's own spread.

Bayesian state. Operational state is carried as spatial planes over a hex lattice, each holding a quantity in explicit units expressed as deviation from plan. At useful resolution most cells are thin, and a maximum-likelihood rate from three observations is noise presented as fact — so the planes carry conjugate posteriors (Gamma–Poisson for counts, Beta–Binomial for binary events) under hierarchical priors that shrink a cell toward its neighbourhood and the neighbourhood toward the region. A sparse cell sits near its prior instead of lurching between confident wrong answers, and the posterior variance survives all the way to the decision, where it is the signal that should make the search prefer reversible actions.

Spatial structure. A GNN-lite tier — one message-passing step over the cell lattice with a fixed aggregator and a boosted readout — captures local supply-and-demand pressure at no graph-training cost, and is the computational form of that hierarchical prior. A learned spatio-temporal graph network over the planes is designed but deliberately not built: published spatio-temporal models are benchmarked on months of dense sensor coverage, and the honest reading of the data on hand is that it does not yet support a learned field. The interim tier is stated as an interim.

Constraints, and the design claim. The constraint set is split three ways: walls, budgets, preferences. Budgets — overtime, contracted spend — are the constrained-MDP layer, where a Lagrange multiplier learned by dual ascent is the right tool. Regulatory limits are not put there. A constrained-MDP limit binds only in expectation, a multiplier is a price, and a large enough reward makes paying it rational; under function approximation the value network also generalises across the constraint boundary and assigns attractive value to regions the policy must never enter. So accessibility and securement requirements are enforced as a legality mask at node expansion instead — the infeasible action is absent from the tree, never expanded, never backed up, never contributing to a value estimate. Much of the safe-RL literature prices regulatory constraints. For automation touching accessible service, the difference between unlikely to and cannot is the whole point.

Live as of September 2026: the distributional world model, the Bayesian posterior layer, GNN-lite spatial features, and PUCT search over the semi-MDP with hard feasibility masking — deployed as a single CPU-only container at roughly 20 ms per decision, reproducing a reference service day record for record. The learned budget layer, joint multi-vehicle search, and the spatio-temporal field model are next.

The GPU cloud layer behind the edge systems: FastAPI/Docker services on AWS for embedding and VLM analysis, remote enrollment, and IoT event delivery into client dashboards. An LLM call-summarization pipeline processes 400+ calls per day for client performance analytics.

Quantized LLM deployment on the edge, and where the hardware says no. Phi-3 Mini on an Orin Nano, 8 GB: quantized to roughly 3 GB, context held at 4K. The model fits. The TensorRT-LLM engine build does not — weights, activations, KV cache and build workspace together ran past 20 GB, and swap is no help because the build wants it resident. llama.cpp was already working; TensorRT-LLM was the reach, chosen to put the language model on one runtime with the rest of the perception stack instead of a separate process. The envelope refused. Knowing exactly where it refused is worth more than the deployment would have been.

Stack: NVIDIA Jetson Orin Nano / NX, DeepStream (C/Python), TensorRT, ONNX, GStreamer, quantization (INT8/GGUF), llama.cpp, vLLM, FastAPI, AWS, Docker, Linux, systemd. Tracking and calibration work in NumPy/SciPy, OpenCV and CARLA.

Sole inventor of REVEAL (provisional patent, USPTO 2026), a conditioning framework for visual attention in reasoning vision-language models. The independent research program (biomechanics, corpus generation, training infrastructure) is described across /projects and /research, and is unaffiliated with his employer.

résumé (pdf) →

github.com/aiaxmaior

linkedin.com/in/arjun-joshi

Arjun.Joshi@Agni.works