The Art of Translation
What if your model could just look where it needed to? Exploring attention mechanisms in sequence-to-sequence learning and how they revolutionize machine translation.
About
Hello! I'm a Master's student at Georgia Tech advised by Prof. Bo Dai. My research currently lies in Long Context Modeling and Reinforcement Learning. I hope to spend my time in RL post-training and/or AI for Science for the foreseeable future.
Education
Computer Science and Mathematics at Georgia Tech. Focused on Machine Learning and Mathematical Modeling.
Recent Past
Most recently I led the software stack at Swish Robotics. Prior to that I worked on Fraud Detection Models and Infrastructure at Credit Karma. Indian National Math Olympiad finalist and a Top 300 rank on the Putnam Math Contest.
Blog
Flash and Furious
Benchmarking Flash Attention v1 and v2 in Triton against a naive PyTorch implementation of Scaled Dot Product Attention and Multi Headed Attention.
The Last Pull
The Exploration vs. Exploitation Dilemma and a brief introduction to Reinforcement Learning
Muon-Improved Chemistry Models
Optimizer Routing can benefit Scientific Discovery
Research
Research Assistant advised by Prof. Bo Dai
Building RL training infrastructure and environments for Machine Learning Engineering tasks on the back of MLE-Dojo. Conducting evals to test whether benchmark failures are actually long-context–bottlenecked.
Undergraduate Research at the Friendly Cities Lab
Worked with Prof. Clio Andris on geographic visualizations and spatial information theory.
The Senator's Problem
A Design Space of Node Placement Methods for Geospatial Network Visualizations.
Projects
Optimizer Routing in Chemistry Models with Muon
This is a small study of different ablations of the Muon optimizer on FAIR Chem's GemNet-OC Architecture on the OMAat24 dataset. The goal was to see whether routing Muon to specific blocks helps this architecture.
LoRA x Distillation
Implemented LoRA adapters on a 7B Mistral Model and distilled into a smaller student.
Flash Attention
Benchmarking Flash Attention v1 and v2 in Triton against a naive PyTorch implementation of Scaled Dot Product Attention and Multi Headed Attention.
Sequence-to-Sequence Learning with Bahdanau Attention
Trained a sequence-to-sequence model with and without the attention mechanism to translate natural language to Python snippets.