Hello There!
I'm Div Garg
I'm a
Biological NN
Coder
Researcher
Scientist
Biological NN
Biological NN
Coder
Researcher
Scientist
Biological NN
Developing autonomous AI agents that learn, adapt, and make complex decisions in dynamic real-world environments, with a focus on reinforcement learning, scalable system design, and deploying intelligent models beyond the lab into practical applications.
About Me
Hi, I'm Div. I am a computer scientist and AI researcher. I am on a leave of absence from a CS PhD at Stanford University. I am building MultiOn - a Personal AI Agent startup with the mission of building general-purpose AI similar to JARVIS to do your digital tasks and bring it to everyday consumer use.
I am passionate about all things Deep Learning as well as its applications to Robotics and Vision. My recent focuses have been on building systems that are generally intelligent and able to reason, designing learning algorithms that can learn as efficiently as humans and making Reinforcement Learning work in the real world.
I created and taught the first course on Transformers at Stanford — CS 25: Transformers United — discussing the latest breakthroughs and broad implications of Transformers in AI. The class invites people at the forefront of Transformers research in various fields to spark cross-collaborative research, including eminent speakers like Prof. Geoffrey Hinton. Our lectures have received an overwhelming public reception with over 1M views on Youtube.
I have given invited talks at OpenAI, DeepMind and Apple on my research. I recently worked on a new RL algorithm — Extreme Q-Learning — that builds on a 2000 Nobel prize work in Economics to solve a previously assumed intractable problem and reaches state-of-the-art in Offline RL. My past work — IQ-Learn — introduces a novel imitation learning framework to have AI agents learn from observing videos of humans, creating agents that can play video games like Atari at human performance and won the #1 place in NeurIPS '21 Minecraft AI competition. The work received media coverage from Stanford HAI.
In my undergrad at Cornell, I created the first working camera-based self-driving car system and published 4 major conference papers over 2 years in 3D Vision. My work was widely covered in the media: Forbes, The Robot Report, Gizmodo.
I have been fortunate to be mentored by Ian Goodfellow (and was his first intern at Apple).
In a past life, I was a child prodigy in Physics and won medals in a few International Olympiads.
Hello!
Hello!
I’m working toward general-purpose AI systems that don’t just predict, but reason, adapt, and act autonomously to solve real-world problems at scale.
I’m working toward general-purpose AI systems that don’t just predict, but reason, adapt, and act autonomously to solve real-world problems at scale.
I’m working toward general-purpose AI systems that don’t just predict, but reason, adapt, and act autonomously to solve real-world problems at scale.
General-Purpose AI
Imitation Learning
Reinforcement Learning
Autonomous Systems
AI Agents
Decision Intelligence
General-Purpose AI
Imitation Learning
Reinforcement Learning
Autonomous Systems
AI Agents
Decision Intelligence
Work Experience
Work Experience
From Research Labs to Real-World Systems
From Research Labs to Real-World Systems
March '20 - Sept '20
Apple Special Projects Group (SPG)
Research Intern
Worked as a research intern in Apple’s Special Projects Group under the direct supervision of Ian Goodfellow. Conducted research in reinforcement learning, inverse reinforcement learning, and generative modeling, contributing to advanced machine learning systems within Apple’s AI research efforts.
March '20 - Sept '20
Apple Special Projects Group (SPG)
Research Intern
Worked as a research intern in Apple’s Special Projects Group under the direct supervision of Ian Goodfellow. Conducted research in reinforcement learning, inverse reinforcement learning, and generative modeling, contributing to advanced machine learning systems within Apple’s AI research efforts.
March '20 - Sept '20
Apple Special Projects Group (SPG)
Research Intern
Worked as a research intern in Apple’s Special Projects Group under the direct supervision of Ian Goodfellow. Conducted research in reinforcement learning, inverse reinforcement learning, and generative modeling, contributing to advanced machine learning systems within Apple’s AI research efforts.
Summer 2019
Google AI
Machine Learning Intern
Interned at Google AI on the Machine Perception team in Mountain View. Designed machine learning models to address real-time computer vision challenges for augmented reality (AR) devices.
Summer 2019
Google AI
Machine Learning Intern
Interned at Google AI on the Machine Perception team in Mountain View. Designed machine learning models to address real-time computer vision challenges for augmented reality (AR) devices.
Summer 2019
Google AI
Machine Learning Intern
Interned at Google AI on the Machine Perception team in Mountain View. Designed machine learning models to address real-time computer vision challenges for augmented reality (AR) devices.
Aug '18 – May '20
CV Research
Conducted computer vision research focused on camera-only depth estimation and 3D object detection. Developed a state-of-the-art model for stereo-only 3D object detection, advancing perception capabilities in vision systems.
Aug '18 – May '20
CV Research
Conducted computer vision research focused on camera-only depth estimation and 3D object detection. Developed a state-of-the-art model for stereo-only 3D object detection, advancing perception capabilities in vision systems.
Aug '18 – May '20
CV Research
Conducted computer vision research focused on camera-only depth estimation and 3D object detection. Developed a state-of-the-art model for stereo-only 3D object detection, advancing perception capabilities in vision systems.
Summer 2018
Uber ATG (Advanced Technologies Group)
Perception Intern — Self-Driving Cars (Pittsburgh)
Interned at Uber’s self-driving car division in Pittsburgh. Worked on the Perception team to improve the autonomous vehicle’s 3D object detection system.
Summer 2018
Uber ATG (Advanced Technologies Group)
Perception Intern — Self-Driving Cars (Pittsburgh)
Interned at Uber’s self-driving car division in Pittsburgh. Worked on the Perception team to improve the autonomous vehicle’s 3D object detection system.
Summer 2018
Uber ATG (Advanced Technologies Group)
Perception Intern — Self-Driving Cars (Pittsburgh)
Interned at Uber’s self-driving car division in Pittsburgh. Worked on the Perception team to improve the autonomous vehicle’s 3D object detection system.
Sept '17 – Dec '18
Cornell Mars Rover
Autonomous Systems Engineer
Worked on a Cornell project team to build a rover for the University Rover Challenge in Utah. Developed autonomous systems using computer vision to enhance object detection and improve the rover’s navigation capabilities.
Sept '17 – Dec '18
Cornell Mars Rover
Autonomous Systems Engineer
Worked on a Cornell project team to build a rover for the University Rover Challenge in Utah. Developed autonomous systems using computer vision to enhance object detection and improve the rover’s navigation capabilities.
Sept '17 – Dec '18
Cornell Mars Rover
Autonomous Systems Engineer
Worked on a Cornell project team to build a rover for the University Rover Challenge in Utah. Developed autonomous systems using computer vision to enhance object detection and improve the rover’s navigation capabilities.
Aug '17 – Dec '17
COMAKE
Machine Learning Engineer
Worked as an ML Engineer to develop a smart file browser leveraging file analysis and context-based workflow management. Designed and implemented a recommendation system to suggest related files and improve user workflow efficiency.
Aug '17 – Dec '17
COMAKE
Machine Learning Engineer
Worked as an ML Engineer to develop a smart file browser leveraging file analysis and context-based workflow management. Designed and implemented a recommendation system to suggest related files and improve user workflow efficiency.
Aug '17 – Dec '17
COMAKE
Machine Learning Engineer
Worked as an ML Engineer to develop a smart file browser leveraging file analysis and context-based workflow management. Designed and implemented a recommendation system to suggest related files and improve user workflow efficiency.
Publications
Advancing Reinforcement Learning & AI Agents
Extreme Q-Learning: MaxEnt RL without Entropy
ICLR 2023 (Oral)
Divyansh Garg*, Joey Hejna*, Matthieu Geist, Stefano Ermon
tl;dr Introduce Gumbel Regression to learn maximal values in RL without needing to sample from a policy and reaches SOTA performance on offline RL.
Extreme Q-Learning: MaxEnt RL without Entropy
ICLR 2023 (Oral)
Divyansh Garg*, Joey Hejna*, Matthieu Geist, Stefano Ermon
tl;dr Introduce Gumbel Regression to learn maximal values in RL without needing to sample from a policy and reaches SOTA performance on offline RL.
Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Driving
CVPR 2019
Yan Wang, Wei-Lun Chao, Divyansh Garg, Bharath Hariharan, Mark Campbell, Kilian Q. Weinberger
tl;dr The First sytem for camera-only autonomous driving
Media Coverage: Cornell Chronicle, Forbes, NSF, Teslarati, Gizmodo
Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Driving
CVPR 2019
Yan Wang, Wei-Lun Chao, Divyansh Garg, Bharath Hariharan, Mark Campbell, Kilian Q. Weinberger
tl;dr The First sytem for camera-only autonomous driving
Media Coverage: Cornell Chronicle, Forbes, NSF, Teslarati, Gizmodo
*denotes equal contribution
Projects
Building Intelligent Systems in Practice
FS-CIS Net
Sept '19 - Dec '19
Designed a novel network architecture - Few-shot Clustering Instance Segmentation Network (FS-CIS Net) - to tackle the problem of proposal-free few-shot instance segmentation. Approach validated on the PASCAL-5i dataset and performs comparably to MaskRCNN inspired methods with significant speedups. Showcased in CS 6670 course.
FS-CIS Net
Sept '19 - Dec '19
Designed a novel network architecture - Few-shot Clustering Instance Segmentation Network (FS-CIS Net) - to tackle the problem of proposal-free few-shot instance segmentation. Approach validated on the PASCAL-5i dataset and performs comparably to MaskRCNN inspired methods with significant speedups. Showcased in CS 6670 course.
Traffic Accident Detection System
Feb '19 - May '19
Created an automated accident detection system utilizing real-time traffic cam feeds to provide instantaneous response to accidents. Designed ML classifier based on CNN+RNN architectures to predict vehicle collisions upto 3 secs in advance. Prototype tested on public New York CCTV feeds and deployed for real-time using Azure Cloud to achieve massive scaling.
Traffic Accident Detection System
Feb '19 - May '19
Created an automated accident detection system utilizing real-time traffic cam feeds to provide instantaneous response to accidents. Designed ML classifier based on CNN+RNN architectures to predict vehicle collisions upto 3 secs in advance. Prototype tested on public New York CCTV feeds and deployed for real-time using Azure Cloud to achieve massive scaling.
Image Captioning System
Feb '18 - Apr '18
Trained a LSTM based neural network using Visual Attention mechanism to generate image captions. Achieved near top level of performance on Flickr Dataset. Showcased in CS 6700 course.
Image Captioning System
Feb '18 - Apr '18
Trained a LSTM based neural network using Visual Attention mechanism to generate image captions. Achieved near top level of performance on Flickr Dataset. Showcased in CS 6700 course.
Automated Keyboard Typing
Feb '18 - May '18
Built an automated keyboard typing system, that uses a single camera to detect a keyboard, recognize keys and calculate 3D coordinates using SfM. Implemented on a rover that can move its arm to the calculated key positions and type autonomously.
Automated Keyboard Typing
Feb '18 - May '18
Built an automated keyboard typing system, that uses a single camera to detect a keyboard, recognize keys and calculate 3D coordinates using SfM. Implemented on a rover that can move its arm to the calculated key positions and type autonomously.
OcamTeX
Sept '17 - Dec '17
Created a subset language of LaTeX geared towards simplicity to make LaTeX typesetting more inituitive and adapted for faster editing, written in OCaml. Included features like automatic Math mode, an elegant indentation-based syntax, and a SublimeText plugin.
OcamTeX
Sept '17 - Dec '17
Created a subset language of LaTeX geared towards simplicity to make LaTeX typesetting more inituitive and adapted for faster editing, written in OCaml. Included features like automatic Math mode, an elegant indentation-based syntax, and a SublimeText plugin.
Face Recognition
May '17 - July '17
Built a face recognition system using a 16 layered ConvNet trained on a 200K image dataset to learn face embeddings and recognize faces on any custom data. Acheived accuracy of 97% on LFW dataset.
Face Recognition
May '17 - July '17
Built a face recognition system using a 16 layered ConvNet trained on a 200K image dataset to learn face embeddings and recognize faces on any custom data. Acheived accuracy of 97% on LFW dataset.
Critter World Project
Aug '16 - Dec '16
Created distributed and concurrent simulation of world containing creatures (critters) able to move, reproduce and evolve. Used Abstract Syntax Trees as genome for critters, and added fault injections for genome mutations. Finished with a nice GUI front-end written in JavaFX.
Critter World Project
Aug '16 - Dec '16
Created distributed and concurrent simulation of world containing creatures (critters) able to move, reproduce and evolve. Used Abstract Syntax Trees as genome for critters, and added fault injections for genome mutations. Finished with a nice GUI front-end written in JavaFX.
EMS Routing
Oct '18
Found optimal routing and placements of ambulances in Ithaca for fast response to emergencies with minimum active vehicles. Submitted in Cornell Mathematical Contest in Modeling (CMCM). Wrote code for experiments and created simulations.
EMS Routing
Oct '18
Found optimal routing and placements of ambulances in Ithaca for fast response to emergencies with minimum active vehicles. Submitted in Cornell Mathematical Contest in Modeling (CMCM). Wrote code for experiments and created simulations.
Service & Awards
Research community involvement and distinguished achievements
Research community involvement and distinguished achievements
Academic Service
Reviewer
ICML
2022
Reviewer
ICML
2022
Reviewer
NeurIPS
2020, 2021
Reviewer
NeurIPS
2020, 2021
Reviewer
ICLR
2020, 2021
Reviewer
ICLR
2020, 2021
Reviewer
CVPR
2020
Reviewer
CVPR
2020
Awards
1st Place (Human Videos Track), 2nd Overall
NeurIPS MineRL BASALT Competition
2021
Using only human demonstration videos
1st Place (Human Videos Track), 2nd Overall
NeurIPS MineRL BASALT Competition
2021
Using only human demonstration videos
Summa Cum Laude Honors
Cornell University
2020
-
Summa Cum Laude Honors
Cornell University
2020
Dean’s List (All Semesters)
Cornell University
-
Academic distinction across all semesters
Dean’s List (All Semesters)
Cornell University
Academic distinction across all semesters
Tata Scholarship
Cornell University
2016
Awarded to 4 students per year
Tata Scholarship
Cornell University
2016
Awarded to 4 students per year
Silver Medal
International Physics Olympiad (IPhO)
2016
-
Silver Medal
International Physics Olympiad (IPhO)
2016
Best Solution Award
National Physics Olympiad
2016
-
Best Solution Award
National Physics Olympiad
2016
Best Science Student Award
-
2016
-
Best Science Student Award
2016
National KVPY Fellowship
Government of India
2015
National research fellowship
National KVPY Fellowship
Government of India
2015
National research fellowship
Gold Medal
International Junior Science Olympiad (IJSO)
2013
-
Gold Medal
International Junior Science Olympiad (IJSO)
2013
Fun Facts
My name is made of two Hindi words: divya + ansh. And has the literal English translation - "divine fragment".
My name is made of two Hindi words: divya + ansh. And has the literal English translation - "divine fragment".
At Cornell, I was famous for solving an exam designed to fail everyone (advanced standing exam for Physics 2214), causing the department to abolish it.
At Cornell, I was famous for solving an exam designed to fail everyone (advanced standing exam for Physics 2214), causing the department to abolish it.
I received an (unofficial) Physics Degree from Cornell. I completed all requirements, but was in the wrong college. I was allowed the title by the Dean.
I received an (unofficial) Physics Degree from Cornell. I completed all requirements, but was in the wrong college. I was allowed the title by the Dean.
I took up adventure sports over the pandemic. Over a year, I learnt scuba diving, skiing and surfing. I also sky dived and flew a plane!
I took up adventure sports over the pandemic. Over a year, I learnt scuba diving, skiing and surfing. I also sky dived and flew a plane!
I stayed a night on a boat in the middle of Atlantic Ocean for an Airbnb.
I stayed a night on a boat in the middle of Atlantic Ocean for an Airbnb.
News
Jan 2024: Announcing MultiOn: Building a Brighter Future for Humanity with AI Agents
Jan 2024: Announcing MultiOn: Building a Brighter Future for Humanity with AI Agents
April 2023: I gave research talks on AI Autonomous Agents at the Stanford NLP group & Langchain Agents Webinar on Youtube.
April 2023: I gave research talks on AI Autonomous Agents at the Stanford NLP group & Langchain Agents Webinar on Youtube.
March 2023: Released a blog post on the future of software: Software 3.0
March 2023: Released a blog post on the future of software: Software 3.0
Feb 2023: Extreme Q-learning was accepted in ICLR 2023 as Oral!
Feb 2023: Extreme Q-learning was accepted in ICLR 2023 as Oral!
Nov 2022: Released my very first blog post on Learning to Imitate to improve current AI systems on the Stanford AI blog.
Nov 2022: Released my very first blog post on Learning to Imitate to improve current AI systems on the Stanford AI blog.
Sept 2022: Got Oral in European Workshop for Reinforcement Learning (EWRL) for IQ-Learn
Sept 2022: Got Oral in European Workshop for Reinforcement Learning (EWRL) for IQ-Learn
Aug 2022: Publicly released our CS 25: Transformers United lectures on Youtube with great reception!
Aug 2022: Publicly released our CS 25: Transformers United lectures on Youtube with great reception!
May 2022: Stanford HAI featured a story on IQ-Learn: Training Smarter Bots for the Real World!
May 2022: Stanford HAI featured a story on IQ-Learn: Training Smarter Bots for the Real World!
March 2022: Released LISA: a heirarchical framework to make robots better understand natural language for solving long-range tasks on Arxiv
March 2022: Released LISA: a heirarchical framework to make robots better understand natural language for solving long-range tasks on Arxiv
Dec 2021: Won #1 place in creating an AI bot to play Minecraft in NeurIPS '21 MineRL Basalt challenge using only recorded videos of human players
Dec 2021: Won #1 place in creating an AI bot to play Minecraft in NeurIPS '21 MineRL Basalt challenge using only recorded videos of human players
Oct 2021: Gave invited research talk at Apple on IQ-Learn
Oct 2021: Gave invited research talk at Apple on IQ-Learn
Sept 2021: IQ-Learn was accepted in NeurIPS 2021 with Spotlight!
Sept 2021: IQ-Learn was accepted in NeurIPS 2021 with Spotlight!
Aug 2021: Teaching Stanford's first class on Transformers: CS 25
Aug 2021: Teaching Stanford's first class on Transformers: CS 25
July 2021: I gave a invited research talk at OpenAI on improving imitation by more than 5x with my recent work: IQ-Learn.
July 2021: I gave a invited research talk at OpenAI on improving imitation by more than 5x with my recent work: IQ-Learn.
June 2021: Preprint on our state-of-art Imitation Learning method IQ-Learn available on Arxiv
June 2021: Preprint on our state-of-art Imitation Learning method IQ-Learn available on Arxiv
Nov 2020: I gave a talk on a youtube channel: Computer Vision Talks
Nov 2020: I gave a talk on a youtube channel: Computer Vision Talks
Sept 2020: My paper for statistical learning of stereo depth was accepted in NeurIPS 2020 with Spotlight!
Sept 2020: My paper for statistical learning of stereo depth was accepted in NeurIPS 2020 with Spotlight!
March 2020: Started Apple internship under Ian Goodfellow.
March 2020: Started Apple internship under Ian Goodfellow.
Feb 2020: New paper on 3D Object Detection accepted in CVPR 2020.
Feb 2020: New paper on 3D Object Detection accepted in CVPR 2020.