Robert Lo

Robert Lo

ML Infra @ Tesla Autopilot

About Me

👨‍💻 Hi! I’m Robert Lo, currently a Machine Learning Engineer on the Tesla Autopilot Vision Team, working on Full Self-Driving (FSD). I earned my M.S. in Computer Science from Carnegie Mellon University (CMU) and my B.S. in Computer Science and B.A. in Economics from National Taiwan University (NTU). At Tesla Autopilot, I have experience on both the scientific and systems sides of scaling up training jobs. I’ve also developed novel driving tokenizers to enhance the safety of FSD. Before joining the Autopilot team, I conducted research on computer-using agents at CMU with Prof. Shuyan Zhou, advised by Prof. Graham Neubig, contributing to projects such as WebArena, VisualWebArena and ASH Prompting. Google Scholar profile.

📋 Previously I’ve conducted SWE internships at quantitative trading startup Kronos, Microsoft, NVIDIA and a Machine Learning Research Engineer Intern at Scale AI.

✏️ I write tutorials about advanced data structure and algorithms here on a random basis. Most topics I wrote are competitive programming stuff. I enjoy problem-solving and won second/fourth place at ICPC 2020/2019 Asia Taipei-Hsinchu Regional as team CRyptoGRapheR.

📚 In 2021, I coordinated a training camp, IOICamp, which helps students prepare for top-tier informatics competitions like IOI and ICPC. I also founded another entry-level coding camp in 2021 summer, APCSCamp, which teaches C++ and basic algorithm for all students from elementry school to college.

🔑 PGP Public Key

Currently I’m…
  • Machine Learning Engineer @ Tesla Autopilot Vision Team, Palo Alto
Education

  • Carnegie Mellon University, 2022 - 2023

    M.S. Computer Science

    • Courses: Distributed Systems, Parallel Computer Architecture, Deep Reinforcement Learning, Advanced Natural Language Processing (TA), Visual Learning and Recognition, Probablistic Graphical Models

  • National Taiwan Unversity, 2018 - 2022, graduate rank 3/140 (2%)

    B.S. Computer Science / Economics

    • Courses: Advanced Statistical Inference, Calculus, Linear Algebra, Machine Learning Theory and Foundation, Operating System, System Programming, Compiler, Micro/Macroeconomics, Financial Engineering

Work Experience

 
 
 
 
 
March 2024 - ??
Palo Alto, CA, USA
Autopilot Engineer
Tesla Autopilot
  • make model train faster with a lot of gpus
  • working on v13 FSD
 
 
 
 
 
June 2023 - Aug. 2023
San Francisco, CA, USA
Machine Learning Research Engineer Intern
Scale AI
  • Desinged and implemented a internal declaritive & modularized machine learning training framework with Python, supporting both supervised and semi-supervised training paradigm.
  • Technology: PyTorch, Jupyter Notebook, Wandb, AWS EC2, visualization, Huggingface
 
 
 
 
 
Jan. 2023 – July 2023
Pittsburgh, PA, USA
Graduate Research Assistant, Advisor: Prof. Graham Neubig
Independent Study, Carnegie Mellon University
  • “WebArena: A Realistic Web Environment for Building Autonomous Agents” (ICLR 2024)
  • “Hierarchical Prompting Assists Large Language Model on Web Navigation” (EMNLP 2023)
 
 
 
 
 
July 2021 - Aug. 2021
Taipei, Taiwan
System Software Engineer Intern
NVIDIA SW RM Team
  • Designed and built GC6 Latency Perf Test, a test that measures sleep-wake latency of engines in the GPU Resource Manager(RM). It will compare the result with historical data and raise error when regression i.e. a higher sleep-wake latency is detected, using C and Python scripts for data analysis.
  • Implemented a website to visualizes the GC6 Latency Perf Test. Designed a RESTful website with NodeJS+Express as backend and ReactJS as frontend to visualize historical sleep-wake latency data.
 
 
 
 
 
Sep. 2020 - June 2021
Taipei, Taiwan
Software Engineer Intern
Microsoft, Bing Maps & Geocoding Team
  • Speed up the Bing Maps Entity Ranking / Confidence Model. Reduced the latency of the (gradient boost decision tree) model by 300+% with only 0.5% accuracy drop via feature selection and model compression techniques.
  • Deployed the BingGC Debugger, a debugger that visualize the entity ranking model. Configured it via IIS(Internet Information Service) manager.
 
 
 
 
 
July 2020 – Sep. 2020
Taipei, Taiwan
Summer Analyst
Kronos Research
  • Designed and built a interface for backtesting system. Designed, documented, and implemented the interface with Python, MySQL and CrobJobs which let traders to launch/compare/analyze multiple tests at one click. Reduced number of manual configuration and increased the productivity of traders.
  • Improved the accuracy of backtesting. Created a linear regression model that increased the accuracy of backtesting. Discovered and fixed lots of issues/bugs in the system while studying the data.
 
 
 
 
 
May 2021 - Aug. 2021
Taipei, Taiwan
Organizer
APCSCamp
  • Organized a 10-day intro programming summer camp. Led a team of 50+ people in organizing a fully online Intro-to-C++/Algorithm Design programming summer camp for 200+ high school/college students from scratch.
  • Raised US$8000 from multiple firms. Secured sponsorships from LINE, Shopee, Conflux.
 
 
 
 
 
July 2020 - Jan. 2021
Taipei, Taiwan
Organizer
IOICamp
  • Organized a 5-day competitive programming winter camp. Led a team of 40+ people, organizing a winter camp that trained 100+ high school/university students for top-tier programming contests like the International Olympiad of Informatics (IOI) and International Collegiate Programming Contest (ICPC).
  • Raised US$10000 from multiple firms. Secured sponsorships from MixerBox, Kronos Research, SYSTEX.

Projects

Exploring the Role of the Bottleneck in Slot-Based Models Through Covariance Regularization
May 2023
  • Improved the performance of slot attention model with self-supervised loss (VICReg).
  • Arxiv Preprint
LIC-GAN Language Information Conditioned Graph Generative GAN Model
May 2023
  • Desinged a new architecture that generate synthetic graph from text description.
  • Arxiv Preprint
Hierarchical Prompting Assists Large Language Model on Web Navigation
Dec 2022
  • Proposed a new way for large language model prompting on sequential decision task. Achieved new SOTA on a simulated web shopping environment WebShop, where the task is to find a product that matches the given natural language instruction.
  • Accepted to Natural Language Reasoning and Structured Explanations (NLRSE) Workshop at ACL’23
  • Arxiv Preprint
Neural Derivative Calculator with Transformer
Aug 2022
  • Desinged a neural derivative calculator with transformer, and achieve 99% validation accuracy. It support basic arithmetic operations, parenthesis and sin, cos, exp.
  • GitHub Repo
Does ”Per Bag Trash Collection Fee” Policy Reduce The Amount of Trash?
June 2022
  • Confirmed the policy effect on trash reduction with causal inference method: We analyzed the effect of Taipei government’s “Per Bag Trash Collection Fee” policy on trash amount of Taipei. We utilized Regression discontinuity design in Time (RDiT) to prove the causal relationship between the policy and the reduction of trash.
  • paper link
Covid19 footprint tracker
June 2021
  • Designed a system for covid19 footprint tracking. Lead a six people team that designed and implemented a cloud-based covid19 footprint tracker. It has a simple APP/Web frontend that enable people to check-in in less then a minute. The backend is deployed on AWS EKS as a serverless service, managed with Knative. Our backend resources can scale up/down(even to 0) automatically in line with the current amount of traffic.
Document Information Extraction with BERT
June 2020
  • Designed a NER model for Japanese bidding documents. Lead a three people team that designed and built a Named Entity Recognition(NER) model with BERT, which extracts a predefined set of tags from Japanese bidding documents. Applied the Question Answering framework and overcomed the scarcity of data by transfer learning and data augmentation. Further improved the model by tree-based pre-processing, prediction post-processing (sanity check), class-balanced loss and ensemble technique.
  • Placed First out of 17 teams with f1-score 97.904% (Team 18: Kaggle link).
  • Tools & technologies: Python, Pytorch, HuggingFace, BERT, NumPy, Pandas, Latex, Matplotlib
Atari Game Bot
May 2020
  • Built an Atari game bot with reinforcement learning with policy gradient and deep-Q learning. The resulting agents outperforms average human performance.
  • Tools & technologies: Python, Pytorch, OpenAI Gym, NumPy
ADA Judge
Sep. 2020 – Present
  • Improved an online judge system for course Algorithm Design and Analysis(ADA). Added new feature and implemented automatic remote backup on modern cloud service e.g. Google drive.
  • Tools & technologies: Python, Javascript, HTML/CSS, Semantic UI, NodeJS, Jade(Node Template Engine), MongoDB
C−− Compiler
Sep. 2020 – Jan. 2021
  • Designed and implemented a RISC-V compiler for C−−, which is a simplified C that supporting basic arithmetic operations, loops, functions and arrays.
  • Tools & technologies: RISC-V, Compiler, flex(lexer), bison(parser), C

Skills

Fast learner, able to pick up new stuffs fast

  • Programming: Python (NumPy, Pandas, Matplotlib, Pytorch) • C/C++ • JavaScript (NodeJS) • HTML/CSS • SQL • MongoDB
  • Technologies & Tools: Git • Latex • Tmux • Vim • Linux • Github Action • Github Pages
  • Languages: English (Fluent) • Chinese (Native)
  • Others: Algorithm Design & Analysis • Teaching • typeracer.com

Awards & Certificates

Awards
  • Placed Second out of 101 teams in 2020 ICPC Asia Taipei Regional Contest (Team CRyptoGRapheR) (scoreboard)
  • Placed Fourth out of 108 teams in 2019 ICPC Asia Taipei Regional Contest (Team CRyptoGRapheR)
  • Placed Fourth out of 25 teams in 2021 AIS3 EOF CTF (Team CRyptoGRapheR) (scoreboard)
  • Placed Third out of 18 teams in 2021 TTCPC (Team CRyptoGRapheR) (scoreboard)
Certificates