Embedded Files
My most recent internship was at Tesla Optimus in Palo Alto, California, where I joined the Navigation team to help advance mapping and localization for humanoid robots. At Tesla, I had the opportunity to contribute to the SLAM and VIO pipelines that form the foundation of Optimusβ perception and navigation stack. My role centered on building more accurate and reliable real-time pose estimation, while also expanding the quality of feature maps that the robot could use for relocalization in complex environments.
To achieve this, I worked extensively with modern computer vision and optimization techniques. I integrated advanced feature detection and matching methods such as SuperPoint, ALIKED, and LightGlue, and improved the global bundle adjustment process with Ceres to align point clouds and export sparse feature maps and occupancy grids. I also developed pipelines to project 2D image features into 3D space using LiDAR depth unprojection, applying filtering methods to remove noise and increase the robustness of landmark reconstruction.
Another significant part of my work involved combining vision with inertial sensing to strengthen the systemβs overall reliability. I transitioned the pipeline from pure visual odometry to full visual-inertial odometry by implementing IMU preintegration, online accelerometer and gyroscope bias estimation, and extrinsic calibration between the IMU and camera. This work gave me the chance to dive deep into multi-sensor fusion and optimization, and to contribute to Teslaβs mission of building scalable embodied intelligence with the Optimus humanoid robot.
During my fifth internship, I worked at the Human Interaction Lab at Reazon in Tokyo, where I focused on developing perception and state estimation systems for humanoid robots. My role centered on building end-to-end pipelines for accurate 6-DoF pose tracking of the robotβs limbs, combining inertial sensing, vision, and kinematic information into a unified framework. This experience allowed me to contribute to both the low-level accuracy of the robotβs motion and the high-level reliability of its interactions.
A major part of my work involved developing algorithms for inertial sensing. I engineered adaptive Madgwick filters to fuse accelerometer, gyroscope, and magnetometer data from multiple 9-axis IMUs mounted across the robotβs joints. I implemented hard- and soft-iron calibration procedures and adaptive drift compensation techniques to ensure stable orientation tracking over time. This gave the robot the ability to maintain precise joint angle estimates even under long operating periods, which was critical for reliable humanoid manipulation.
Beyond inertial sensing, I also worked extensively with vision-based calibration and optimization. I performed fisheye camera intrinsic calibration and used Apriltag detections to solve the Perspective-n-Point problem, aligning camera frames to the robotβs kinematic model. To refine accuracy, I implemented a joint optimization procedure using LevenbergβMarquardt over both camera extrinsics and encoder offsets, ensuring that the robotβs perception and kinematics aligned seamlessly.
Finally, I fused all available signalsβIMU data, Apriltag-based camera poses, and joint encoder readingsβwithin an Extended Kalman Filter framework. This fusion pipeline produced robust, low-latency 6-DoF estimates for the robotβs hands and fingers, enabling stable manipulation and interaction tasks. Overall, my time at Reazon gave me the opportunity to explore sensor fusion and calibration in a humanoid robotics context, and to contribute to research that pushes the boundaries of how robots perceive and interact with the physical world.
My fourth internship took place at Noah Medical, a company specializing in innovative surgical robotics. As part of the Imaging & Navigation team, I was immersed in projects focused on enhancing the surgical precision of bronchoscopy procedures. My role centered on developing algorithms for localization, state estimation, and navigation to support the GALAXY surgical robot, a state-of-the-art platform designed to perform minimally invasive procedures.
One of my key contributions was engineering a robust IMU sensor fusion system to improve the accuracy of the robotβs 3D pose estimation. I designed and implemented advanced filtering pipelines, applying techniques such as complementary and Kalman filtering to fuse accelerometer, gyroscope, and magnetometer data in real time. In addition, I explored alternative approaches to sensor fusion, building Extended and Unscented Kalman Filters, as well as particle filters, to achieve precise localization even in challenging operating conditions. These efforts gave me the opportunity to deep dive into state estimation theory while directly impacting the reliability of a surgical robot.
Beyond inertial sensing, I also worked on perception and mapping tasks. I trained and applied deep learning models such as PackNet-SfM for monocular depth estimation, converting bronchoscopy video streams into 3D depth images. These depth maps were then transformed into point clouds, where I implemented custom voxel downsampling and bilateral filtering algorithms to reduce noise while preserving important details. I further developed vision-based SLAM pipelines using Iterative Closest Point (ICP) and Coherent Point Drift (CPD) registration methods to align point clouds and enable real-time mapping. This combination of deep learning, geometry, and filtering allowed me to contribute to navigation algorithms that help surgeons visualize and localize within the lung.
Overall, my time at Noah Medical was incredibly rewarding. Working at the intersection of robotics, perception, and medical technology taught me how advanced algorithms can have real impact in high-stakes environments. I am grateful for the chance to work alongside brilliant engineers on transformative technologies that are redefining what is possible in surgical robotics.
Noah Medical.mp4Robotics Software Engineer Intern
Robotics Software Engineer Intern
September 2023 - December 2023
During my third internship as a Robotics Software Engineer at Impossible Metals on the Underwater Robotics team, I had the incredible opportunity to work on groundbreaking projects in the realm of autonomous underwater vehicles (AUVs). At Impossible Metals, I played a pivotal role in advancing our Eureka II robot with a focus on precision control, stability improvement, motion planning, writing driver code, and general system software.
During my time here, I worked on developing a Model Predictive Control (MPC) system for the AUV, leveraging a linearized 6-DOF dynamic system model and a quadratic cost function for precise linear and angular position and velocity control. In parallel, I delved into the realm of PID control, fine-tuning a robust system that integrated 12 PID controllers for linear and angular position and velocity control. Utilizing advanced tuning methodologies, including Cohen-Coon and Ziegler-Nichols I was able to significantly improve the robot's control system. These enhancements not only optimized the AUV's performance but also showcased our dedication to achieving excellence in control system design.
Working alongside a team of dedicated engineers at Impossible Metals, I found immense satisfaction in contributing to projects that push the boundaries of robotics innovation. This experience has not only expanded my technical skill set but also deepened my appreciation for the impactful intersection of robotics and real-world challenges in underwater nodule collection. I am truly grateful for the opportunity to be part of such a visionary team, driving advancements in autonomous underwater vehicles.
My second internship was at an extraordinary Aerospace company called Canadensys. At Canadensys, various high-level innovative projects ranging from spaceflight cameras to lunar rovers are being developed. On the Lunar Rover Team, I developed an obstacle detection/avoidance algorithm for Canada's first ever lunar rover!
To design and program an obstacle avoidance algorithm for this rover, I used C++ and various technologies such as the Sentis-ToF-M100 3D LiDAR, OpenCV, and the Eigen C++ library. I developed median filtering, Gaussian filtering, and bilateral filtering algorithms to filter out random noise in the LiDAR data. Using an algorithm called Least Squares Plane, I performed linear regression on XYZ LiDAR data read in from CSV files to estimate the ground plane. I then used rotation matrices and translations to transform points in the ground's reference frame to the LiDAR's reference frame. Based on their heights and the slopes of their surfaces, I detected obstacles and performed semantic segmentation on the LiDAR images to classify terrain such as boulders and ditches. I also established a TCP server-client system for image transfer between NISA spaceflight cameras, and researched various path planning algorithms for obstacle avoidance such as Artificial Potential Field, A* search, Bug 1 & 2, and RRT.
Overall, I really enjoyed my experience at Canadensys. I'm really thankful and fortunate I had the opportunity to work alongside brilliant engineers on historical projects like developing obstacle avoidance for Canada's first ever lunar rover!
My first ever internship was at an incredible Space robotics company called Mission Control and what an experience it was! At Mission Control, I was fortunate to work alongside some of the most brilliant and passionate engineers I've ever met. Our work has been launched to the moon and in Spring 2023, Mission Control will become the first company in the world to demonstrate artificial intelligence on the moon!
During my time at Mission Control, I was lucky to have the opportunity to work on absolutely unreal tech! participated in a Lunar Mapping Mission where I used a LiDAR sensor and ZED 2 camera's visual odometry to perform SLAM, creating 2D and 3D maps of a 4000ft2 moon-yard with RTAB-Map. I also used OpenCV, C++, and a hazard detection algorithm to identify hazards in the rover's view and autonomously stopped the rover within 1m of several hazards with 89% accuracy. Additionally, I used C++ and Eigen to compute the axis of rotation for a PTZ camera and implemented UR16e robot arm depth perception and spatial perception by integrating ZED Mini and AXIS stereo cameras into the robot arm with C++.
I genuinely believe that working at Mission Control was the best first work term I could have asked for. I was able to not only discover my love for robotics and autonomous tech and learn various technical skills, but also meet incredible people who are friendly, intelligent, and passionate. Mission Control is a company that I won't forget because of the unbelievable opportunity and experience it gave me. It's a company I'm very thankful for and will recommend it to anyone.
WATonomous is a creative student design team at the University of Waterloo, with the ambitious goal of developing a fully autonomous Level 5 self-driving vehicle. In 2021, WATonomous placed 2nd in the SAE International AutoDrive Challenge! On WATonomous,Β I've been a member of the Perception and State Estimation team.
On the Perception Team, I performed Multi-Modal Object Detection by implementing the BEVFusion sensor fusion algorithm. This algorithm fused 2D images from stereo cameras with 3D point cloud data from LiDAR to create 3D bounding boxes used for detecting pedestrians, vehicles, and traffic signs. Additionally, I developed a ROS2 node in Python that subscribed to ROS2 topics containing 2D images and 3D LiDAR point cloud data from the NuScenes dataset. After performing BEVFusion with PyTorch, this node published bounding box predictions to the detections_3d topic.
On the State Estimation team, I worked on developing a ROS2 node in C++ that publishes ego vehicle pose estimates from the NovAtel OEM7 GNSS sensor to a localization node. This localization node then performs state estimation by fusing IMU data and wheel odometry with NovAtel OEM7 pose estimates using an Unscented Kalman Filter. Next, the node calculates the translation and rotation from the initial position with the wheel odometry and IMU data, estimates the current velocity, and localizes the ego vehicle.
These experiences at WATonomous have not only honed my technical prowess but also fostered a passion for pushing the boundaries of autonomous vehicle technology.
UWAFT is a reputed student design team at the University of Waterloo, with the goal of winning the EcoCAR Mobility Challenge, a four-year autonomous driving competition between 12 North American universities. On UWAFT, I've been a member of the Connected and Automated Vehicle (CAV) Team as a Perception Engineer.
During my time here, I had the opportunity to work on cutting-edge perception systems in the realm of autonomous driving. I developed a semantic segmentation model using a ResNet-based CNN architecture in PyTorch, enabling the classification and segmentation of complex road scenes. This model achieved impressive results, marking a significant leap in understanding scene semantics.
In parallel, I tackled state estimation and localization by integrating IMU data with LiDAR point cloud data through an Unscented Kalman Filter (UKF). This fusion process proved critical in achieving reliable positioning in dynamic environments. To further refine pose estimation, I employed the Iterative Closest Point (ICP) algorithm to align 3D point clouds from successive LiDAR scans, substantially improved the systemβs overall robustness.
These experiences at UWAFT have not only expanded my expertise in autonomous systems but also fueled my passion for advancing the frontiers of automotive technology.
Page updated
Google Sites
Report abuse