The goal of this project is to design and implement an autonomous cake-decorating robotic system capable of applying icing patterns accurately and consistently without human intervention. The system uses a 6-DOF UR7e robotic arm equipped with a custom syringe-based icing end effector, along with an overhead vision system to detect cake geometry and generate corresponding 2D decoration trajectories. By combining perception, planning, and precise actuation, the robot is able to adapt its motion to different cake sizes and shapes while maintaining smooth and controlled icing flow.
Click here to view the full website for this project
Why Cake Icing?
Creating a system that senses and decorates different types of cake requires combining multiple subsystems into a single pipeline, and that complexity is what makes this such an interesting problem to solve. To achieve end-to-end functionality, we needed to solve problems in computer vision (detecting cake boundaries and extracting clean outlines), trajectory planning (generating smooth, continuous 2D paths that respect robot kinematics), and robot control (coordinating arm motion with consistent icing extrusion). Additionally, the system must handle real-world imperfections such as camera noise, alignment errors, and variations in icing behavior, making robustness and calibration important components of the design.
While hardware for this project was not complex, precise setup was an important part of creating accurate trajectories. As "cake" position is determined based on the transform from the base link to the camera, any slight deviation from measured location is amplified.
Implementation of Trajectory Planning
1. Subscribes to sensing topics: /shape, /shape_center, /raw_shape_waypoints
2. Use sensing topics as inputs for determining trajectory of cake
3. Create bounds for the trajectory to fit in based on sensing-calculated cake size
4. Create designs for shape (either outline cake or draw new shape) depending on the type of shape
5. Publish to trajectory planning topics: /trajectory_list
Implementation of Actuation
1. Subscribe to sensing topic /shape_center and UR7e driver readings from controller /joint_states
2. Trajectories determined using information from topics
3. Load IKPlanner to compute IK and plan trajectories
4. Create job queue created by appending joint states, executed with client
5. Action client for robot arm following trajectory
6. Gripper position set using socket client (not in job queue)
7. Custom gripper for dispensing icing starts dispensing after first job finishes
It took multiple tries to adjust our trajectories and achieve accurate shapes. Each shape is comprised of a series of waypoints. As the arm goes from waypoint to waypoint, icing is dispensed.
What improvements would we make if we had additional time?
With additional time, the system could be made more robust and accurate. The transform from the camera pixels to robot coordinates requires manually measuring the distance from the robot base link to the camera every time it is placed, a process that results int he need for tuning and offsets. A more robust transform could mitigate this. A mounting system that places the camera in the exact same spot each every time would result in more consistency. Another approach could involve AR tags located directly on the cakes.
For the design trajectories, the code could be made more modular, such as using static_transform.py rather than coding the transform in shape_detector.py and utilizing functions for each set of waypoints.
The trajectories could be improved to create cleaner designs. Currently, the velocity of the end effector changes, accelerating and decelerating point-to-point. This could be improved by adjusting the maximum velocity. Alternatively, a different approach to computing trajectories could be used, allowing the way-points to be incorporated into one trajectory instead of several.
At the end of each trajectory, the robot arm blocks the camera, making it impossible to detect subsequent shapes on the table. This could be resolved by changing the placement of the camera, or ensuring the arm is out of view at the end of each design trajectory.
For the version of the project shown on demo day. knowing the height of each cake and the distance relative to the base link, the height of the end effector could be hard-coded. However, cakes come in many shapes, sizes, and heights. For a more versatile implementation, depth sensing was explored. A topic was created to report the distance of the cake from the camera, establishing proof of concept. With more time, this topic could be implemented to apply icing to cakes of different heights.
While there is room for improvement, the design effectively senses the cakes and deposits icing, resulting in various patterns.