Coffee Barista

A robot arm is tasked with pouring beads representing different coffee ingredients

Team: Ankit Aggarwal, Aayush Fadia, Shreya Shri Ragi, Swastik Mahapatra

In a simulated coffee preparation environment, a robot arm is tasked with pouring beads representing different coffee ingredients (milk, coffee, foam, chocolate) into a cup based on user input for various coffee types (latte, americano, espresso). The system must:

  • Controlled Pouring: Pour beads in a controlled manner to achieve precise ingredient ratios
  • Constrained Manipulation: Pick and move ingredient cups containing beads without spilling
  • Obstacle Avoidance: Avoid collisions with task apparatus while moving in the environment

Setup

  • RealSense Camera
  • Weighing Scale
  • 2 Ingredient Cups
  • Output Cup (Placed anywhere on scale)


Methodology

Perception

The major perception component of our project is to localize and check the output coffee cup.

  1. Detect cup placed on the platform: Confirm if an output cup is present on the preparation platform from a set perceive pose.
  2. Localize the rim of the cup: Transform the data to global map for planner.
  3. Localize the cup in 3D space using depth data and determine its bounding cylinder (radius + depth).
  4. Register the resultant cylinder to the collision space.

Motion Planning

Motion planning is divided into three phases - Grasp Ingredient Cup, Bring ingredient cup above Coffee Mug and Replace Ingredient Cup

  1. Grasp Ingredient Cup
    • Start: Arm Home Configuration.
    • Goal: Place fingers horizontal and around the cup.
    • Obstacles to Avoid: All objects on table - tray, cups, mug, scale, virtual walls, and arm itself.
    • Approach: Inform MoveIt about all obstacles (known/predefined and from perception). Allow MoveIt to plan and execute a path.
  2. Bring Ingredient Cup above Coffee Mug
    • Start: End effector grasps Ingredient Cup in its original location.
    • Goal: Ingredient cup is directly above coffee mug.
    • Constraints: Keep cup vertical.
    • Approach: MoveIt plans a non-colliding, constraint respecting path.
  3. Replace Ingredient Cup
    • Start, Goal: Reverse of Above.
    • Constraints: Same as above.
    • Approach: Plan a path via MoveIt to place ingredient cup back in original spot.

Pouring

Pouring beads in a controlled and precise manner requires robust and accurate feedback.

  1. Feedback Method
    • A weighing scale will be used to weigh the amount of beads being poured into a cup.
    • Fixed weight values will be defined as goals for the lookup table.
  2. Lookup Table Method
    • A lookup will be used to control the pouring with a focus on the amount of beads in the ingredient cup to select the correct angle to control flow rate to satisfy target pour amount.
    • Weight of beads in the ingredient cup is obtained as feedback from the end effector’s force torque sensor
    • A control angle and offset weight will be defined for each ingredient cup weight range
  3. Manipulation using Control Values
    • The control angle will used to create a goal pose of the end effector.
    • The goal pose will be reached using inverse kinematics of the franka arm.
    • The offset weight will be used as a stop condition for pouring to control over-pour.