Lesson 28: Gripper/Manipulator Integration

1. Introduction to Robotic Manipulation

Robotic manipulation is the ability of a robot to interact with and modify its environment through physical contact. This involves:

Key Components:

• End Effector: The gripper or tool at the end of the robotic arm
• Actuators: Servo motors that control joint movement
• Sensors: Force, position, and vision sensors for feedback
• Control System: Algorithms that coordinate movement and force

Types of Grippers:

• Parallel Jaw: Two fingers that move parallel to grasp objects
• Angular: Fingers that rotate around a pivot point
• Suction: Uses vacuum to pick up smooth, flat objects
• Magnetic: Uses electromagnets for ferrous materials

2. Servo Motor Control Fundamentals

Servo motors provide precise position control essential for robotic manipulation:

Servo Control Basics:

// Basic servo control setup
#include <Servo.h>

Servo gripperServo;
Servo armServo;

void setup() {
  Serial.begin(115200);
  
  // Attach servos to pins
  gripperServo.attach(9);   // Gripper servo on pin 9
  armServo.attach(10);      // Arm servo on pin 10
  
  // Initialize to neutral positions
  gripperServo.write(90);   // Open position
  armServo.write(90);       // Neutral arm position
  
  delay(1000);
}

void loop() {
  // Demonstrate basic movements
  pickAndPlace();
  delay(2000);
}
                

Precise Position Control:

// Smooth servo movement function
void moveServoSmoothly(Servo &servo, int startPos, int endPos, int stepDelay) {
  int currentPos = startPos;
  int step = (endPos > startPos) ? 1 : -1;
  
  while (currentPos != endPos) {
    servo.write(currentPos);
    delay(stepDelay);
    currentPos += step;
  }
}

// Gripper control functions
void openGripper() {
  moveServoSmoothly(gripperServo, gripperServo.read(), 180, 15);
  Serial.println("Gripper opened");
}

void closeGripper() {
  moveServoSmoothly(gripperServo, gripperServo.read(), 0, 15);
  Serial.println("Gripper closed");
}
                

3. Gripper Mechanics and Force Control

Understanding the mechanical principles behind effective gripping:

Force and Torque Considerations:

• Grip Force: Sufficient force to hold objects without crushing
• Contact Points: Multiple contact points for stable grasping
• Friction: Surface texture affects grip reliability
• Object Geometry: Shape determines optimal grip strategy

Adaptive Gripping Algorithm:

// Adaptive gripper with force feedback
int gripperCurrentPin = A0;  // Current sensor for force feedback
int maxGripCurrent = 500;    // Maximum safe current reading

bool adaptiveGrip(int targetObject) {
  Serial.println("Starting adaptive grip sequence...");
  
  // Open gripper fully
  openGripper();
  delay(500);
  
  // Slowly close until contact detected
  for (int pos = 180; pos >= 0; pos -= 2) {
    gripperServo.write(pos);
    delay(50);
    
    // Check current draw (force feedback)
    int current = analogRead(gripperCurrentPin);
    
    if (current > maxGripCurrent) {
      Serial.println("Object detected - grip achieved");
      return true;
    }
  }
  
  Serial.println("No object detected");
  return false;
}
                

4. Vision-Guided Object Manipulation

Integrating computer vision with manipulation for autonomous object handling:

Object Detection and Positioning:

// Vision-guided picking system
struct ObjectPosition {
  int x, y;
  int size;
  bool detected;
};

ObjectPosition detectObject() {
  ObjectPosition obj = {0, 0, 0, false};
  
  // Capture image from ESP32-S3 camera
  camera_fb_t *fb = esp_camera_fb_get();
  if (!fb) {
    Serial.println("Camera capture failed");
    return obj;
  }
  
  // Simple color-based object detection
  // (In practice, use more sophisticated algorithms)
  int targetPixels = 0;
  int centerX = 0, centerY = 0;
  
  for (int y = 0; y < fb->height; y += 4) {
    for (int x = 0; x < fb->width; x += 4) {
      // Check if pixel matches target color
      if (isTargetColor(fb->buf, x, y, fb->width)) {
        centerX += x;
        centerY += y;
        targetPixels++;
      }
    }
  }
  
  if (targetPixels > 50) {  // Minimum object size
    obj.x = centerX / targetPixels;
    obj.y = centerY / targetPixels;
    obj.size = targetPixels;
    obj.detected = true;
    
    Serial.printf("Object detected at (%d, %d), size: %d
", 
                  obj.x, obj.y, obj.size);
  }
  
  esp_camera_fb_return(fb);
  return obj;
}
                

Coordinate Transformation:

// Convert camera coordinates to robot coordinates
void moveToObject(ObjectPosition obj) {
  if (!obj.detected) return;
  
  // Camera-to-robot coordinate transformation
  // These values need calibration for your specific setup
  float pixelToMM = 0.5;  // mm per pixel
  float cameraOffsetX = 50;  // Camera offset from gripper center
  float cameraOffsetY = 30;
  
  // Calculate real-world coordinates
  float objectX = (obj.x - 160) * pixelToMM + cameraOffsetX;  // 160 = camera center
  float objectY = (obj.y - 120) * pixelToMM + cameraOffsetY;  // 120 = camera center
  
  Serial.printf("Moving to object at (%.1f, %.1f) mm
", objectX, objectY);
  
  // Move robot base to position gripper over object
  positionRobotBase(objectX, objectY);
  
  // Lower arm and attempt grip
  lowerArm();
  if (adaptiveGrip(1)) {
    Serial.println("Object successfully grasped");
    raiseArm();
  }
}
                

5. Coordinated Movement Strategies

Coordinating base movement with arm manipulation for complex tasks:

Multi-DOF Control:

// Coordinated pick and place operation
void pickAndPlace() {
  Serial.println("Starting pick and place sequence");
  
  // Phase 1: Search for object
  ObjectPosition target = scanForObject();
  if (!target.detected) {
    Serial.println("No object found");
    return;
  }
  
  // Phase 2: Approach object
  approachObject(target);
  
  // Phase 3: Pick up object
  if (pickUpObject()) {
    Serial.println("Object picked up successfully");
    
    // Phase 4: Move to drop location
    moveToDropZone();
    
    // Phase 5: Release object
    releaseObject();
    
    // Phase 6: Return to home position
    returnHome();
  }
}

ObjectPosition scanForObject() {
  Serial.println("Scanning for objects...");
  
  // Rotate base while scanning with camera
  for (int angle = -90; angle <= 90; angle += 30) {
    rotateBase(angle);
    delay(500);  // Allow movement to settle
    
    ObjectPosition obj = detectObject();
    if (obj.detected) {
      Serial.printf("Object found at base angle %d degrees
", angle);
      return obj;
    }
  }
  
  // Return to center if no object found
  rotateBase(0);
  return {0, 0, 0, false};
}
                

6. Inverse Kinematics Basics

Mathematical approach to determining joint angles for desired end-effector positions:

2-DOF Arm Kinematics:

// Simple 2-DOF arm inverse kinematics
struct ArmConfiguration {
  float shoulderAngle;
  float elbowAngle;
  bool reachable;
};

ArmConfiguration calculateIK(float targetX, float targetY) {
  ArmConfiguration config = {0, 0, false};
  
  // Arm segment lengths (in mm)
  float L1 = 100;  // Upper arm length
  float L2 = 80;   // Forearm length
  
  // Calculate distance to target
  float distance = sqrt(targetX * targetX + targetY * targetY);
  
  // Check if target is reachable
  if (distance > (L1 + L2) || distance < abs(L1 - L2)) {
    Serial.println("Target position not reachable");
    return config;
  }
  
  // Calculate elbow angle using law of cosines
  float cosElbow = (L1*L1 + L2*L2 - distance*distance) / (2*L1*L2);
  config.elbowAngle = acos(cosElbow) * 180.0 / PI;
  
  // Calculate shoulder angle
  float alpha = atan2(targetY, targetX) * 180.0 / PI;
  float beta = acos((L1*L1 + distance*distance - L2*L2) / (2*L1*distance)) * 180.0 / PI;
  config.shoulderAngle = alpha + beta;
  
  config.reachable = true;
  
  Serial.printf("IK Solution: Shoulder=%.1f°, Elbow=%.1f°
", 
                config.shoulderAngle, config.elbowAngle);
  
  return config;
}
                

7. Advanced Applications

Real-world applications of robotic manipulation:

Industrial Applications:

• Assembly Lines: Automated part insertion and assembly
• Quality Control: Automated inspection and sorting
• Packaging: Pick and place operations for packaging
• Material Handling: Warehouse automation and logistics

Service Robotics:

• Healthcare: Surgical assistance and patient care
• Domestic: Household cleaning and organization
• Agriculture: Automated harvesting and planting
• Food Service: Automated cooking and serving

8. Assessment & Homework

Quick Check Questions:

1. What are the main components of a robotic manipulation system?
2. How does force feedback improve gripping performance?
3. What is inverse kinematics and why is it important?
4. How do you coordinate base movement with arm manipulation?
5. What safety considerations are important for robotic arms?

Programming Assignment:

Design Challenge: Create a "Smart Warehouse" system where your miniAuto robot can:

• Identify different colored blocks representing inventory items
• Pick up items and sort them into designated storage areas
• Keep track of inventory levels and report when areas are full
• Handle objects of different sizes with adaptive gripping
• Optimize movement paths to minimize sorting time

Bonus Challenge: Implement a "restocking" mode where the robot can retrieve specific items on command and deliver them to a pickup location.