EMG-controlled hand exoskeleton

Overview

Mr. Handy is a wearable hand exoskeleton developed by the Sentry Robotics team at The Ohio State University. It assists people with limited hand mobility from conditions such as osteoarthritis or post-surgical stiffness. The device translates forearm muscle activation, captured by surface EMG sensors, into classified hand gestures. Those gestures drive a tendon-based motor system that physically assists finger movement.

My role centered on the data pipeline: choosing electrode placements across three forearm muscle groups, designing and running a multi-subject data collection protocol, training a gesture classification model, and implementing the serial link between the ESP32 microcontroller and the Raspberry Pi motor controller.

Problem

The primary goal was a reliable real-time gesture classifier that could distinguish 12 hand gestures from three-channel forearm EMG with greater than 85% accuracy and under 1 second of latency. A secondary goal was a repeatable, scalable data collection protocol that produces a balanced, high-quality training set across a diverse subject pool, so the model can generalize.

Approach

Three MyoWare 2.0 muscle sensors were placed over the flexor carpi ulnaris, flexor carpi radialis, and extensor pollicis brevis, sampled at 20 Hz through the ESP32 ADC. Per-channel moving average and standard deviation over 50 ms windows served as features for two classifiers: a manually calibrated threshold classifier for early prototype validation, and a CNN-based model trained with Google Cloud Vertex AI AutoML.

A structured 9-minute guided video protocol collected 9,500 labeled samples across 12 gesture classes from 14 subjects. The ESP32 and Raspberry Pi communicate over UART, with matching baud rates in the ESP32 firmware and a Python serial listener on the Pi.

Results

The threshold classifier was integrated and validated end to end. It drove motor actuation correctly for the REST, FIST, and PINCH gestures within the required latency.

The first machine learning iteration produced a degenerate classifier, caused by class imbalance and a data ingestion misconfiguration in the cloud training pipeline. That pointed to concrete fixes for the next iteration: rebalancing classes, correcting the labeling window, and validating the dataset before retraining.

The project built practical skills in analog electronics, embedded firmware, experimental design, and applied machine learning across a full biomedical system development cycle.