Master's Thesis in Robotics · JAMK University of Applied Sciences, Finland
Hybrid-Adaptive BCI
Robotic Hand Control System
An end-to-end Brain–Computer Interface pipeline that transforms EEG motor-imagery signals into semantic robotic commands through adaptive AI and a robot-independent ROS2 architecture.
[02]Interactive Digital Twin
Interactive system visualization — digital twin / conceptual visualization. The scene below is a presentation layer for the implemented research architecture described in the sections that follow — it renders the documented pipeline and lets you step through it, but it is not a live EEG backend and not a live Gazebo backend running in your browser.
Interactive 3D Experiment
Explore the EEG-to-robotic-hand pipeline
A real-time WebGL digital twin of the implemented thesis pipeline — a research prototype, not a live backend. Select a command, start the experiment, and follow it through preprocessing, classification, the adaptive confidence gate, ROS 2 and Gazebo, to the simulated robotic hand.
[03]Research Overview
The system is an end-to-end pipeline from brain signal to robot motion — a single architecture spanning three disciplines, each with its own well-established tools, meeting at explicit interfaces rather than one monolithic pipeline.
Neuroscience
- EEG
- Signal processing
- Filtering
- ICA
- Epoching
- CSP
Artificial Intelligence
- Machine learning
- CNN
- LSTM
- CNN-LSTM
- Adaptive threshold
- Real-time prediction
Robotics
- ROS2 Jazzy
- Robot abstraction
- ros2_control
- Gazebo Harmonic
- MoveIt2
- Robotic hand
[04]System Pipeline
EEG → preprocessing → CSP feature extraction → CNN/LSTM → adaptive threshold → real-time prediction → ROS2 bridge → robot abstraction → ros2_control → Gazebo Harmonic → MoveIt2 → robotic hand. Select any lab or ROS2 stage below for its purpose, data flow and real source file.
[05]EEG Signal Processing
EEGBCI subject 1, run 4
Dataset
29
Epochs
64
Channels
160 Hz
Sample Rate
3
Classes
Preprocessing
Signal Flow
[06]CSP Feature Extraction
Components
4
Feature Type
Log-variance features
Fit Scope
Fitted on training epochs only
CSP is fit exclusively on the training split, then applied unchanged to validation/test epochs — the fitted spatial filters never see held-out data, which is what makes the resulting features leakage-free.
[07]AI Classification
Six classifiers were evaluated against the same leakage-free CSP features — three classical machine-learning baselines and three deep-learning architectures.
Deployed Model
CNN-LSTM
Model Size
≈ 356 KB
CPU Inference
Sub-millisecond per window on CPU
Architecture
- Input layer→
- Conv1D (32 filters)→
- MaxPooling1D→
- LSTM (64 units)→
- Dropout (30%)→
- Dense hidden layer (32 neurons)→
- Softmax output
Training
Adam optimizer · EarlyStopping, ModelCheckpoint, ReduceLROnPlateau
[08]Adaptive Decision
A confidence gate decides whether a prediction becomes a command. It adapts to recent accuracy: a model that has been reliable gets a more permissive gate; a model that hasn't, or one that hasn't collected enough feedback yet, is held to a stricter one.
High Accuracy
More permissive — the gate relaxes as the decoder proves reliable.
Low Accuracy
More conservative — the gate tightens rather than risk an unintended command.
Cold Start
Conservative by design — fewer than 10 feedback samples holds the threshold at 0.90, regardless of measured accuracy.
Current Adaptive-Layer State
- Feedback samples
- 3
- Feedback accuracy
- 100%
- Samples remaining
- 7
- Status
- WAITING
The interactive demo above gates each run against a fixed 0.75 threshold (documented as the Lab 14 simulated real-time gate) — distinct from this cold-start bookkeeping, which governs when the online classifier itself is allowed to update.
[09]Robot-Independent ROS2 Architecture
Six single-purpose ROS2 Jazzy packages carry a prediction from the AI pipeline's output file to a robot in Gazebo, with MoveIt2 handling motion planning.
bci_interfaces
BCICommand / RobotState messages + ExecuteGesture service — the robot-independent vocabulary
bci_bridge
Reads the AI pipeline's Lab 13/14 outputs, gates by confidence, publishes BCICommand
bci_robot_abstraction
RobotInterface contract, gesture library, robot registry, robot_node
bci_hand_description
URDF/Xacro, ros2_control, Gazebo launch
bci_hand_moveit_config
MoveIt2 planning group + six named gesture states
bci_bringup
Hand trajectory adapter + full-system launch
Command Flow
Topic / Service Interface
| Topic | Type | Direction |
|---|---|---|
| /bci/command | bci_interfaces/BCICommand | bridge → robot_node |
| /bci/robot_state | bci_interfaces/RobotState | robot_node → monitors |
| /bci/joint_targets | std_msgs/Float64MultiArray | robot_node → hand adapter |
| /bci/execute_gesture | bci_interfaces/ExecuteGesture | any client → robot_node |
| /hand_controller/joint_trajectory | trajectory_msgs/JointTrajectory | hand adapter → ros2_control |
Engineering Contribution — Robot Independence
Above /bci/command everything is semantic — a gesture id plus a confidence. The AI pipeline and the bridge never know what a "hand" is. robot_node selects a concrete robot from a registry via a single robot_type parameter and calls RobotInterface.execute_gesture(); it still contains no hand-specific code. The hand trajectory adapter, in bci_bringup, is the only place that knows the hand's radian range and controller topic.
Adding a new robot
- 1.Implement the contract — a new RobotInterface subclass
- 2.Register it in the robot registry
- 3.Select it at launch via the robot_type parameter
These are documented as future extension targets, not implemented robots — adding one is a registry entry and a new RobotInterface subclass, with no change to the EEG/AI pipeline.
[10]Robotic Hand
5
Fingers
1 revolute
Joints / Finger
0–1.4 rad
Range
6
Gestures
Gesture Explorer
OPEN HAND
[11]Gazebo + MoveIt2
[12]Results
The scientific contribution of v1.0 is a leakage-free, reproducible pipeline rather than a claim of high generalization performance.
6-sample test set · single subject · post-leakage-fix scientific validation
| Model | Accuracy | |
|---|---|---|
| SVM | 0.33 | |
| Random Forest | 0.50 | |
| XGBoost | 0.33 | |
| CNN | 0.50 | |
| LSTM | 0.50 | |
| CNN-LSTMdeployed | 0.67 |
“These numbers are low but trustworthy — they are the true difficulty of a 29-epoch, single-subject dataset.”
[13]Engineering Validation
7
AI Scientific-Invariant Tests
36
ROS2 Tests
43
Total
Verification Split
- AI pipeline validation (CSP + model predict)Development hostGitHub Actions CI
- Leakage-free CSP fit (train-only)Development hostGitHub Actions CI
- Validation/test separation (deep learning)Development hostGitHub Actions CI
- Adaptive threshold + cold-startDevelopment hostGitHub Actions CI
- 36 ROS2 unit/structural testsDevelopment hostGitHub Actions CIROS2 host
- URDF validation (expands, valid kinematic tree)Development hostGitHub Actions CI
- Gesture consistency (SRDF ↔ abstraction ↔ URDF)Development hostGitHub Actions CI
- colcon buildGitHub Actions CIROS2 host
- colcon testGitHub Actions CIROS2 host
- Gazebo Harmonic spawn + controllersROS2 host
- MoveIt2 planning to named statesROS2 host
- Live ROS2 transportROS2 host
[14]Limitations
Known, stated plainly
- The current real-time path does not yet produce robotic motion from live EEG: available real-data prediction confidence remains below the adaptive acceptance threshold, so the gate correctly withholds a command rather than acting on an uncertain one.
- Bundled high-confidence demo predictions are used to exercise the complete motion path end-to-end (bridge → robot abstraction → ros2_control → Gazebo → MoveIt2), independent of that gating behavior.
- The evaluated classifier is trained and tested on a single-subject, 29-epoch dataset with a 6-sample test set — the results above are honestly small-sample, not a generalization claim.
- The trained EEG classifier currently distinguishes 3 motor-imagery classes from that dataset; the full 6-gesture robotic vocabulary is exercised through the bundled demo predictions, not yet driven end-to-end by the live 3-class classifier.
[15]Future Research
- 01
Multi-Subject Training
Scale training data across EEGBCI subjects (109 available) instead of a single subject.
- 02
Subject-Wise Cross-Validation
Validate with a subject-wise split, the proper protocol for generalization claims.
- 03
Live EEG Prediction Topic
Replace CSV replay with inference running directly on a live EEG stream.
- 04
Causal Streaming Filtering
Replace offline filtering with causal real-time filtering (streaming lfilter with carried state).
- 05
UR5e / Franka Panda Extension
Add a second robot to demonstrate the abstraction layer on an industrial arm.
- 06
Collision-Aware Planning
Use MoveIt2 planning scenes for collision-aware grasp tasks.
- 07
Physical Robotic-Hand Bridge
Replace the Gazebo controller with a real hand's driver behind the same interface.
- 08
True Online Adaptive Learning
Deploy online fine-tuning on accumulated user feedback, beyond the current threshold adaptation.
- 09
Larger Gesture Vocabulary
Extend the trained classifier beyond 3 classes to drive all 6 supported gestures directly.
[16]Technology Stack
EEG / Signal Processing
- MNE-Python
- SciPy
- NumPy
Feature Extraction
- MNE CSP
Machine Learning
- scikit-learn
- XGBoost
Deep Learning
- TensorFlow
- Keras
- CNN
- LSTM
- CNN-LSTM
Robotics
- ROS2 Jazzy
- ros2_control
Simulation
- Gazebo Harmonic
Motion Planning
- MoveIt2
[17]Research Repository
v1.0 — complete — EEG→AI→ROS2→robotic-hand pipeline implemented across six ROS2 packages plus the Labs 01–14 AI pipeline.
The AI pipeline and the ROS2 workspace meet at one seam — the bridge reads the AI pipeline's own output files — so the AI code is never modified by the robotics code.