Enhancing human-robot interaction in virtual environments through brain-computer interfaces

We are looking for a PhD student to work on Brain-Computer Interfaces in the human-robot collaboration topics with two groups: 1. Smart Industry research group; 2. Automated Production Systems and Real-Time Monitoring and AI Models research group.

Brain-Computer Interfaces (BCI), concern with the idea of directly interfacing human user's brain activity into the digital system. This research field is getting more traction recently with efforts from companies like Neuralink, Valve, and OpenBCI. Devices developed in the upcoming years are likely to lay the ground for new, unconventional, and more effective ways of interacting with manufacturing systems and robots.

The proposed research's primary purpose is to develop the Virtual Reality (VR) framework, which will utilize BCIs to enable interactions that exceed the scope of conventional 3D tracking provided by modern VR headsets. It includes using brain signals for attention and engagement analysis and adding extra perception channels to simulation (haptics, touch, temperature). Naturally, this research results would convert into further improvement of Digital Twin interfaces being developed in IVAR Laboratory [1], [2].

Ultimately, the task is to prepare a base for developing a way to decode human body motions through the BCI. This can eliminate the need for external tracking, providing a way for controlling VR avatars, robotic- limbs, and prosthetics without external tracking. Though medical research is out of the scope on the initial stage of the project, potential results of the related BCI investigation could also serve as a benefit in projects between IVAR Laboratory and the medical institutions.

References:

[1] Kuts, V.; Otto, T.; Tähemaa, T.; Bondarenko, Y. (2019). Digital twin based synchronised control and simulation of the industrial robotic cell using virtual reality. JOURNAL OF MACHINE ENGINEERING, 19 (1), 128−145.2010.5604/01.3001.0013.0464

[2] https://ivar.taltech.ee/ [Online]

Responsibilities and tasks:

• Research on the defined topic
• Virtual Reality framework development
• Universal robot programming methodology through Digital Twins BCI/VR/AR interfaces
• Representation of the Industrial VR/AR laboratory at events, conferences, demos [2]
• Teaching assistance in Digital Manufacturing, Production Digitalization courses (labs)

Mandatory:

• Programming languages: C#/C++, Python
• Familiar with VR/AR domain
• Unity3D software knowledge
• Machine Learning / Machine Vision experience
• MatLab

Good to have:

• Robot Operation System (ROS)
• 3D Modelling (CAD/Blender/3DMax etc.)
• Knowledge of manufacturing equipment domain, especially Industrial and Mobile robots
• Knowledge about industrial telemetry solutions