A new research project THEIA XR aims to improve human-machine interaction in mobile machinery by enhancing user technology fit and extended reality (XR) technologies and functionalities. The project’s goal is to make the invisible visible and extend the perceivable range of the human operator without impacting their performance. The three-year project is funded with a total of six million euros from the European Commission’s Horizon Europe programme.
THEIA XR will be validated and tested in three use cases in the off-highway domain: snow grooming, logistics, and construction. The project will be carried out by a consortium of 11 project partners from industry and academia. It is coordinated by high-tech company TTControl, a joint venture of TTTech and HYDAC International, based in Vienna and Brixen.
“Integrating XR technologies into TTControl’s HMI products for mobile machinery is another step towards more intelligent, trustworthy off-highway operations,” said Christiana Seethaler, vice president of product development at TTControl.
Enhancing conventional human-machine interfaces with XR
Mobile machines, such as excavators, snow groomers, or cranes, are often large and bulky, and sometimes controlled remotely. It can be a challenge for a human operator to maintain an overview of the machine and its environment. XR technologies can expand the operator’s field of vision and improve his or her confidence in human-machine interaction.
For instance, augmented reality (AR) and virtual reality (VR) technologies may be used to display machine status in the operator’s immediate field of vision. Haptic feedback could be used to send tactile signals to operators, for example via a joystick, and acoustic feedback can be used to provide warning signals to the user. Combined with external sensors, these technologies can also help detect obstacles and other factors of the environment, improving not just the safety of the operator and bystanders, but also increasing efficiency as work can be completed with more foresight.
A human-centered approach
The project applies a transdisciplinary co-design approach, where all relevant stakeholders of the use case partners are involved in the specification, implementation, testing, and validation activities and decisions of the project, contributing as experts in their respective domains.
Martijn Rooker, Innovation Projects & Funding Manager at project lead TTControl explains: “The XR technology used in mobile machinery must be intuitive enough to enable low-threshold operation in many different scenarios. Therefore, THEIAXR will apply a human-centered, scenario-based co-design methodology. This means that real end-users and real-life data from industrial environments will be integrated to guarantee all solutions are as intuitive and user-friendly as possible.”
This way, the project will ensure a positive impact of extended reality technologies on the safety, security, trustworthiness, and societal aspects of the interaction between the machine, the human operator, and the public. Rooker: “Being able to interact with the machine using their inherent human intuition, operators should experience an increased sense of self-efficacy and meaningfulness in their work. Ultimately, that is our goal.”
THEIA XR will run until the end of 2025. The project has received funding from Horizon Europe, the EU’s key funding program for research and innovation, under grant agreement No 101092861.
