We’re like a dog with two tails to announce we’ve got a 3-million grant to build a forklift operator training system. It will utilize augmented reality (AR) and computer vision to automatically verify the correctness of the performed tasks and swiftly identify mistakes.
The project will last until the end of November 2023, and its total value is PLN 5,481,245, including PLN 3,288,747 grant from the National Center for Research and Development.
Artificial intelligence instead of instructors, is it even possible?
The project’s aim is to support hands-on training in industrial environments. Trainees will follow the instructions and receive feedback displayed in front of their eyes thanks to AR glasses.
Automatic verification of the performed tasks correctness will be the system’s most essential feature. The operation will be possible thanks to data collected by cameras and sensors mounted on the forklift, maneuvering area, and on the transported load. On top of that, the deep learning models will estimate human poses and recognize sequences of movements.
We plan to use a detection algorithm based on ArUco markers implemented in an OpenCV library and develop a positioning methodology based on various markers. This way, we’ll be able to track objects in the entire maneuvering area.
Answering the initial question, we have reason to believe that it is possible for AI to be serving the role of instructors. Even better, we’ve already built a prototype of the system for one of our clients in the automotive industry.
The expected results
We want to tackle some common challenges that industrial companies face regarding knowledge transfer.
An automated system will:
- significantly reduce the costs of practical training
- solve the problem of qualified trainers low availability
- provide a unified training process, so that all employees in a company have the same knowledge and skills
- allow a quick introduction of curriculum changes resulting from, for example, new regulations or equipment.
Moreover, we can use the system further on to automate other industrial processes, such as quality control, safety, or workstation ergonomics.
