The AI-Consult project develops a system that provides intuitive access to contextual information. Through natural and low-threshold communication in combination with optical recognition methods, a multifunctional 2D/3D scanner is intended to provide savvy users with direct and non-contact access to a wide range of functions. Communication between humans and the system takes place via a multimodal interface. This fuses the capture and output of natural language, the graphical representation of data, and the three-dimensional representation of the interlocutor. To ensure data protection, the project partners implement personal image and speech data processing by an integrated computing unit.
First, the system requirements are determined on the basis of application scenarios in the logistics and construction industries. These differ in terms of user experience, process complexity and environmental conditions. A user study evaluates the applicability of the overall system. The hardware and software development consists of the development of different AI models and the creation of tools for the management of the training data. A data acquisition campaign extends the training data and the system will be adapted to the application scenarios.
The project had a duration from 01.04.2022 to 31.07.2024 and the results are published both on the project website and in a video.
