realvirtual.io AI Builder Overview
Last updated
Last updated
The AI Builder can be found in the Unity project directory under realvirtual-AIBuilder. This directory is integrated into the realvirtual.io Professional environment and includes several key components and demo scenes. Below is an overview of the folder structure and its contents.
The main folder contains example scenes that demonstrate how to use AI Builder for various tasks, such as object detection and automation. These scenes showcase how AI models can be trained, tested, and deployed for virtual commissioning.
This scene demonstrates the full AI workflow using a Lego brick sorting system. In the demo, four different types of Lego bricks are detected and automatically sorted into four corresponding boxes by a robotic arm.
The scene can be used for:
Training: Preparing the AI model to recognize different types of Lego bricks.
Testing: Verifying the model's detection capabilities within a virtual environment (Digital Twin).
Virtual Commissioning: Testing the real robot control system in conjunction with the trained AI model to ensure accurate sorting in real-world conditions.
The CameraDeviceDemo scene is designed for real-world detection tasks using a trained ONNX model from the AIBuilder demo. In this scene, you can perform object detection (inference) by leveraging live video input from a device camera, such as a webcam. This scene extends the functionality of AI Builder from simulated environments into real detection scenarios.
The LabelConditions folder contains label definitions and conditions that are used during the training process. These labels represent the categories that the AI model needs to recognize, such as different types of objects or Lego bricks in the demo.
The ONNX folder contains ONNX models, which are pre-trained or exportable AI models used for object detection and recognition. The AI Builder leverages the ONNX format for running AI inferences within Unity, allowing the system to make real-time decisions based on visual data.
The Variators folder contains a collection of scripts that are designed to introduce variations into the training process. These variations help in generating large amounts of synthetic AI training data, making the AI model more robust and better equipped to handle real-world scenarios with diverse data.
The Demo folder contains all the assets and scripts required to run the demo scenes in realvirtual.io AIBuilder. These assets showcase the core functionalities of AIBuilder, including object detection, sorting, and AI training within Unity.
