Quality Assurance (QA)¶
General Description¶
The AI-enabled quality monitoring tool for built machinery aims to provide valuable features for monitoring and ensuring the quality of manufactured products. It supports various standardized data modalities used in industrial product inspection across different manufacturing domains.
One of the key features of the tool is the AI-supported 3D analysis and comparison tool, known as ZG3D, developed by ITI. This tool enables the detection of differences between an expected or ideal 3D product model and the actual 3D model captured from the produced object. These differences can include geometric tolerances, shape deformations, and texture defects such as voids, cracks, or other surface anomalies. By comparing the captured model with the expected model, the tool can identify and highlight any discrepancies, allowing for effective quality monitoring and analysis.
In addition to 3D analysis, the tool also supports visual surface anomaly detection in 2D. This is achieved through unsupervised approaches developed by XLAB, which eliminate the need for laborious data collection and labeling of anomalous samples. Instead, the algorithms can be trained using defect-free samples, enabling straightforward adaptability to new product lines right from the start. This approach streamlines the process of detecting surface anomalies and ensures flexibility in monitoring product quality across different manufacturing scenarios.
Overall, the AI-enabled quality monitoring tool leverages advanced techniques in 3D analysis and comparison, as well as unsupervised approaches for surface anomaly detection, to provide comprehensive and efficient quality monitoring capabilities for built machinery.
Usage Viewpoint¶
AI-QA module consists of two submodules – namely one for 2D visual inspection (AI-QA-2D) and one for 3D visual inspection (AI-QA-3D). Viewpoints are presented separately for each of the submodules.
Use Model AI-QA-2D¶
Description¶
Training: User defines the process (machine and material characteristics) and uploads a set of defect-free samples. The AI-QA-2D method is trained on the samples. Additionally, user uploads samples with labelled defects to support training under mixed supervision and to enable evaluation of the performance. This can be performed by the operator or supervisor.
Environment configuration: The operator configures the application with machine and material characteristics. Additional metadata can also be provided.
Deployment: AI-QA-2D solution is deployed and applied to input 2D imagery data to perform a visual inspection (QA). The operator inspects the results (correct detection of presence/absence of visual defects, as well as localization accuracy), The operator can modify the results and as such provide feedback to the AI-QA-2D module to facilitate the feedback loop (optional).
Upload 2D Image¶
Use Model AI-QA-3D¶
Description¶
Training: User defines the process (machine and material characteristics) and uploads a set of objects labeled captures. The AI-QA-3D method is trained on the samples. This can be performed by the operator or supervisor.
Environment configuration: The operator configures the application with machine characteristics. Additional metadata can also be provided.
Deployment: AI-QA-3D solution is deployed and applied to input 3D imagery data to perform a visual inspection (QA). The operator inspects the results (correct classification of incoming objects), the operator can modify the results and as such provide feedback to the AI-QA-3D module to facilitate the feedback loop (optional).
Upload 3D Image¶
Usage activity diagram¶
The following image depicts the interaction between the tasks and roles, currently targeting one pilot. Nevertheless, similar approaches will be reused, as these are the same independent of the use case. This diagram illustrates the dynamic flow of interactions, offering a representation of the activities executed by the solution providing a clear understanding of the activities executed using the AIQA solution.
Functional Viewpoint¶
Functional diagram¶
Functional description¶
What: The main functionality of this component is to detect visual defects in 2D imagery input of the product that machine is producing or represents and input to the machine.
Who: Machine operator. QA supervisor
Where: Operated on the machine itself. Solution deployed in cloud, integrated to the machine existing interfaces by the machine producer using the provided APIs.
Why: The main objective is to detect visual defects (2D) in the input or produced material to reduce waste and improve utilization of the material.
Input and output data for each component¶
Configuration¶
Inputs:
Operator needs to configure the machine characteristics (e.g., camera settings) and the material type on which visual detection will be performed.
Outputs:
Machine configuration and material type.
Training¶
Inputs:
Input represents a set of imagery samples with no visual defects for which we want AI-QA-2D module to detect defects. The AI-QA-2D module will be able to capture normal appearance and when deployed, detect deviations. Additionally, the input is also a set of imagery samples with labelled defects, which can further improve the performance of the AI-QA-2D module, when such data is available.
Outputs:
Trained model file.
Interference & Evaluation¶
Inputs:
Pre-trained model from the model repository (e.g., MLFlow) and the current imagery sample from the machine.
Additionally: Visual defects labelled by the operator for the evaluation and optimization purposes (feedback loop).
Outputs:
Decision about the presence or absence of defects on the provided imagery sample.
Localization information about the visual defects.
Updated AQ-QA-2D module statistics to the Machine Passport.
Functional diagram¶
Functional description¶
What: The main functionality of this component is to classify incoming objects in 3D imagery input of the product that machine is producing or represents and input to the machine.
Who: Machine operator. QA supervisor.
Where: Operated on the machine itself. Solution deployed in cloud, integrated to the machine existing interfaces by the machine producer using the provided APIs.
Why: The main objective is to classify objects (3D) in the input or produced material to improve the accuracy of the main visual inspection system.
Input and output data for each component¶
Configuration¶
Inputs:
Operator needs to configure the machine characteristics (e.g., camera setting).
Outputs:
Machine Configuration.
Training¶
Inputs:
Input represents a set of labeled captured samples of the incoming objects we want AI-QA-3D module to classify. The AI-QA-3D module will be able to capture appearance and when deployed, classify objects.
Outputs:
Trained model file.
Interference & Evaluation¶
Inputs:
Pre-trained model from the model repository (e.g., PyTorch) and the current labeled capture sample from the machine.
Outputs:
Decision about the object classification.
Updated AI-QA-3D module statistics to the Machine Passport.
Software requirements¶
Software Component |
Description/Role |
Required Version/Configuration |
|---|---|---|
Linux OS or Windows OS |
Operating system needed to use the tool |
Ubuntu 22.04 / Windows 10 Pro |
Docker |
Build, share and run containers |
latest |
Lifecycle¶
Training: The user defines the process (machine and material characteristics) and uploads a set of images of objects labelled as defect-free or/and with defects detected and annotated. The AIQA -2D method is trained on these samples. The training is usually performed and verified by the supervisor.
Environment configuration: The operator configures the application with machine characteristics.
Deployment: AIQA -2D solution is deployed to (usually) on-premises edge/machine environment and applied to 2D images to perform a visual inspection (QA). The operator might inspect the results (correct classification of incoming objects), the operator might also modify the results and provide feedback to the AIQA -2D module to facilitate the feedback loop which is optional.
Objects:¶
Supervisor: Represents the User interacting with the AIQA solution while generating/updating the data model.
Operator: Represents the user interfacing the edge machine in production.
Training: Represents the process of training from collected defect annotated and defect-free images of products for building the decision model.
Configuration: Represents the setup and configuration of AIQA solution on the edge device (customer/pilot machine).
Deployment: Represents the user interaction with the AIQA solution deployed
Implementation Viewpoint¶
Description of implementation Component:¶
AI-QA-2D is a 2D visual QA inspection submodule of AI-QA.
Technical Description of its Components:¶
Dependencies:
Development Language: - Python.
Libraries: PyTorch.
Container: Docker, Nvidia-Docker.
Database need: MLFlow/Databricks will be used for model and experiment versioning. Cloud-based object storage will be used for storing unstructured imagery data. MongoDB might be used for the storage of metadata.
Interfaces:
User Interface: No user interface is envisioned, only APIs for the integration to the existing end-user machine and applications. Demo user interfaces will be created using Streamlit and/or Gradio.
Synchronous/Asynchronous Interface: RESTful APIs.
Network/Protocols: HTTP/HTTPS.
Data Repository: MLFlow/Databricks will be used for model and experiment versioning. Cloud-based object storage will be used for storing unstructured imagery data. MongoDB might be used for the storage of metadata.
Requires:
Other AIDEAS Solutions: none
Solution architecture¶
AI-QA-2D solution provides a cloud-based computer-vision-based AI solution for the detection and localization of visual defects in 2D imagery data. The AI solution will be developed in PyTorch1 and provided as a service for further integration into existing pilot’s machines and/or existing software for the processing and management of images. The service will be deployed in a Dockerized fashion and will support GPU (default), as well as CPU-based execution. The application’s AI lifecycle (e.g., experiment and model versioning) will be managed using MLFlow2 (e.g., cloud managed Databrick’s MLFlow). Data will be primarily stored on the user premises (existing databases). The application will additionally support storage of imagery data in a cloud-based object storage/Datalakes3. The associated metadata will be stored primarily in NoSQL databases (e.g., MongoDB). Microsoft Azure Cloud will be used for the deployment of the solution. We don’t envision specific development of user interfaces, as the main purpose of the solution is the integrations into existing applications - either on the machine itself, or other applications that are currently used to process imagery data on the pilot side. Nevertheless, we plan to provide demo user interfaces based on Streamlit4 or Gradio5 to enable pilot-independent demonstration of the solution.
Description of implementation Component:¶
AI-QA-3D is a 3D visual QA inspection submodule of AI-QA.
Technical Description of its Components:¶
Dependencies:
Development Language: - C, Python.
Libraries: OpenCV, Tensorflow, PyTorch.
Container: Docker.
Database need: SQLite.
Interfaces:
User Interface: No user interface is envisioned, only APIs for the integration to the existing end-user machine and applications.
Synchronous/Asynchronous Interface: RESTful APIs.
Network/Protocols: HTTP/HTTPS.
Data Repository: SQLite.
Requires:
Other AIDEAS Solutions: none
Solution architecture¶
AI-QA-3D solution provides a computer-vision based AI solution for the classification of the incoming objects with 3D capture data. The AI solution will be developed in C and presented as a Python developed service for further integration into existing pilot’s machines and/or existing software for the processing of incoming objects. The service will be deployed in a Dockerized fashion. Data will be mainly stored on the user premises. We don’t envision specific development of user interfaces, as the main purpose of the solution is the integration into existing applications. Nevertheless, we plan to provide demo user interfaces based on Streamlit1 to enable pilot-independent demonstration of the solution.