Industrial Automation: ‘Electronic Eyes’ Support Manufacturing Workflows
A German automotive manufacturer is deploying ‘electronic eyes’ and image processing technology from VMT Vision Machine Technic Bildverarbeitungssysteme and IDS Imaging Development Systems to automate the loading and unloading of pressed metal components.
Robots perform monotonous workflows and repetitive tasks brilliantly. When paired with image processing technologies, they can become “seeing” members of production teams and support humans reliably. They are used in quality assurance to check components, help with the assembly and positioning of components, detect errors and deviations in production processes, and thereby to increase the efficiency of entire production lines.
VMT Vision Machine Technic Bildverarbeitungssysteme GmbH from Mannheim developed robotics-based 3D measurement solution FrameSense to automate loading and unloading of containers, allowing pressed metal parts to be inserted into or removed safely with precision. Four Ensenso 3D cameras from IDS Imaging Development Systems GmbH provide the basic data and platform for the process automation.
Application
The workflow FrameSense automates is one of many manufacturing operations. A component comes out of a machine – in this instance, a press – and runs on a conveyor belt to be stacked within a container. When full, each container is subsequently transported to the next stage of production – for example, assembly or installation in a vehicle.
Previously, employees undertook all container loading which required calculation of the volume of free space and whether it would be sufficient to fit any given component. The process was complicated further by ‘non-conforming’ factors, including interlocks which had to be removed and any defects which might be identified by a final check of each fully-laden box.
The project required all of these tasks to be performed by a robot with a vision system – presenting a technological challenge as containers are produced by different manufacturers, can be of different types and vary in their dimensions.
Type, shape and position inspection with the aid of four 3D cameras
For their fully automatic loading and unloading, the position of several relevant features of the containers must be determined for a so-called multi-vector correction of the robot. The basis is a type, shape and position check of the respective container. This is the only way to ensure process-reliable and collision-free path guidance of the loading robot. All this has to be integrated into the existing production process. Time delays must be eliminated and the positioning of the components must be accurate to the millimetre.
To counter this, VMT uses four 3D cameras per system. The four sensors each record a part of the entire image field. This can consist of two containers, each measuring approximately 1.5 x 2 x 1.5 metres (D x W x H). Two of the cameras focus on one container. This results in data from two perspectives each for a higher information quality of the 3D point cloud. These point clouds of all four sensors are combined for the subsequent evaluation. In the process, registrations of relevant features of the container take place in ROIs (Regions of Interest) of the total point cloud. A registration is the exact positioning of a feature using a model in all 6 degrees of freedom. In other ROIs, interference contours are searched for which could lead to collisions during loading. Finally, the overall picture is compared with a stored reference model. In this way, the containers can be simultaneously checked for their condition and position in a fully automated manner. Even deformed or slanted containers can be processed. All this information is also recorded for use in a quality management system where the condition of all containers can be traced. The calibration as well as the consolidation of the measurement data and their subsequent evaluation are carried out in a separate IPC (industrial computer) with screen visualisation, operating elements and connection to the respective robot control.
The main result of the image processing solution is the multi-vector correction. In this way, the robot is adjusted to be able to insert the component at the next possible, suitable deposit position. Secondary results are error messages due to interfering edges or objects in the container that would prevent filling. Damaged containers that are in a generally poor condition can be detected and sorted out with the help of the data. The entire image processing takes place in the image processing software MSS (Multi Sensor Systems) developed by VMT. FrameSense is designed to be easy to use and can also be converted to
High projector performance and resolution together with fast data processing were our main technical criteria when selecting the camera.other components directly on site.
Robust 3D camera system
On the camera side, VMT relies on Ensenso 3D cameras – initially on the X36 model. The current expansion stage of FrameSense is equipped with the Ensenso C variant. The reasons for the change are mainly the better projector performance – thanks to a new projection process – as well as a higher recording speed. In addition, the Ensenso C enables a larger measuring volume. This is an important criterion for FrameSense, because the robot can only reach the containers to be filled up to a certain distance. The specifications of the Ensenso C thus correspond exactly to VMT’s requirements, as project manager and technology manager Andreas Redekop explains: “High projector performance and resolution together with fast data processing were our main technical criteria when selecting the camera. The installation in a fixed housing was also an advantage.”
The Ensenso C addresses current challenges in the automation and robotics industry. Compared to other Ensenso models, it provides both 3D and RGB colour information. Customers thus benefit from even more meaningful image data. The housing of the robust 3D camera system meets the requirements of protection class IP65/67. It offers a resolution of 5 MP and is available with baselines from current to approx. 455 mm. This means that even large objects can be reliably detected. The camera is quick and easy to use and addresses primarily large-volume applications, e.g. in medical technology, logistics or factory automation.
Outcome
By automatically loading and unloading containers and the integrated 3D container inspection, manual workstations can be automated with the help of FrameSense. Against the background of the shortage of skilled workers, the system can thus make an important contribution to process automation in the automotive industry, among others. It meets the prevailing challenges of the industry. Ensenso C provides the crucial basis for data generation and exceeds the requirements of many applications. Lukas Neumann from Product Management sees their added value especially here: “The high projector power and large sensor resolutions are particularly advantageous in the field of intralogistics. Here, high-precision components have to be gripped from a great distance with a large measuring volume.” For other stacking or bin-picking applications in classic logistics, he could imagine a similar camera with high projector power but lower resolution and fast recording.
So nothing stands in the way of further developments and automation solutions in conjunction with “seeing” robots.
Camera
Ensenso C: Coloured 3D Vision
Model used: Ensenso C57-8-M
