Machine learning (ML) and Machine Vision (MV) are closely linked. But "a traditional machine vision system doesn't understand the difference between a round, red balloon on a photo and a red blob on a piece of writing," is how Dr. Erik Marquardt, responsible for optical technologies and additive manufacturing at the VDI, sums it up. Conventional MV needs programmable rules. But when there are errors that cannot be captured in such rules or require a disproportionate amount of programming, this is when AI comes into play.

What is AI?

The term artificial intelligence was the brainchild of John McCarthy of MIT, when he was trying to make an impression with a grant proposal to the Rockefeller Foundation in the run-up to the Dartmouth Conference in the summer of 1956.

In the beginning, AI was seen almost as a panacea that would solve all the problems of MV, robot control, etc. In contrast to humans, AI—equipped with appropriate computing power— can learn an enormous amount very quickly. And especially in pattern recognition, AI is unrivaled.

AI is based on neural networks (NN). Neurons correspond to the nerve cells and glial cells of a human brain, which are connected via synapses. Without going into biological details, neurons alone are not really intelligent. They are intelligent in interaction with other neurons.

The neurons of the AI are computer programs that receive and process information and send it to other neurons, which in turn process the information, and send it on, etc.. Now, with training data, AI learns which forwarding path is the right one in each case, depending on the result. If new data then comes in, the network applies this weighting and delivers the correct results, as long as the training data is unique and representative. The risk in using NN is that afterwards it is no longer possible to understand why a decision was made. One can ask a human why he saw a dog as a wolf, but not a NN.

Now, to process image data with higher accuracy, simple NNs are not enough. Convolutional neural networks (CNNs) consist of complex layers of neurons.

Typical questions for these CNNs are then:

  • Classification (recognition of one class of objects)
  • Multi-object recognition (recognition of numerous similar elements)
  • Multiple classification (distinction between several classes)
  • Anomaly detection (detection of deviation from a reference)
  • Segmentation (division into different areas, e.g. foreground vs. background)

Deep neural networks, schematic representation

AI needs Big Data for learning

For training, AI requires image data. When classifying, an image area is assigned to a predefined class such as 'okay' or 'not okay'. In order to also recognize subtleties, "the system must learn like a small child what edges, circles, colors, textures, etc. look like. In its 'apprentice time', AI then learns what a weld seam looks like, for example," summarizes Marian Schlüter from the Automation Technology/Department of Machine Vision at Fraunhofer IPK.

A variant of NN learning is supervised learning. Here, the system receives the desired solution, and the network changes its parameters until it provides the desired result to a sufficient extent. In reinforcement learning, the system only learns whether the generated solution is better or worse than the previous one. Through "punishments" and "rewards" the system parameters are optimized.

In unsupervised learning, the system does not learn whether its solution is correct. These networks classify information according to similarities between data. In this way, for example, unknown defects on components can be detected.

But our expert Dr. Udo Bürgel warns:

No matter which method and which network you use, you should not underestimate the effort involved. Initial results can be achieved very quickly. However, high accuracies take time.
Dr. Udo Bürgel R&D Manager Technology at VITRONIC

The effort required to collect data for AI often requires weeks of work. However, if a certain error rarely occurs or the corresponding machine is only in development, there is sometimes a lack of meaningful data. This can be remedied by simulations and pre-training of the NN. For example, test images can be generated from CAD data. Image data sets such as ImageNet are available for pre-training the networks. Afterwards, a few application images are sufficient to fine-tune the parameters.

Novelty Detection and Active Learning

Should new types of errors arise, then Novelty Detection (Anomaly Detection) comes into play. In this case, only images of good parts are necessary for training.

A continuation of this is Active Learning. If the original training images are missing in some scenarios and only the newly added ones are used for training, the network can forget. "It can even happen that an NN then does not know anything at all and cannot classify. This is described as ‘catastrophic forgetting’", reports Marian Schlüter. This is countered by active and incremental learning, e.g. by using subsets of the old data sets for new training.

AI in the industry

AI in MV is just on its way to industrial use. Roughness, reflections, etc. of components can still throw AI out of step.

And there is also an enormous amount happening on the hardware side for AI applications. We also have to make sure that the cycle times for MV are maintained in the production line.
Carsten Born IT Expert at VITRONIC

We will revisit the issue of performance in a future post.


There are (still) mostly psychological obstacles to the use of AI. "Companies still have reservations because AI is a black box for them," says Marian Schlüter. That is why a lot of research is currently being conducted in the area of Explainable AI, with the aim of making it more comprehensible why which decisions were made. This is particularly important in healthcare production, where 100% control is required. You can find more information here.

Sustainability also plays a role here. "Not everything that can be done with AI also has to be done with it; it requires immense computing time and energy," resumes Marian Schlüter. So, there is also room for long-established MV algorithms, but AI will improve them.

In many areas of VITRONIC's machine vision solutions, AI therefore already plays an important role. In the next article, we will take a closer look at this using our VIRO WSI weld seam solution.


In brief

  • Large amounts of high-quality data are needed to train AI.
  • AI in MV is just now making its way into industrial use.
  • There remains room for long-proven MV algorithms, but AI will improve them.


Conventional MV needs programmable rules. But when errors cannot be captured in such rules or require a disproportionate amount of programming, AI is needed. AI already plays an important role in many areas of machine vision solutions.

Kaan Fidan

Kaan Fidan

Sales Manager
+49 611 71527381
Kaan Fidan is Sales & Partner Manager at VITRONIC. Since 2018 he is the contact person for Automotive Solutions. Through is study background and his experience in the automotive industry, he understands the high-quality & efficiency requirements in the weld seam inspection well. He consults and assists manufacturing companies to achieve their goals through appropriate automation.



Intelligent quality inspection for maximum safety
How inspection data optimize pharmaceutical production processes
Subscribe to Newsletter
the machine vision people
Whether in automation or traffic engineering: With our systems our customers see more. The combination of hardware, data and intelligent evaluation makes applications and analyses possible that were unthinkable until recently. A previously hidden world becomes visible and usable.