An application case of machine vision inspection s

Dry goods: an application case of machine vision inspection system

in order to produce seamless steel pipes, the billet needs to be transported to the furnace for heating first. Next, the blank is perforated to form a thick walled hollow shell, and then the mandrel is inserted into the shell. Then the elongation rolling is carried out in the mandrel seamless tube mill. After the elongation process, the blank is conveyed to the propulsion table, where it is pushed through a series of roll stands. Finally, a hollow long steel pipe with continuous smaller wall thickness is formed

although as effective as the hot rolling process, the roll stand in the mill stand can occasionally produce marks and defects on the steel surface, which are very difficult to detect under hot conditions. Therefore, in quality improvement projects, many manufacturers hope to identify these defects as early as possible to avoid producing a large number of defective materials at a considerable cost

vision system

in order to solve these problems, engineers of tecnalia company in Spain developed a machine vision system called surfin ', which enables steel manufacturers to detect such defects when steel plates are pushed out of the propulsion table (see Figure 1). The detection of such defects provides the manufacturer with instructions for any problems in the production process, so that the manufacturer can carry out preventive maintenance on the propulsion table at an early stage to prevent any defective steel pipes from being delivered to the customer

typical defects on the surface of this kind of steel pipe caused by the roll stand usually follow the repeated pattern and continue to appear until the mill support changes. These defects include cracks on the surface, mill support blocking marks, cracks, and separated steel, which will then stick to another part of the steel pipe surface

developers face severe challenges. The conditions in this production environment are extremely bad: not only steel pipes are produced at a relatively high speed of 6-7m/s (surfin'can work at up to 10m/s), but also the temperature of steel when it is pushed out of the roll base is about 1000 ℃. In addition, the environment is dirty and there is water and oil vapor, which makes defect detection more difficult

since the light radiated by the hot surface of steel is directly related to the thermal emission in IR, red, orange and yellow bands, capturing the image of all the light reflected by the steel surface will saturate the sensor in the camera, because the camera is sensitive to all the light radiated by the steel pipe. In order to solve this problem, the wavelength of light used by surfin'system (patent numbers es and EP) is far away from the wavelength of the emission spectrum of hot steel

then, the image captured by the camera in the system passes through a narrow-band optical bandpass filter (with a central wavelength of 470nm and a bandwidth of 10nm) and an infrared (IR) radiation filter of Edmund optics. These two filters enable the CCD camera to receive only the radiation in the required band, and IR filters are added to protect the electronic system from thermal radiation. Controlled lighting technology allows the system to capture an image of the entire surface of the steel pipe as if it were cold

in order to enable the system to capture 360 ° images of the steel pipe surface, the system uses three sets of 14000 lines/s Spyder 3-wire scanning cameras of Teledyne DALSA company in Canada, which are perpendicular to the axial plane of the rolled steel pipe at 120 ° angle intervals, installed in the protective shell, and surrounded the output end of the propulsion table. In the previous version of the system, two 200MW 473nm blue laser sources from laserglow technologies of Canada were used on both sides of each camera to illuminate the surface of the steel pipe in a dark field. Due to the geometry of the system, it can continuously capture a complete image of the steel pipe in real time (see figures 2a and b)

due to the high temperature of the environment, it is very important to keep the camera cool continuously. Therefore, the compressed cooling air is injected into the protective housing to protect the camera and laser equipment from the influence of heat and harsh environment. The air not only cools the system, but also after that, the excess air is discharged through the window, through which the laser outputs light beams, and the camera captures images to prevent the deposition of scales, oxides, dust and liquids

image processing

once the image of the steel surface is captured, the image is then transmitted 100m through the optical Gigabit Ethernet link to the PC based server in the control room. Here, firstly, the image is preprocessed to enhance the contrast of the image by using customized image enhancement algorithms such as histogram equalization. Since the available data in the original image is represented by the near contrast value, this technology increases the global contrast of the image

after image enhancement, customized software is used for processing. In the previous version of the system, the software adopts the auxiliary learning system based on support vector machine (SVM). Once the system is taught to recognize defects from different samples through texture, contrast and size, the algorithm can automatically detect and classify the most important production defects in the production environment

the PC based server is used to store the images from the camera, the defect data found, and the location of the defect on the steel pipe. It will also store the alarms of pressure, temperature, speed signals, communication and other steel pipe production data in the Oracle database for quality control and traceability. You can also remotely check the data on the server by installing client applications on computers connected to the corporate local area network (LAN)

since the initial development of the system, it has undergone several enhancements. The structure of the system has now been redesigned to make it easier to align and adjust the camera and adjust the lighting

the newer version of the system also uses liquid rather than air cooling technology, so that the lighting and sensors can be placed closer to the steel pipe, so as to achieve a hotter or larger area of steel pipe imaging. The LED light source of metaphase technologies has also replaced the early lasers, increasing the service life of the light source from 2000 hours to 50000 hours, and eliminating factors such as speckle that may damage the image captured by the camera

the software user interface has also been improved. Now factory operators can see the location and specific properties of defects on steel when they appear (see Figure 4). Now it is also possible to store several months of production data in the database, so that the plant manager can view the periodicity of any errors that may occur and schedule regular preventive maintenance operations. The system can also support multiple users, who can not only access the system locally, but also access the system through interconnection

classification change

surfin'the most important latest development of the system is to replace the previous SVM based classifier with an internally developed candidate window detection platform and convolutional neural network (CNN) for defect classification. CNN can learn to extract the relevant features representing each type of defect from the training image and perform classification, while SVM only maps its input to some high-dimensional spaces that can reveal the differences between defect categories

by assuming that all objects of interest (such as defects) share a common visual attribute that distinguishes them from the background, the candidate window detection platform outputs a set of areas that may contain defects. Subsequently, CNN extracts learning features in China's plastic machinery market and performs actual defect classification on image data

cnn classifier is verified by the customized image database of hot steel pipe images with defects, and it is found that the method based on deep learning can reduce the number of false positives and false negatives detected, which is significantly better than the previous SVM classifier

what are the use skills and protection and maintenance of bellows ring stiffness testing machine? When performing two types of classification (such as defect and no defect), the most relevant performance index is the area under the AUC, or ROC (receiver operating characteristics) curve. By plotting the false positive rate on the X axis and the true positive rate on the Y axis, then calculate the area under this function (see Figure 5)

ideally, the value of this function is 1.00 for each value on the x-axis, so the better the model, the closer its AUC is to 1. In this way, when comparing several models, the best model can be selected by taking the highest AUC value. Although the AUC value of SVM classifier is 0.88, the AUC value of CNN surfin 'classifier is 0.997 for two types of classification

in addition, for a given model, a threshold can be selected to enable the system to determine whether the sample is defective. Since the output of the model is usually a probability value between 0 and 1, if the probability value is greater than the threshold, the sample will be marked as NOK, otherwise it will be marked as OK

by moving the threshold to 1.0, the number of false positives can be reduced at the cost of "increasing the number of false negatives", and vice versa. Then you can visualize where the inspection system works by plotting the threshold on the x-axis and the specificity or true negative rate (= 1-false positive rate) and sensitivity or true positive rate (= 1-false negative rate) on the y-axis

the vertical line corresponding to the threshold cuts the points of the two curves, resulting in false positive rate and false negative rate. The common choice of threshold is to produce approximately equal false positive and false negative rates. For CNN surfin', a false positive rate of 1.58% and a false negative rate of 1.49% were obtained (see Figure 6). In contrast, the false positive rate of SVM version of surfin'is 17.98%, and the false negative rate is 18.00%. The number of classification errors made by surfin' is reduced by 12 times

the current solution is based on different fibers and different substrates

now, the new classifier is ready to run in the production environment. In addition, tecnalia engineers are working hard to continue to improve the system in order to enable steel producers to produce zero defect steel. For example, CNN surfin 'has been used to evaluate four types of problems (OK and three types of defects), which are classified by AUC (average extension), and AUC = 0.9956. However, more samples are being collected to make the results more statistically significant

tecnalia has established a relationship with the Spanish sarralle group to distribute the surfin'system worldwide