Machine Vision

Introduction

Many electrical devices, such as PCBs(Printed Circuit Boards), LCDs, transistors, etc., have been smaller and smaller so it has been difficult to find defects on the device by human beings. Finding defects is very tedious and hard for human beings.Recent camera systems produce high resolution and high contrast images. Also a development of accurate and fast computer vison algorithms replaces humans to machine in a inspection process.This research is for developing fast and accurate inspection algorithms using computer vision technology. Especially , we are focusing on parallel processing and developing adequate vision alogirhm with recently powerful computing machine such as multicore-CPUs, GPUs and APUs.

Machine vision researches performed in our laboratory can be divided into 2 categories :

Example for Automated Inspection in a product line


Defect detection and image alignment

Vision based inspection algorithms are depended on the devices since the defects appears various and different according to them. Mainly our research target for defect detection is PCB. In below, the figure shows the defect items can be found on PCB.

Many algorithms can be applied to detect defects. Generally, simple pattern matching with golden template image and morphological image processing to compare original one with defect specification have been used.
The inspection accuracy is depended on not only image processing but also image alignment. Although a inspection system has robust defect detection algorithms, Wrong image alignment increases fault detection rate over all system.
To robustly align images, usually some fiducial marks that only used for image alignment was also printed when PCB was made.

A example of Fiducial mark

So our research is focused on image alignment and appropriate defect detection methods.

Robust and Accurate pattern matching in Fuzzy space for Fiducial mark alignment

We made a new pattern matching method for fiducial mark alignment in a fuzzy space. The membership functions of fuzzy sets are designed by distance transforms, and their levels are set in the fuzzy space for fast matching of a specific fiducial mark. After the fuzzification, a sub-pixel level translation is estimated by a fuzzy similarity measure and an interpolation using fuzzified model and target images. We also propose a method of coarse-to-fine rotation estimation in sub-pixel level. Experiments show that the proposed fuzzy space pattern matching algorithm outperforms commercial pattern matching algorithms based on correlation or edge.

Development of PCB image alignment algorithm Based on SIFT descriptor

We proposes an image alignment algorithm for application of AOI (Automatic optical inspection) based on SIFT. Since the correspondences result using SIFT descriptor have many wrong points for aligning, this paper modified and classified those points by five measures. Experimental results show that the proposed method has similar rotation and robust translation accuracy in comparison to the commercial software MIL 8.0.

Discolored Metal Pad Image Classification Based on Gabor Texture Features Using GPU

We propose a Gabor texture feature extraction method for classification of discolored Metal pad images using GPU(Graphics Processing Unit). The proposed algorithm extracts the texture information using Gabor filters and constructs a pattern map using the extracted information. Finally, the golden pad images are classified by utilizing the feature vectors which are extracted from the constructed pattern map. In order to evaluate the performance of the Gabor texture feature extraction algorithm based on GPU, a sequential processing and parallel processing using OpenMP in CPU of this algorithm were adopted. Also, the proposed algorithm was implemented by using Global memory and Shared memory in GPU. The experimental results were demonstrated that the method using Shared memory in GPU provides the best performance. For evaluating the effectiveness of extracted Gabor texture features, an experimental validation has been conducted on a database of 20 Metal pad images and the experiment has shown no mis-classification.

Our published research papers related to this are listed to below.

Related papers :

    • Xuenan Cui, Hakil Kim, Eunsoo Park, Hyohoon Choi, “Robust and accurate pattern matching in fuzzy space for fiducial mark alignment,” Machine Vision and Application, May 2012.
    • 김준철, 최학남, 박은수, 최효훈, 김학일, “잡음과 회전에 강인한 SIFT 기반 PCB 영상 정렬 알고리즘 개발,” 제어·자동화·시스템공학회 논문지, 16권 7호, pp.695~702, 2010.
    • 최학남, 박은수, 김준철, 김학일, “GPU를 이용한 Gabor Texture 특징점 기반의 금속 패드 변색 분류 알고리즘,” 제어·로봇·시스템학회 논문지, 제 15권 제 8호, pp.778~785, 2009.
    • 김현주, 윤두현, 김학일, “라인스캔 카메라를 이용한 3차원 정밀 측정,” 한국광학회지, 19권 5호, pp. pp.376~380, 2008.
    • 윤두현, 김학일, “오목거울을 이용한 3차원 형상측정을 위한 모아레 영상 획득 방법,” 한국광학회지, 19권 4호, pp.327~333, 2008.
    • 김준철, 홍성욱, 최학남, 김학일, “SIFT를 기반으로 하는 PCB 영상의 정렬 알고리즘 개발,” 대한전자공학회 하계종합학술대회, 제31권 제1호, pp.1346~1347, 2009.

Back to top


Parallel Processing

Recently, since a very high resolution image can be acquired in a inspection system, fast processing time is needed. To increase processing time, our research is focused on parallel image processing. At first, we develop many image processing algorithm operated in Single Instruction Multiple Data(SIMD) scheme. It means that a one command processes multiple data using special 16byte or 32byte register. Below figure demonstrates SIMD scheme. Generally to process 8 data, we need a 8 cycle for-loop. But if we use special SIMD register and SIMD command, the for-loop is not necessary and only one operation can compute all data.

Since this data level parallelism can be more parallelized in multi-core processor, we can make two stage parallel processing on inspection system based on CPU. Below figure shows the example of data processing on dual-core CPU. To develop multi-core processing algorithm, we use OpenMP. It easily makes parallelism to insert OpenMP phrase on the existing code.

In addition to CPU level parallelism, General purpose GPU(GPGPU) can be used to reduce processing time. Like below figure, GPUs have many cores than CPUs and Multi-GPU archtecture can be easily made. Using GPUs many core architecture and CUDA, the GPU programming language developed by NVIDIA, we made parallel processing algorithms operated in GPUs.

Our research to make parallel inspection system is demonstrated below figure. We has replaced many sequential image processing code to parallel ones coded by SSE, AVX, OpenMP and CUDA for the proposed inspection system.

Parallel Processing Architecture for inspection system

A Fast Feature Extraction in Object Recognition Using Parallel processing on CPU and GPU

Due to the advents of multi-core CPU and GPU, various parallel processing techniques have been widely applied to many application fields including computer vision. This project presents a parallel processing technique for realtime feature extraction in object recognition by autonomous mobile robots, which utilizes both CPU and GPU by combining OpenMP, SSE (Streaming SIMD Extension) and CUDA programming. Firstly, the algorithms and codes for feature extraction are optimized and implemented in parallel processing. After the parallel algorithms are assured to maintain the same level of performance, the process for extracting key points and obtaining dominant orientation with respect to the key points is parallelized. Following the extraction is the construction of a parallel descriptor via SSE instructions. Finally, the GPU version of SIFT is also implemented using CUDA. The experiments have shown that the CPU version of SIFT is almost five times faster than the original SIFT while maintaining robust performance. Further, the GPU-Parallel descriptor achieves acceleration up to five times higher than the CPU-Parallel descriptor at a cost of a bit lower performance.

Comparative study on Performance of object detection and Method to Speed up

we survey various Robust Object Recognition Algorithms. One of the core technologies for local feature detector is Scale Invariant Feature Transform. And we compared several algorithms with SIFT based on IPP technology. As a result, the conversion of source codes using IPP is sped up. And this will be more improved recognition speed using SIMD Instructions.

Benchmark for performance comparison

This is the performance evaluation about parallel processing plat-form and commercial library. we made many parallel vision algorithm for the comparisons. Below tables show the result.

Our published research papers related to this are listed to below.

Related papers:

    • Junchul Kim, Eunsoo Park, Xuenan Cui, Hakil Kim and William A, Gruever, “A Fast Feature Extraction in Object Recognition Using Parallel processing on CPU and GPU,” International Conference on Systems, Man and Cybernetics, 2009.
    • 김준철, 정용한, 박은수, 최학남, 김학일, “실시간 물체인식을 위한 특징 추출 알고리즘의 병렬화,” Image Processing and Image Understanding, pp.772~777, 2009.
    • 김준철, 정용한, 박은수, 최학남, 김학일, 허욱렬, “CPU와 GPU의 병렬처리를 이용한 고속 물체인식 알고리즘 구현,” 제어·로봇·시스템학회 논문지, 제 15권 제 5호, pp.487~494, 2009.
    • 박은수, 최학남, 김준철, 임유청, 김학일, “SSE 명령어를 이용한 영상의 고속 전처리 알고리즘,” 대한전자공학회 논문지, 제 49권 제 2호, pp.65~77, 2009.
    • 최학남, 박은수, 김준철, 정용한, 임유청, 최정주, 정기봉, 이종문, 김학일, “GPU를 이용한 고속 Texture 특징점 추출 알고리즘 개발,” Image Processing and Image Understanding, pp.826~830, 2009.
    • 최학남, 박은수, 김준철, 정용한, 김학일, “GPU를 이용한 고속 영상 보간법 개발,” 정보 및 제어 학술대회 논문집, pp.300~301, 2008.
    • 김준철, 정용한, 박은수, 최학남, 김학일, “병렬처리를 이용한 특징점 추출 알고리즘의 고속화,” 제어자동화 시스템 심포지엄, pp.613-618, 2008.
    • 김준철, 김학일, “물체인식의 성능 및 속대개선 방향에대한 비교 연구,” 대한전자공학회 하계종합학술대회, 제31권 제1호, 2008.
    • 박은수, 최학남, 김준철, 김학일, “SSE 명령어를 이용한 고속 Sobel Edge Detector 구현,” 정보 및 제어학술대회 논문집, pp. 113-114, 2007.
    • 최학남, 박은수, 김준철, 김학일, “멀티프로세서 기반의 고속 영상처리 기술에 대한 벤치마킹,” 정보 및 제어학술대회 논문집, pp. 111-112, 2007.
    • 김준철, 박은수, 최학남, 김학일, “SIMD 명령어를 이용한 영상의 평균화 필터 최적화,” 정보 및 제어학술대회 논문집, pp. 431-432, 2007.
    • 최학남, 박은수, 김준철, 김학일, “SSE 명령어를 이용한 고속 모폴로지 알고리즘 개발,” Image Processing and Image Understanding, pp.111-112, 2007.

Back to top