When IC Meets Text: Towards a Rich Annotated Integrated Circuit Text Dataset

Chun Chet Ng^1,*, Che-Tsung Lin^2,*, Zhi Qin Tan¹, Xinyu Wang³, Jie Long Kew¹, Chee Seng Chan¹, Christopher Zach²

¹Universiti Malaya, Kuala Lumpur, Malaysia
²Chalmers University of Technology, Gothenburg, Sweden
³The University of Adelaide, Adelaide, Australia
^*Equal Contribution

Paper GitHub Repo ICText Dataset

Abstract

Automated Optical Inspection (AOI) is a process that uses cameras to autonomously scan printed circuit boards for quality control. Text is often printed on chip components, and it is crucial that this text is correctly recognized during AOI, as it contains valuable information. In this paper, we introduce ICText, the largest dataset for text detection and recognition on integrated circuits. Uniquely, it includes labels for character quality attributes such as low contrast, blurry, and broken. While loss-reweighting and Curriculum Learning (CL) have been proposed to improve object detector performance by balancing positive and negative samples and gradually training the model from easy to hard samples, these methods have had limited success with one-stage object detectors commonly used in industry. To address this, we propose Attribute-Guided Curriculum Learning (AGCL), which leverages the labeled character quality attributes in ICText. Our extensive experiments demonstrate that AGCL can be applied to different detectors in a plug-andplay fashion to achieve higher Average Precision (AP), significantly outperforming existing methods on ICText without any additional computational overhead during inference. Furthermore, we show that AGCL is also effective on the generic object detection dataset Pascal VOC.

ICText Dataset

Samples of annotations in ICText dataset. Besides the common annotations (i.e. bounding box and class), we also include the multi-label character quality attributes. They are represented by (i) red dot for low contrast character, (ii) green dot dot for blurry characters, and lastly (iii) blue dot for broken characters.

The ICText dataset contains a total of 10,000 images with 100,152 legible characters. Of the 69,750 annotated samples in the training set, 50,597 have quality defects (72.5%). In the testing set, 20,114 out of 30,402 characters (66.2%) have quality attributes. This indicates a high number of flawed characters in ICText, which is also the first-ever dataset with quality labels on characters. All images and annotations are fairly distributed with a 7:3 ratio between the training set (69.64% of complete annotations) and the testing set (30.36%). Statistics of ICText are visualized in the following figures:

Figure 1:

Number of images and annotated characters (defective and perfect) in the training and testing splits of ICText.

Figure 2:

Histogram of character classes in ICText.

Figure 3:

Stacked bar chart of quality attributes in ICText.

The ICText dataset is also extended to a long-tail classification dataset, ICText-LT. A new Frequency Weighted Focusing and Dynamic Probability Smoothing (FFDS) loss is then proposed to dynamically reduces the influence of outliers and assigns class-dependent focusing parameters. More details can be found in this GitHub repo.

AGCL

Attribute-Guided Curriculum Learning (AGCL) loss proposes to zero out the gradient of difficult characters. It can also balances the contribution of negative samples through weighting factors and focusing parameters. The training is split into two phases where AGCL loss is used in the first phase, and the Cross Entropy loss is used in the second phase.

The figure below shows the differences between common existing loss functions used by object detectors and our proposed AGCL on positive and negative cases during training.

Comparison of existing loss functions and AGCL.

Experiment Results

Quantitative results of all methods on ICText's test set are shown in the table below. Models marked with * are tested on a subset of easier images. Both inference speed and GPU memory are tested on Titan X, and the rest of the hardware specifications can be found in our paper. ± marks the standard deviation calculated over five runs, and ↑ shows the relative AP improvement of AGCL-enabled methods over the baseline methods.

Qualitative results of ABCNet, PAN++, YOLOv4 (baseline), and our proposed YOLOv4-AGCL are shown in the figures below. Green boxes = true positives; Red boxes = false positives; Blue boxes = false negatives. The character class prediction is shown in the top left corner of each box.

We show that asking non-AGCL detectors to learn directly from flawed characters has side effects, i.e., more false positives and false negatives. Additionally, both ABCNet and PAN++ suffer from granularity issues. In contrast, there are significantly fewer false-positive and false-negative boxes in our proposed method (i.e., YOLOv4-AGCL), showing that training a detector in an easy-to-hard fashion guided by quality attributes can achieve better results.

BibTeX

If you wish to cite the ICDAR 2021 Competition on Integrated Circuit Text Spotting and Aesthetic Assessment paper:

@inproceedings{icdar2021_ictext,
author= {Ng, Chun Chet and Nazaruddin, Akmalul Khairi Bin and Lee, Yeong Khang and Wang, Xinyu and Liu, Yuliang and Chan, Chee Seng and Jin, Lianwen and Sun, Yipeng and Fan, Lixin},
title= {ICDAR 2021 Competition on Integrated Circuit Text Spotting and Aesthetic Assessment},
booktitle= {Document Analysis and Recognition -- ICDAR 2021},
year= {2021},
publisher= {Springer International Publishing},
pages= {663--677}}

If you wish to cite the latest version of the ICText dataset and AGCL, published at Pattern Recognition, Elsevier:

@inproceedings{ictext_agcl,
author= {Ng, Chun Chet and Lin, Che-Tsung and Tan, Zhi Qin and Wang, Xinyu and Kew, Jie Long and Chan, Chee Seng and Zach, Christopher},
title= {When IC meets text: Towards a rich annotated integrated circuit text dataset},
journal= {Pattern Recognition},
year= {2024},
volume= {147},
pages= {110124},
issn= {0031-3203},
publisher= {Elsevier},
doi= {https://doi.org/10.1016/j.patcog.2023.110124}}