Project Background

This project addressed a challenge in the 2023 National College Student Integrated Circuit Innovation and Entrepreneurship Competition (AWCloud Cup). The task focused on automated defect inspection for industrial aluminum-can and aluminum-sheet production lines. The organizers provided a dedicated dataset, the AWCloud platform based on an Intel Cyclone V SoC, and a reference AI inference accelerator with its associated toolchain.

The dataset comprises four common defect types found in aluminum products:

• zhen_kong (Pinhole)
• ca_shang (Scratch)
• zang_wu (Dirt/Stain)
• zhe_zhou (Crease/Wrinkle)

Figure 1. Dataset samples

The AWCloud platform provides the graphical management interface shown in Figure 2. Its accelerator toolchain (Figure 3) supports models from TensorFlow, PaddlePaddle, and Caffe; during the Analysis stage, optimized and compiled operators are mapped to the AWCloud CNN accelerator for execution.

Figure 2. AWCloud visual management platform

Figure 3. CNN accelerator demo toolchain

Core Tasks of the Competition:

• Train, quantize, and compress the defect detection network model.
• Deploy the core algorithm on the AWCloud platform.
• Develop ARM-side drivers to establish an end-to-end video pipeline (“Camera -> AI Accelerator -> Display”).
• (Optional/Advanced) Modify and optimize the hardware architecture of the AI inference accelerator demo.

  Scoring Criteria: The final submission was comprehensively evaluated based primarily on algorithmic accuracy (mAP) and end-to-end inference frame rate (FPS).

Technical Approach Overview

• Developed a lightweight MobileNet-SSD-lite model for aluminum-surface defect detection.
• Combined FPGM pruning with quantization-aware training (QAT) to further reduce the model footprint.
• Deployed the model on the AWCloud FPGA platform and rewrote the Cyclone V SoC ARM-side driver to implement a complete video-processing pipeline.

Personal Contributions

• Removed 13% of the model parameters through FPGM pruning and quantized the weights from FP32 to INT8 using QAT while retaining high detection accuracy.

• Improved the end-to-end inference throughput on the cloud FPGA platform by 2.3× through algorithmic and software optimizations.

Detailed Technical Implementation

💡 Tip: Click on the respective modules to expand and read.

Joint 'Pruning + Quantization' Compression and Deployment

The project used the single-stage SSD detector as its baseline. To reduce computational cost, the conventional VGG backbone was replaced with MobileNet and the detection heads were implemented using depthwise separable convolutions, yielding the MobileNet-SSD-lite architecture shown in Figure 4.

Figure 4. SSD Detector Architecture

The FP32 model, trained in PaddlePaddle, achieved 98.45% mAP. PaddleSlim was then used to perform joint FPGM pruning and quantization-aware training (QAT). After removing 13% of the parameters and quantizing the weights from FP32 to INT8, the model retained 97.05% mAP. The exported model was compiled with the vendor toolchain and deployed successfully on AWCloud. Figure 5 summarizes the complete training, compression, compilation, and deployment workflow.

Figure 5. Model training, compression, and deployment pipeline

Deployment and Performance

Joint compression improved inference performance while preserving high detection accuracy. The measured results are summarized below:

Figure 6 shows the deployed system in operation. Given 300×300 RGB inputs, it consistently localizes aluminum-surface defects with accurate bounding boxes, demonstrating the practical balance achieved between detection accuracy and edge-side inference performance.

Reflections and Summary

Personal Evaluation: From Theory to Practice

This project translated the joint compression principles studied in the CAS Innovation Training Program into a deployed system. Because depthwise separable convolution is already highly parameter-efficient, the model proved particularly sensitive to pruning: removing more than 13% of its parameters caused a pronounced loss of accuracy. The reference accelerator also supported only coarse-grained per-tensor quantization, which is poorly matched to the heterogeneous channel-wise distributions of depthwise convolution. Quantization-aware training and subsequent fine-tuning were therefore used to recover most of the resulting accuracy loss.

The competition schedule did not permit a substantial redesign of the reference hardware accelerator, so hardware-efficiency bottlenecks continued to limit the final frame rate. This limitation nevertheless motivated my subsequent research into accelerator microarchitecture and system-level optimization.

AI Systems: The Constraints of Amdahl’s Law

The practical inference efficiency of an AI system is determined not only by a model’s operation count and parameter count, but also by how effectively the underlying hardware executes the complete workload. At the system level, Amdahl’s law bounds the achievable speedup by the portion of execution that remains serial or inefficient. Meaningful optimization must therefore target the dominant end-to-end bottleneck rather than the compute kernel in isolation.

The accelerator demo provided a direct example: HWC-to-CHW tensor rearrangement was implemented as serial loops on the ARM core, leaving the compute array idle and reducing end-to-end utilization. Two complementary remedies are possible:

• Software parallelization: Vectorize the rearrangement kernel with ARM NEON SIMD instructions.
• Dedicated rearrangement hardware: Implement a pipelined data-layout conversion module in the FPGA fabric to offload the ARM processor.

Data movement limiting computation is not unique to this platform; it is a central challenge in contemporary AI accelerators. The rapid growth of model parameters and context lengths in the LLM era has made this constraint even more prominent. Consequently, accelerator research has shifted from maximizing peak arithmetic throughput alone toward reducing external-memory traffic and improving on-chip data reuse. Preventing compute arrays from stalling because of data starvation is now a primary objective of AI-system design.