SinoTechIntel Academic Portal
Official PDF TranslationFrontiers of Information Technology & Electronic Engineering

An adaptive outlier correction quantization method for vision Transformers

Authors: Zheyang LI; Chaoxiang LAN; Kai ZHANG; Wenming TAN; Ye REN; Jun XIAO

DOI: 10.1631/FITEE_2400994Status: Verified Translated Edition
Sponsored AdvertisementAd Placement Area
reCAPTCHA Bot Shield Active

Preparing Secure Academic Download

Verifying human reader & generating high-resolution document...

Verifying Document Integrity15s remaining
← Back to Article
Protected by Google reCAPTCHA v3.PrivacyTerms
Sponsored ContentAdSense In-Feed Ad Slot

Key Findings in This Report

• Introduces AOCQ, a three-level quantization method (operator, framework, loss) that adaptively corrects channel and token outliers in vision Transformers, reducing quantization error. • Achieves 81.57% accuracy on DeiT-Base with 8-bit post-training quantization, only a 0.28 percentage point drop, and 4× faster runtime. • Enables ultra-low 4-bit weight quantization for Swin and DeiT across classification and object detection tasks, with a minimal accuracy loss of about 2% and nearly 8× less memory. • Demonstrates that AOCQ effectively mitigates the uneven activation distributions that limit standard PTQ methods, supporting efficient edge deployment.