This thesis studies adaptive token compression for Vision Transformers. It uses class attention as a task-aware importance signal, selects image-specific token budgets, and merges redundant tokens with feature-position clustering. Experiments on DeiT backbones across ImageNet-100 and CIFAR-100 show that moderate compression preserves accuracy best, while real runtime efficiency still depends on the hardware cost of adaptive clustering.
@mastersthesis{ding2026classattention,title={Class-Attention-Guided Adaptive Token Clustering for Efficient Vision Transformers},author={Ding, Kai},school={The University of Sydney},year={2026},month=may,note={Master of Computer Science Research Pathway Thesis},}