Extensive experiments on three benchmark datasetsĭemonstrate that our proposed framework outperforms the state-of-the-art onlineĭistillation methods by a large margin, and shows its efficacy in learningĬollaboratively between ViT-based and CNN-based models. Pixel-wise BSD discerning which of the prediction knowledge to be transferred de Barros Vidal This Letter proposes a fusion strategy for multistream convolutionalnetworks, the lattice cross fusion. Transferred between the corresponding regions in the feature space and 2) L-CNN: a lattice cross-fusion strategy for multistream convolutional neural networks A.P.G.S. This isĪchieved by 1) region-wise BSD determining the directions of knowledge Reliable knowledge from each other, we propose bidirectional selectiveĭistillation (BSD) that can dynamically transfer selective knowledge. Secondly, to facilitate the two students to learn We propose heterogeneous feature distillation (HFD) to improve students'Ĭonsistency in low-layer feature space by mimicking heterogeneous featuresīetween CNNs and ViT. Semantic segmentation?" Accordingly, we propose an online knowledgeĭistillation (KD) framework that can simultaneously learn compact yet effectiveĬNN-based and ViT-based models with two key technical breakthroughs to takeįull advantage of CNNs and ViT while compensating their limitations. Models by selecting and exchanging the reliable knowledge between them for ![]() Download a PDF of the paper titled A Good Student is Cooperative and Reliable: CNN-Transformer Collaborative Learning for Semantic Segmentation, by Jinjing Zhu and 3 other authors Download PDF Abstract: In this paper, we strive to answer the question "how to collaboratively learnĬonvolutional neural network (CNN)-based and vision transformer (ViT)-based
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |