Swish‐Gumbel: A New Activation Function for Image Classification
CONCURRENCY COMPUTATION PRACTICE AND EXPERIENCE, cilt.38, sa.14, ss.1-19, 2026 (SCI-Expanded, Scopus)
- Yayın Türü: Makale / Tam Makale
- Cilt numarası: 38 Sayı: 14
- Basım Tarihi: 2026
- Doi Numarası: 10.1002/cpe.70822
- Dergi Adı: CONCURRENCY COMPUTATION PRACTICE AND EXPERIENCE
- Derginin Tarandığı İndeksler: Applied Science & Technology Source, Scopus, Technology Collection (ProQuest), Aerospace Database, Science Citation Index Expanded (SCI-EXPANDED), Compendex, INSPEC, zbMATH
- Sayfa Sayıları: ss.1-19
- Açık Arşiv Koleksiyonu: AVESİS Açık Erişim Koleksiyonu
- Ankara Üniversitesi Adresli: Evet
Özet
In this study, we present Swish-Gumbel, a new family of activation functions generated by combining the Swish and Gumbel functions through both multiplication and linear composition. We evaluate their performance on several image classification benchmarks, including MNIST, CIFAR-10, Imagenette, and Beans, using different deep learning architectures. The results show that the multiplicative version consistently outperforms well-known activation functions such as ReLU, Swish, Mish, and Gumbel, while the linear combination variants provide competitive results particularly on simpler datasets, with performance gains becoming more pronounced for the multiplicative form on complex benchmarks. The strength of the proposed approach lies in its ability to maintain more stable gradients in the positive domain, which improves feature extraction and generalization. Overall, our findings suggest that Swish-Gumbel can serve as an effective alternative to existing activation functions for image classification, with potential to be applied in other machine learning tasks as well.
In this study, we present Swish-Gumbel, a new family of activation functions generated by combining the Swish and Gumbel functions through both multiplication and linear composition. We evaluate their performance on several image classification benchmarks, including MNIST, CIFAR-10, Imagenette, and Beans, using different deep learning architectures. The results show that the multiplicative version consistently outperforms well-known activation functions such as ReLU, Swish, Mish, and Gumbel, while the linear combination variants provide competitive results particularly on simpler datasets, with performance gains becoming more pronounced for the multiplicative form on complex benchmarks. The strength of the proposed approach lies in its ability to maintain more stable gradients in the positive domain, which improves feature extraction and generalization. Overall, our findings suggest that Swish-Gumbel can serve as an effective alternative to existing activation functions for image classification, with potential to be applied in other machine learning tasks as well.