Transformer-Based HamNoSys Transcription Using Skeleton Sequences
2026 International Conference on Control, Automation and Diagnosis, ICCAD 2026, Lisbon, Portekiz, 7 - 09 Temmuz 2026, (Tam Metin Bildiri)
- Yayın Türü: Bildiri / Tam Metin Bildiri
- Doi Numarası: 10.1109/iccad69956.2026.11642983
- Basıldığı Şehir: Lisbon
- Basıldığı Ülke: Portekiz
- Anahtar Kelimeler: Deaf Communication, HamNoSys, Multilingual Sign Language, Seq2Seq, Sign Language Recognition
- Ankara Üniversitesi Adresli: Evet
Özet
The development of robust sign language processing systems is crucial for bridging the communication gap between deaf and hearing communities. Sign language transcription into formal notation systems remains challenging due to the complex spatiotemporal nature of signing. Moreover, most existing approaches rely on gloss annotations, which are language-specific and require extensive manual labeling. This paper investigates transformer-based HamNoSys transcription from skeleton sequences. The language-independent nature of HamNoSys allows our approach to potentially support multilingual processing with reduced annotation costs. To improve the representation of dynamic signing characteristics, we augment the baseline keypoint features with velocity and angular features derived from joint movements. Experiments conducted on the Ham2Pose dataset, encompassing German, Polish, Greek and French sign languages, demonstrate that incorporating kinematic features achieves a word error rate of 66.1%.