EXPERT SYSTEMS WITH APPLICATIONS, cilt.304, 2026 (SCI-Expanded, Scopus)
With the rapid growth of multimedia transmission, image encryption (IE) has become an essential component of modern secure communication systems. However, many conventional chaos-based encryption algorithms often suffer from a limited key space and insufficient diffusion capability, having security risks to highly correlated image. To address these critical challenges and enhance cryptographic strength, this study proposes a 3D hyperchaotic system-driven Multi-Layer Multi-Directional IE (MLMD-IE) algorithm. The MLMD-IE algorithm is driven by a newly developed hyperchaotic three-dimensional Xin-She Yang (3D-XSY) map. The 3D-XSY map, constructed by hybridizing three nonlinear optimization functions, enhances entropy and unpredictability through exponential modulation and sinusoidal coupling. The MLMD-IE algorithm integrates hexadecimaldomain permutation with four-layer, four-directional diffusion, providing strong pixel decorrelation and high randomness. Experimental results show that the scheme achieves high entropy, low correlation, and robustness against noise and cropping attacks while maintaining superior decryption quality compared with existing methods. These findings demonstrate that the proposed MLMD-IE framework offers a highly efficient and secure solution for contemporary image encryption applications.