A Comparative Study of Humans and Machine Learning in Metaphor Detection: Translations of Legal Metaphors in English and Turkish HUDOC Judgments


Şen Bartan Ö., ARICA AKKÖK E., Us K. Y.

International Journal for the Semiotics of Law, 2026 (ESCI, Scopus) identifier

  • Publication Type: Article / Article
  • Publication Date: 2026
  • Doi Number: 10.1007/s11196-026-10468-z
  • Journal Name: International Journal for the Semiotics of Law
  • Journal Indexes: Emerging Sources Citation Index (ESCI), Scopus, IBZ Online, Index Islamicus, Linguistic Bibliography, Philosopher's Index
  • Keywords: Conceptual metaphor theory (CMT), Legal metaphors, Machine learning, OpenAI, Translation studies
  • Ankara University Affiliated: Yes

Abstract

This study aims to investigate the effectiveness of few-shot metaphor detection using Large Language Models, to compare their performance with that of human annotators, and to analyze similarities and differences in conceptual legal metaphors in English and Turkish, by analyzing the source and target texts of European Court of Human Rights judgments. The study database consists of 12 English–Turkish judgments, preprocessed using regular expressions and automatic alignment. The sentence-aligned texts of the database were parsed by Natural Language Processing tools to extract their part-of-speech, sentence dependency, named entity and morphological information. The linguistic-information labeled database was then parsed by ChatGPT Large Language Model version GPT-4o (“Omni”) via OpenAI API with few-shot learning for automatic legal domain conceptual metaphor detection, following a prompt-improval trial phase on a test case. The performance of Large Language Model in conceptual metaphor, target domain, source domain and frame detection by ChatGPT were annotated and evaluated for performance by human annotators within the framework of Conceptual Metaphor Theory. The results show that the few-shot learning is highly effective (100%) in automatic metaphor detection. Following the refinement of the system prompt and few-shot examples after tests on a larger dataset, the model’s performance in source and target domain and frame labeling improved markedly. Comparative linguistics analyses reveal metaphorical density is preserved across English and Turkish judgments in translation, but systematic shifts are observed in conceptual domains: whereas English favors FORCE and STRUCTURE metaphors, Turkish translations show increased use of PATH metaphors, reflecting language-specific cognitive preferences, indicating English to Turkish legal metaphor translation maintains quantitative stability, but involves qualitative conceptual reconfiguration. Our results support the view that metaphor translation entails cultural and cognitive adaptation rather than direct equivalence.