Skip to main content

Introduction

This is a 2-minute introduction to our task, Entity-Aware Machine Translation (EA-MT), for SemEval-2025.

What is SemEval?​

SemEval is a long-standing series of international natural language processing (NLP) research workshops whose mission is to advance the current state of the art in semantic analysis and to help create high-quality annotated datasets in a range of increasingly challenging problems in natural language semantics.

More on SemEval

You can find more information about SemEval on the official website.

EA-MT: Entity-Aware Machine Translation​

Let's dive into our SemEval-2025 task, Entity-Aware Machine Translation (EA-MT).

Background Image

What is it about?​

We invite participants to develop machine translation systems that can accurately translate text that includes potentially challenging named entities in the source language. The task is to translate a given input sentence from the source language (English) to the target language, where the input sentence contains named entities that may be challenging for machine translation systems to handle. The named entities may be entities that are rare, ambiguous, or unknown to the machine translation system. The task is to develop machine translation systems that can accurately translate such named entities in the input sentence to the target language.

Why is it important?​

We believe that the ability to accurately translate named entities is crucial for machine translation systems to be effective in real-world scenarios. Named entities are entities that are referred to by proper names, such as people, organizations, locations, dates, and more. Named entities are often challenging even for human translators, as sometimes there are cultural or domain-specific references that are not easily translatable. This happens more often for some entity types or categories, such as movies, books, TV series, products, and more.

Examples​

Here are some examples of sentences that you may encounter in the EA-MT task:

Example 1: English to French​

  • English Sentence: "I watched the movie 'The Shawshank Redemption' last night."
  • French Sentence: "J'ai regardé le film 'Les Évadés' hier soir."

Example 2: English to Italian​

  • English Sentence: "I bought a new book called 'The Catcher in the Rye'."
  • Italian Sentence: "Ho comprato un nuovo libro chiamato 'Il Giovane Holden'."

Example 3: English to Chinese​

  • English Sentence: "I watched the TV series 'Breaking Bad' last week."
  • Chinese Sentence: "我上周看了电视剧《绝命毒师》。"

Example 4: English to Korean​

  • English Sentence: "Who is the author of the book 'The Great Gatsby'?"
  • Korean Sentence: "'위대한 개츠비'의 저자는 누구입니까?"

Language Pairs​

The EA-MT task will focus on the following language pairs:

  • English to Arabic
  • English to Chinese
  • English to French
  • English to German
  • English to Italian
  • English to Japanese
  • English to Korean
  • English to Spanish
  • English to Thai
  • English to Turkish
note

This edition of EA-MT will focus on translating from English to the target languages mentioned above. We may consider adding more language pairs in future editions.

Next steps​

Please, stay tuned for more updates on the EA-MT task for SemEval-2025. We will be releasing more information on the dataset, evaluation metrics, and submission guidelines soon. If you have any questions, feel free to reach out to us.

Join the Google Group

We invite you to join our Google Group for the latest updates and discussions: SemEval 2025 - Task 2: EA-MT.