Edit Distance Variations

Edit distance refers to the percentage of modifications applied to a translated text relative to the original content. It measures how much a translator alters a machine-generated translation to achieve the desired outcome.

For instance, if six words appear in a sentence and three are altered, removed, or rearranged, the edit distance is 50%. This metric serves as a useful indicator to gauge how close a machine-translated output is to the final, polished translation.

Variability in Language Pairs and Domains

Edit distance varies not only across different languages but also according to the type of content being translated. According to a large-scale study examining over 50 million words in multiple domains, the average edit distances for various language pairs were recorded as follows:

• English to Spanish: 5.5%
• English to French: 12.4%
• English to Danish: 4.5%
• English to Thai: 11.8%

The range of these percentages highlights the nuances of different languages. However, individual user behavior adds further complexity to understanding and optimizing translation processes.

Exploring User Behavior in Translation

One of the most fascinating aspects revealed in the study was the variation in how much individual translators modify content, regardless of the language pair or domain. For example:

• Some translators make less than 1% of edits, suggesting they might be merely confirming machine translations without many adjustments.
• Others make up to 77% of edits, rewriting significant portions of the original output.

As our CEO, Gabriel Fairman observes:

‍"You see users who are... diving in headfirst to this AI editing thing... and you also see users who are... almost fighting it, or denying it, and really putting in the full effort to rewrite most sentences."

This wide range of behavior raises key questions:

• Are fewer edits indicative of high-quality machine output or poor-quality human oversight?
• Do excessive edits signal a thorough rewriting process or inefficient translation efforts?

Striking the Right Balance Between Edits and Efficiency

Translation today, especially when combined with AI and machine translation, requires striking a delicate balance between the number of edits and overall productivity. Translators face two main challenges:

• Too few edits: May lead to inaccuracies if critical adjustments are missed.
• Too many edits: This can result in inefficiencies, reducing productivity and increasing costs.

As our CEO, Gabriel Fairman, mentions in his Edit Distance-related video series:

‍"You have users who are changing less than 1%, which means they're basically just reading things and confirming things... And then you have other translators who are translating 77% of that content, which is nearly more than 13 times more [than average]."

Key considerations for optimal editing:

• Edit only what is necessary: Translators should avoid redundant changes, such as swapping synonymous terms ("platform" for "system").‍
• Focus on impactful edits: Critical adjustments, such as maintaining a proper noun or correcting terminology, have a lasting impact on quality.

Productivity vs. Ownership in Translation

Another challenge lies in maintaining a sense of ownership over the translated material. Translators must balance their desire to feel like the author with the need to work efficiently. Economic realities demand that translators complete tasks quickly, but too many changes risk slowing them down and increasing the likelihood of errors.

This tension becomes especially apparent in tasks like post-editing AI outputs, where translators aim to find the “sweet spot” – making just enough edits to improve quality without diminishing productivity. Translators who make excessive changes to non-creative content (like technical documentation) risk becoming unproductive, potentially jeopardizing their relevance in the industry.

Defining the Ideal Edit Distance

Ultimately, determining the ideal amount of edits – or what “good” looks like – depends not only on the language pair but also on user behavior and content type. For translators to stay competitive, they must learn to adapt, striking a balance between speed and accuracy. Excessive or minimal edits both come with risks and only by finding the right balance can they maintain productivity and quality.

The journey toward achieving this optimal balance in post-editing AI content remains ongoing, but understanding the dynamics of edit distance offers a valuable lens into the challenges and opportunities that lie ahead.