Views on machine translation: An analysis on the performance of DeepL and Google Translate
Keywords:
Machine translation output error typology, Neural machine translation, Machine translation evaluationAbstract
This paper comparatively evaluates the performance of two machine translation systems, namely DeepL and Google Translator. It starts with a brief overview of machine translation using neural networks, followed by a reflection concerning the performance of machine translation output using human and automatic evaluation. After that, this study addresses the difference in performance between both systems and identifies possible linguistic problems, possibly generated from these machine translation output. To carry on such comparison, this study employs an excerpt from Machado de Assis's Dom Casmurro, taken form the chapter "Olhos de Ressaca". This investigation uses the automatically generated translation outputs to critically analyze which system conveys more quality in their translation output and to comparatively examine linguistic errors generated by DeepL e Google Translate results. The evaluation criteria include the accuracy of the translation, the ability to maintain the meaning and structure of the original sentence, as well as fluency and adequacy of the resulting translation.
Downloads
References
ASSIS, M. de. Dom Casmurro. Rio de Janeiro: Edições Câmara, 2019.
BANITZ, B. Machine translation: a critical look at the performance of rule-based and statistical machine translation. Cadernos de Tradução, 40(1), 2020. Disponível em:
https://doi.org/10.5007/2175-7968.2020v40n1p54 . Acesso em 17 de novembro de 2023.
BROWNLEE, J. A Gentle Introduction to Calculating the BLEU Score for Text in Python. 2019. https://machinelearningmastery.com/calculate-bleu-score-for-text-python/ > Acesso em 17 de novembro de 2023.
CALLISON-BURCH, C., OSBORNE, M., & KOEHN, P. Re-evaluating the Role of BLEU in Machine Translation Research, 2006. https://aclanthology.org/E06-1032.pdf . Acesso em 17 de novembro de 2023.
DEEPL PRESS INFORMATION. (n.d.). https://www.deepl.com/en/press.html. Acesso em 17 de novembro de 2023.
DENSMER, L.. Interview with an Expert: How Do You Measure MT? RWS. 2019. https://www.rws.com/blog/interview-with-an-expert-how-do-you-measure-mt/ . Acesso em 17 de novembro de 2023
FORCADA, M. Machine translation today. In Handbook of Translation Studies (pp. 215–223). John Benjamins Publishing Company, 2010.
GOOGLE CLOUD. Como avaliar modelos. 2022. https://cloud.google.com/translate/automl/docs/evaluate?hl=pt-br. Acesso em 17 de novembro de 2023.
HUTCHINS, W. J., & SOMERS, H. L. An introduction to machine translation (pp. 215-223). Academic Press, 1992.
ISABELLE, P., CHERRY, C., & FOSTER, G. A challenge set approach to evaluating machine translation. Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, 2486–2496, 2017. https://aclanthology.org/D17-1263.pdf . Acesso em 17 de novembro de 2023.
ISABELLE, P., & KUHN, R. A Challenge Set for French -> English Machine Translation. 2018. https://arxiv.org/pdf/1806.02725.pdf. Acesso em 17 de novembro de 2023.
MATTHES, E. Curso Intensivo de Python. Novatec Editora. 2016.
PLANETCALC. Levenshtein distance.2022. http://planetcalc.com/1721. Acesso em 17 de novembro de 2023.
SCHUSTER, M., JOHNSON, M., & THORAT, N. Zero-Shot Translation with Google’s Multilingual Neural Machine Translation System. 2016.
https://ai.googleblog.com/2016/11/zero-shot-translation-with-googles.html . Acesso em 17 de novembro de 2023.
TAN, Z., WANG, S., YANG, Z., CHEN, G., HUANG, X., SUN, M., & LIU, Y. Neural machine translation: a review of methods, resources, and tools. AI Open, 5–21, 2020.
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 Babel: Revista Eletrônica de Línguas e Literaturas Estrangeiras
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Os autores detém os direitos autorais sem restrições, porém ao submeter os originais, concordam em transferir a este periódico os direitos da primeira publicação. Isto deve ser informado em caso de nova edição do texto. As produções que derivarem deste material, devem obrigatoriamente citar a fonte. Os textos publicados nesta revista, salvo indicações contrárias, encontram-se sob uma licença Creative Commons Atribuição 4.0 Internacional.