Özet
The aim of this thesis is to compare the temporal and technical effort spent by translators to post-edit machine translation outputs with editing and translation from scratch. The research also compares a statistical machine translation engine specially trained for the experiment with a public neural machine translation engine. For the purposes of the thesis, a sample of higher education students took part in an experiment in which they had to post-edit machine translation output, edit human translation, or translate from scratch from English to Turkish. The experiment was conducted with news texts on a common computer-assisted translation tool. The amount of time participants spent editing sentences and the amount of editing they did were quantitatively measured. The results showed that there was no significant difference between the specially trained statistical machine translation engine and the neural machine translation engine. It was found that the participants spent more technical and temporal effort when translating from scratch and editing human translation than post-editing machine translation. This thesis aims to serve as a guide for stakeholders in evaluating the benefits of post-editing, a relatively new service in the translation industry, for the client, the project and the translator, and to encourage much-needed similar studies in this language pair.
Künye
Alotaibi, H. (2014). Teaching CAT Tools to Translation Students: an Examination of Their Expectations and Attitudes. AWEJ, 3, 65–74.
Anastasiou, D. (2008). Idioms in example-based machine translation. https://www.academia.edu/30082514/Idioms_in_example_based_machine_translation
Aranberri, N. (2017). What Do Professional Translators Do when Post-Editing for the First Time? First Insight into the Spanish-Basque Language Pair. HERMES - Journal of Language and Communication in Business, 56, 89–110. https://doi.org/10.7146/hjlcb.v0i56.97235
Aziz, W., Castilho, S., & Specia, L. (2012). PET: a Tool for Post-editing and Assessing Machine Translation. Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC’12), 3982–3987. http://www.lrec-conf.org/proceedings/lrec2012/pdf/985_Paper.pdf
Bundgaard, K., & Christensen, T. P. (2019). Is the concordance feature the new black? A workplace study of translators’ interaction with translation resources while post-editing TM and MT matches. The Journal of Specialised Translation, 31, 14–37.
Cadwell, P., O’Brien, S., & Teixeira, C. S. C. (2018). Resistance and accommodation: factors for the (non-) adoption of machine translation among professional translators. Perspectives: Studies in Translatology, 26(3), 301–321. https://doi.org/10.1080/0907676X.2017.1337210
Carl, M., Dragsted, B., Elming, J., Hardt, D., & Jakobsen, A. L. (2011). The Process of Post-Editing: a Pilot Study. 8th International NLPCS Workshop, August, 131–142.
CAT Tools | Software Comparison Tool. (n.d.). Retrieved May 18, 2022, from https://www.proz.com/software-comparison-tool/cat/cat_tools/2
Çetiner, C. (2019). Makine çevirisi sonrası düzeltme işleminin çeviri öğrencilerinin tutum ve çeviri performanslarına etkisi [Unpublished PhD Thesis, Gazi University]. https://doi.org/10.29000/rumelide.649333
Chemvura, T. (2017). LARMAS - Language Resource Management System. https://doi.org/10.13140/RG.2.2.34784.38405
Daems, J., Vandepitte, S., Hartsuiker, R. J., & Macken, L. (2017). Translation Methods and Experience: A Comparative Analysis of Human Translation and Post-editing with Students and Professional Translators. Meta: Journal Des Traducteurs, 62(2), 245. https://doi.org/10.7202/1041023ar
Denkowski, M., & Lavie, A. (2011). Meteor 1.3: Automatic metric for reliable optimization and evaluation of machine translation systems. Proceedings of the Sixth Workshop on Statistical Machine Translation, 85–91. https://doi.org/10.1080/00288306.2004.9515087
Depraetere, I., Sutter, N. de, & Tezcan, A. (2014). Post-edited quality, post-editing behaviour and human evaluation: a case study. 78-108. https://hal.archives-ouvertes.fr/halshs-01060447
Diño, G. (2018). Google, Facebook, Amazon: Neural Machine Translation Just Had Its Busiest Month Ever | Slator. Slator. https://slator.com/technology/google-facebook-amazon-neural-machine-translation-just-had-its-busiest-month-ever/
DuPont, Q. (2018). The Cryptological Origins of Machine Translation. AModern. https://amodern.net/article/cryptological-origins-machine-translation/
Egdom, G. van, & Pluymaekers, M. (2019). Why go the extra mile? How different degrees of post-editing affect perceptions of texts, senders and products among end users. The Journal of Specialised Translation, 31, 158–176.
Farooq, U. (2018, March 21). Neural Machine Translation with Code. Medium. https://medium.com/@umerfarooq_26378/neural-machine-translation-with-code-68c425044bbd
Fiederer, R., & O’Brien, S. (2009). Quality and Machine Translation: A realistic objective? The Journal of Specialised Translation, 11, 52–74.
Flanagan, M., & Christensen, T. P. (2014). Testing post-editing guidelines: how translation trainees interpret them and how to tailor them for translator training purposes. The Interpreter and Translator Trainer, 8(2), 257–275. https://doi.org/10.1080/1750399X.2014.936111
Folaron, D. A. (2010). Networking and volunteer translators. In Handbook of Translation Studies (pp. 231–234). John Benjamins Publishing Company. https://doi.org/10.1075/hts.1.net1
Garcia, I. (2010). Is machine translation ready yet? Target, 22(1), 7–21. https://doi.org/10.1075/target.22.1.02gar
Gaspari, F., Toral, A., Kumar Naskar, S., Groves, D., & Way, A. (2014). Perception vs Reality: Measuring Machine Translation Post-Editing Productivity. AMTA 2014.
Gerlach, J., Porro Rodriguez, V., Bouillon, P., & Lehmann, S. (2013). Combining pre-editing and post-editing to improve SMT of user-generated content. Proceedings of MT Summit XIV Workshop on Post-Editing Technology and Practice, 2(1), 45–53.
Gross, A. (1992). Limitations of computers as translation tools. In J. Newton (Ed.), Computers in Translation: A Practical Appraisal. Routledge.
Guerberof Arenas, A. (2008). Productivity and quality in the post-editing of outputs from translation memories and machine translation. Localisation Focus The International Journal of Localisation, 7(1), 11–21.
Hutchins, J. (1999). Milestones in machine translation — No.6: Bar-Hillel and the nonfeasibility of FAHQT. http://www.mt-archive.info/Weaver-1949.pdf
Hutchins, W. J. (1986). Machine translation: past, present, future. Ellis Horwood ; Halsted Press.
Jia, Y., Carl, M., & Wang, X. (2019). How does the post-editing of neural machine translation compare with from-scratch translation? A product and process study. Journal of Specialised Translation, 31, 60–86.
Koehn, P. (2016). CASMACAT: Advanced Computer Aided Translation Beyond Postediting. https://ec.europa.eu/info/sites/info/files/tef2016_koehn_en.pdf
Koponen, M. (2012). Comparing human perceptions of post-editing effort with post-editing operations. 181–190. http://www.statmt.org/wmt12/
Koponen, M. (2015). How to teach machine translation post-editing ? Experiences from a post-editing course. Proceedings of 4th Workshop on Post-Editing Technology and Practice (WPTP4).
Koponen, M. (2018). Learning to post-edit: An analysis of post-editing quality and processes of translation students. International Association for Translation and Intercultural Studies (IATIS) 6th International Conference, July. https://doi.org/10.13140/RG.2.2.24675.04648
Koponen, M., Aziz, W., Ramos, L., & Specia, L. (2012). Post-editing time as a measure of cognitive effort. AMTA Workshop on Postediting Technology and Practice, 47(3), 11–20. https://doi.org/10.1111/j.1469-8986.2009.00947.x.Pupillometry
Koponen, M., & Salmi, L. (2017). Post-editing quality: Analysing the correctness and necessity of post-editor corrections. Linguistica Antverpiensia, New Series: Themes in Translation Studies, 16, 137–148.
Krings, H. P. (2001). Repairing Texts: Empirical Investigations of Machine Translation Post-editing Processes (G. S. Koby, Ed.). The Kent State University Press. https://books.google.com.tr/books?id=vsdPsIXCiWAC
Lacruz, I., & Jääskeläinen, R. (2018). Innovation and Expansion in Translation Process Research. John Benjamins Publishing Company.
Lacruz, I., Shreve, G. M., & Angelone, E. (2012). Average Pause Ratio as an Indicator of Cognitive Effort in Post-Editing: A Case Study. AMTA 2012 Workshop on Post-Editing Technology and Practice (WPTP 2012), 21–30.
Lardilleux, A., & Lepage, Y. (2017). CharCut: Human-Targeted Character-Based MT Evaluation with Loose Differences. Proceedings of IWSLT 2017.
Läubli, S., Fishel, M., Massey, G., Ehrensberger-Dow, M., & Volk, M. (2013, September). Assessing post-editing efficiency in a realistic translation environment. Proceedings of the 2nd Workshop on Post-Editing Technology and Practice. https://aclanthology.org/2013.mtsummit-wptp.10
Lauscher, S. (2000). Translation Quality Assessment. The Translator, 6, 149–168. https://doi.org/10.1080/13556509.2000.10799063
Le, Q. v., & Schuster, M. (2016, September 27). A Neural Network for Machine Translation, at Production Scale. Google AI Blog. https://ai.googleblog.com/2016/09/a-neural-network-for-machine.html
Lilly, P. (2016, September 28). Google Translate Has Reached Human-Like Accuracy Thanks To Neural Machine Translation Engine. HotHardware. https://hothardware.com/news/google-translate-human-like-accuracy-neural-machine-translation-engine
Lommel, A. (2016, June 22). MT is Changing the Industry, Just Not in the Way Mainstream Media Thinks It Will. https://csa-research.com/Blogs-Events/Blog/MT-is-Changing-the-Industry-Just-Not-in-the-Way-Mainstream-Media-Thinks-It-Will
Massardo, I., van der Meer, J., O’Brien, S., Hollowood, F., Aranberri, N., & Drescher, K. (2016). MT POST-EDITING GUIDELINES. TAUS Signature Editions.
Milengo GmbH. (2019, January 30). How NMT-Based Translation Services Can Reduce Enterprise Translation Costs by up to 80%. Slator. https://slator.com/how-nmt-based-translation-services-can-reduce-enterprise-translation-costs-by-up-to-80/
Moorkens, J., O’Brien, S., da Silva, I. A. L., de Lima Fonseca, N. B., & Alves, F. (2015). Correlations of perceived post-editing effort with measurements of actual effort. Machine Translation, 29(3–4), 267–284. https://doi.org/10.1007/s10590-015-9175-2
Nitzke, J., & Oster, K. (2016). Comparing Translation and Post-editing: An Annotation Schema for Activity Units. In M. Carl, S. Bangalore, & M. Schaeffer (Eds.), New Directions in Empirical Translation Process Research. New Frontiers in Translation Studies. Springer, Cham. https://doi.org/https://doi.org/10.1007/978-3-319-20358-4_14
O’Brien, S. (2002). Teaching Post-Editing: A Proposal for Course Content. Proceedings of the 6th EAMT Workshop: Teaching Machine Translation. https://aclanthology.org/2002.eamt-1.11
O’Brien, S. (2006). Pauses as Indicators of Cognitive Effort in Post-editing Machine Translation Output. Across Languages and Cultures, 7(1), 1–21. https://doi.org/10.1556/acr.7.2006.1.1
O’Brien, S. (2012). Translation as human–computer interaction. Translation Spaces, 1, 101–122. https://doi.org/10.1075/ts.1.05obr
Paas, F., Tuovinen, J. E., Tabbers, H., & van Gerven, P. W. M. (2003). Cognitive Load Measurement as a Means to Advance Cognitive Load Theory. Educational Psychologist, 38(1), 63–71. https://doi.org/10.1207/S15326985EP3801_8
Plitt, M., & Masselot, F. (2010). A Productivity Test of Statistical Machine Translation Post-Editing in a Typical Localisation Context. The Prague Bulletin of Mathematical Linguistics, 93(1), 7–16. https://doi.org/10.2478/v10108-010-0010-x
Poibeau, T. (2017). Machine Translation. Cambridge, MA: The MIT Press.
Popović, M., Lommel, A., Burchardt, A., Avramidis, E., & Uszkoreit, H. (2014). Relations between different types of post-editing operations, cognitive effort and temporal effort. Proceedings of the 17th Annual Conference of the European Association for Machine Translation, 191–198. https://www.researchgate.net/publication/268446855%0Ahttp://www.dfki.de/web/forschung/publikationen/renameFileForDownload?filename=finalVersion48.pdf&file_id=uploads_2255
Pym, A. (2013). Translation Skill-Sets in a Machine-Translation Age. Meta: Journal Des Traducteurs, 58, 487. https://doi.org/10.7202/1025047ar
R Core Team. (2018). R: A Language and Environment for Statistical Computing. https://www.r-project.org/
Rico, C., & Torrejón, E. (2012). Skills and Profile of the New Role of the Translator as MT Post-editor. Revista Tradumàtica: Tecnologies de La Traducció, 2012(10), 166–178. http://revistes.uab.cat/http://revistes.uab.cat/tradumatica
Sánchez-Gijón, P., Moorkens, J., & Way, A. (2019). Post-editing neural machine translation versus translation memory segments. Machine Translation, 33(1–2), 31–59. https://doi.org/10.1007/s10590-019-09232-x
Sankaravelayuthan, R., & Vasuki, G. (2013). English to Tamil machine translation system using parallel corpus. LAP LAMBERT Academic Publishing. https://www.perlego.com/book/3413858/english-to-tamil-machine-translation-system-using-parallel-corpus-pdf
Schwartz, L. (2018). The history and promise of machine translation. American Translators Association Scholarly Monograph Series, 18, 161–190. https://doi.org/10.1075/ata.18.08sch
Screen, B. (2019). What effect does post-editing have on the translation product from an end- user’s perspective? The Journal of Specialised Translation, 500(31), 133–157.
Sin-wai, C. (2017). The Future of Translation Technology: Towards a World without Babel. Routledge.
Stefaniak, K. (n.d.). Post-editing in DGT. European Commission, Directorate-General for Translation.
Sukkhwan, A., & Sripetpun, W. (2014). Students’ attitudes and behaviors towards the use of google translate [Unpublished master’s thesis]. Prince of Songkla University.
Teixeira, C. S. C. (2014). Perceived vs. measured performance in the post-editing of suggestions from machine translation and translation memories. Proceedings of the 11th Conference of the Association for Machine Translation in the Americas, 45–59. https://aclanthology.org/2014.amta-wptp.4
Temizöz, Ö. (2012). Machine translation and postediting. Est-Translationstudies.Org, 19. http://www.est-translationstudies.org/intranet/research/MT.pdf
Temizöz, Ö. (2016). Postediting machine translation output: subject-matter experts versus professional translators. Perspectives: Studies in Translatology, 24(4), 646–665. https://doi.org/10.1080/0907676X.2015.1119862
Thames, J. (2019, June 24). Machine Translation. LanguageSolutions. https://langsolinc.com/machine-translation/
Toral, A., Wieling, M., & Way, A. (2018). Post-editing Effort of a Novel With Statistical and Neural Machine Translation. Frontiers in Digital Humanities, 5. https://doi.org/10.3389/fdigh.2018.00009
Trados Studio - Translation Software. (n.d.). Retrieved May 18, 2022, from https://www.trados.com/products/trados-studio/
Turovsky, B. (2016, November 15). Found in translation: More accurate, fluent sentences in Google Translate. The Keyword. https://blog.google/products/translate/found-translation-more-accurate-fluent-sentences-google-translate/
Vanroy, B., de Clercq, O., & Macken, L. (2019). Correlating process and product data to get an insight into translation difficulty. Perspectives: Studies in Translation Theory and Practice, 27(6), 924–941. https://doi.org/10.1080/0907676X.2019.1594319
Vieira, L. N. (2016). How do measures of cognitive effort relate to each other? A multivariate analysis of post-editing process data. Machine Translation, 30(1–2), 41–62. https://doi.org/10.1007/s10590-016-9188-5
Vieira, L. N. (2018). Automation anxiety and translators. Translation Studies, 0(0), 1–21. https://doi.org/10.1080/14781700.2018.1543613
Yamada, M. (2019). The impact of Google Neural Machine Translation on Post-editing by student translators. The Journal of Specialised Translation, 31, 87–106.
