- you have to make sure you are using the proper translation.
There is no unique solution to automating localization testing. We know we cannot use most of the defaults the automation programs offer. And we know several tricks that can work in particular situations. We even have a professional approach able to automate 70 to 80 percent of the testing by building resource files.
But each time we write a script we want to run in a set of languages, we will face different issues, and the solution will have to be adapted to the situation. One good option to automate localization testing is skipping the automation programs on the market and developing your own software using Visual Basic, but this would be the subject of another article.
When Do You Automate?
First of all, automation cannot substitute for manual testing when we talk about localization. It can be useful to avoid human errors when trying to reach a particular dialog, and it is very good as help for translators not very skilled in computers. It also saves a lot of time, as you can run setups during the night, so all the environments are ready by the time testers arrive in the morning. But you need the tester or the translator to manually check the final dialog your automated script can reach.
Also, if you want to automate, you will need to fulfill some previous requirements:
- You need the original software well in advance of the testing start day. Depending on the number of tests you want to automate, you could need two weeks or two months.
- You need to test a pilot language first. You have created and run the tests against the original language software. The first time you run your tests against a localized version, you will get a lot of errors. The pilot language will help you identify those errors and create a revised set of scripts for the remaining languages.
- You need a dedicated team of automation engineers to handle the tester and translator issues with your tests. The size of the team will depend on the number of testers and translators, but one automation engineer for every ten users is usually a good ratio.
- You need to use a "controlled" environment for the testing. The localized machines must be a "mirror" of the original machine: no extra software, no different drivers or graphical cards. The more similar they are, the more chance of success you'll have.
We can think of at least four scenarios where the efforts to automate localization testing are worthwhile:
1) When you are running a lot of test cases in three or more languages. Let's say you have to run 1,000 test cases in six languages. The automation will help you to reach the dialogs you need to test, avoiding human errors during the steps you have to perform to reach those dialogs. You can write a test that performs all the operations needed to find that dialog your tester has to review.
You can even check the duplicated hotkeys (most automation programs have a function just for this). But you need the tester to check if the translation is precise; if the concatenation of different strings makes sense; if there are truncations or inconsistencies between translations in that dialog and other parts of the software; and any other particular check you need for your project. There is no way to fully automate those tasks with enough reliability
But if you are able to automate those 1,000 tests, so your team doesn't need to learn about the program functionality,