Guide To Automating the Internationalization(i18n) Process

For any web applications that target the global stage, internationalization(i18n) is a must. Internationalization is often implemented through i18n libraries or through self-implemented code. This article is written with the front-end developers who are suffering from the manual “copy-and-paste” process in mind. Follow this guide and executing a single line of script shall set you free.

General Internationalization Support

Libraries that use gettext or libraries like i18next provide functions that take the key and return the translated string like the example below.

// Translated Data
const ko_KR = {
    'name': '이름'
};
// Using gettext
const translation = gettext('name');
// 이름
// Using i18next
const translation2 = i18next.t('name');
// '이름'

The returned string will differ according to the language configured by the user.

Why Is Internationalization Necessary?

I’m contributing to Dooray!, a global, all-in-one collaboration tool that supports Korean, English, Japanese, and Mandarin. Because it supports most of the tools required for online collaboration, there is a lot of text in the application. The number of keys used for translation was something around ten thousand. As more strings needed to be translated, the cost of collaboration among developers, organizers, and translators increased, and we had to go through a lengthy translation process every time we released a new version.

There will be new projects; it is inevitable as long as the development continues. As the entire world connects through the internet and in order to service all of your ideas globally, internationalization must also continue. Therefore, the repetitive, error-prone, and manual labor-internationalization-must be improved.

Issues With the Original Internationalization Process

Since there are innumerable issues with the original method, let’s focus on the parts that require manual labor.

• The key must be added to the JavaScript module that collectively manages the keys.
• Added key must be added to Excel file in order to be passed to the translator with notes on what needs to be translated.
• Manually copy-and-paste from the translated file and update the ko-KR.json file. Repeat for different languages.

During this process, you can save a lot of time by using a scanner that scans the keys added in the source code. However, it makes the process of passing around the file troublesome, and it can also make it hard to differentiate which branch the file was scanned from. Furthermore, the file itself is saved onto the repository, so we had to manage the conflicts from the translation file, separately.

Collaboration Using Google Spreadsheets

Google spreadsheets make the process of passing around the file obsolete and more convenient. Using the automation tools that update and download the Google spreadsheet data allows developers to request translation regardless of which branch it is based on since all keys are included in the spreadsheet. The clear advantages lie in the fact that you don't need version control, unlike passing the file around. Since Google spreadsheets is available for anyone with a browser, organizers and translators can easily access the data without installing a separate tool.

Now, let’s see how we can improve the parts of internationalization that is time-consuming and error-prone using Google spreadsheets and automation tools.

Automation Using JavaScript

We will build two features in order to automate internationalization.

• Scanning the key from the source code and uploading the scanned key to Google spreadsheets.
• Downloading the translated strings from Google spreadsheets when building the source code.

Installing the Tools

There are some Node.js packages that we need to install in order to implement automation.

npm install i18next               // v19.6.3 as of date written
npm install -D i18next-scanner    // v2.11.0 as of date written
npm install -D google-spreadsheet // v3.0.1 as of date written

i18next is a JavaScript internationalization library. Its features include pluralization, interpolation, and context handling, and the i18next.t('key') function is used to display the translated results.

const translation = i18next.t('name');
// '이름'

i18next-scanner scans the source code for designated patterns like i18next.t() and i18n.t() to extract the key and generates a json file for each language.

{
  "translation": {
    "name": "",
    "keys...": ""
  }
}

google-spreadsheet is a wrapper library that lets you use the Google Sheets API (v4) with Node.js. It can be used to create a new sheet or to read, write, and manage cells and rows.

const doc = new GoogleSpreadsheet('spreadSheetDocId');
...
const sheet = doc.sheetsById['sheetId'];
const rows = await sheet.getRows();
...
await doc.addRows(newRows);

Adding the Configuration File

The config file is available in the following link, and I will go over the main aspects of the code.

• gist Link

Add the script to the package.json

{
  "scan:i18n": "i18next-scanner --config i18next-scanner.config.js",
  "upload:i18n": "npm run scan:i18n && node translate/upload.js",
  "download:i18n": "node translate/download.js",
  "serve": "npm run download:i18n && vue-cli-service serve"
}

scan:i18n extracts the key from the source code and generates a json file composed of key and value for each language. upload:i18n creates a single table by combining all of the json files for different languages and uploads it to Google spreadsheets, and download:i18n, on the other hand, applies the translated values to each json language file. Add them to the npm scripts in order to ensure that the translation files are included for both local development and production builds.

Google Spreadsheet Authorizations and Adding New Sheets

Now, we need to create a new spreadsheet to be used for translation. The spreadsheet id needed for the source code can be found in the sheet’s URL. The id will be followed by the string spreadSheetDocId.

• https://docs.google.com/spreadsheets/d/{spreadSheetDocId}/edit?usp=sharing

Then, since JavaScript needs to manipulate the spreadsheet with google-spreadsheet, follow the following document to provide appropriate Google account authorizations.

• Google Spreadsheet Authentication — Service Account

Add a bot user as the service account and use the downloaded json credentials (creds).

const {GoogleSpreadsheet} = require('google-spreadsheet');
const creds = require('./.credentials/your-app-some-unique-id.json');
const doc = new GoogleSpreadsheet('spreadSheetDocId');
await doc.useServiceAccountAuth(creds);
await doc.loadInfo();

The json credentials need to be handled with care as it can create holes in security. Make sure that the credentials are not uploaded to a public Github repository. If you prefer not handling it as a file, the client_email and private_key can be made into environment variables and managed separately.

await doc.useServiceAccountAuth({
  client_email: process.env.GOOGLE_SERVICE_ACCOUNT_EMAIL,
  private_key: process.env.GOOGLE_PRIVATE_KEY,
});

Next, open the created spreadsheet in the browser and share the document with the service account. Then, the google-spreadsheet can be used to manipulate the spreadsheet.

Automatically Syncing Translation Files

The setup is now complete. Run npm run upload:i18n to upload all keys from the source code to the spreadsheet in the following format.

The translator can now just look at the table and fill in the empty cells.

(While i18next supports singularization and pluralization, the pluralizations are different for different languages. Therefore, same key value from the source code have postfixes like "key_0", "key_1", and "key_plural" in language specific json files. Since different languages render the key differently, I have labelled "_N/A" to indicate where pluralization is not necessary.)

Simple Overview of Automatic Internationalization Process

• Developer: Run npm run upload:i18n then request translation.
• Translator: Enter translations in the spreadsheet

This is it. Every time you build, the npm run download:i18n will be executed and the most recent translation will be applied to the build, so you don't have to worry about anything. Add the script to the appropriate npm script for any situation whether you're running it for local server, production build, testing, or storybook.

{
  "scripts": {
    "build": "npm run download:i18n && webpack"
  }
}

Personal Tip (Conditional Formatting)

Set the background color for the cells that require translation. Then, it makes it easier for the translator to distinguish which cells need translations. Select the language column (Korean, English, Japanese, Chinese) in Google spreadsheet, and add the conditional formatting rules in Menu > Format > Conditional formatting.

• For empty cells: set a background color to indicate that the cell needs to be translated.
• For “_N/A”: dim the background and text color to indicate that the cell does not need to be translated.

It may take you some time to set everything up in the beginning, but once you have it set up, all it takes is running the single npm script. I hope this helps.

This is it. For real.