@lobehub/i18n-cli

<div align="center"><a name="readme-top"></a> <img height="120" src="https://gw.alipayobjects.com/zos/kitchen/T6E4BDoMNb/lobe-cli.webp"> <img height="120" src="https://gw.alipayobjects.com/zos/kitchen/qJ3l3EPsdW/split.svg"> <img height="120" src="https://registry.npmmirror.com/@lobehub/assets-emoji/1.3.0/files/assets/globe-showing-asia-australia.webp"> <h1>Lobe i18n</h1> Lobe i18n is a CLI workflow tool that uses ChatGPT for automated i18n. English ・ [简体中文](./README.zh-CN.md) ・ [Changelog](./CHANGELOG.md) · [Report Bug][issues-url] · [Request Feature][issues-url]  [![][npm-release-shield]][npm-release-link] [![][github-releasedate-shield]][github-releasedate-link] [![][github-action-test-shield]][github-action-test-link] [![][github-action-release-shield]][github-action-release-link]<br/> [![][github-contributors-shield]][github-contributors-link] [![][github-forks-shield]][github-forks-link] [![][github-stars-shield]][github-stars-link] [![][github-issues-shield]][github-issues-link] [![][github-license-shield]][github-license-link] ![](https://gw.alipayobjects.com/zos/kitchen/AH7rvv06qn/preview-i18n.webp) </div> <details> <summary><kbd>Table of Contents</kbd></summary> #### TOC - [✨ Features](#-features) - [📦 Installation](#-installation) - [🤯 Usage](#-usage) - [Step 1: Initialize Configuration](#step-1-initialize-configuration) - [Step 2: Run Translation Commands](#step-2-run-translation-commands) - [Configuration](#configuration) - [Environment Variable](#environment-variable) - [🌏 Locale Configuration](#-locale-configuration) - [File Structure Selection](#file-structure-selection) - [Running](#running) - [📝 Markdown Configuration](#-markdown-configuration) - [File Structure](#file-structure) - [Running](#running-1) - [⌨️ Local Development](#️-local-development) - [🤝 Contributing](#-contributing) - [🔗 Links](#-links) - [More Products](#more-products) - [Credits](#credits) #### </details> ## ✨ Features - [x] 🤖 Utilize ChatGPT for automated i18n translation - [x] ✂️ Support automatic splitting of large files without worrying about ChatGPT token limits. - [x] ♻️ Support incremental i18n updates, automatically extract new content based on `entry` files. - [x] 🗂️ Support single file mode `en_US.json` and folder mode `en_US/common.json` to work perfectly with `i18next`. - [x] 🌲 Support `flat` and `tree` structure for locale files. - [x] 🛠️ Support customizing OpenAI models, API proxies, temperature and topP. - [x] 📝 Support automated i18n translation of `Markdown` files. - [x] 🔍 Support linting translation files for language correctness using eld. <div align="right"> [![][back-to-top]](#readme-top) </div> ## 📦 Installation To install Lobe i18n, run the following command: ```bash npm install -g @lobehub/i18n-cli ``` > \[!IMPORTANT]\ > Please make sure you have `Node.js` version **>= 18**. <div align="right"> [![][back-to-top]](#readme-top) </div> ## 🤯 Usage ### Step 1: Initialize Configuration To initialize the Lobe i18n configuration, run the following command: ```shell $ lobe-i18n -o # or use the full flag --option ``` ### Step 2: Run Translation Commands This command will guide you through creating a configuration file step by step. Alternatively, you can manually create a configuration file in one of the formats supported by [cosmiconfig](https://github.com/cosmiconfig/cosmiconfig) (see [Configuration](#configuration) below). Here's an example of a configuration file (`.i18nrc.js`):\ This example demonstrates how to generate `locales/zh_CN.json` and `locales/ja_JP.json` automatically based on `locales/en_US.json`. ```javascript const { defineConfig } = require('@lobehub/i18n-cli'); module.exports = defineConfig({ entry: 'locales/en_US.json', entryLocale: 'en_US', output: 'locales', outputLocales: ['zh_CN', 'ja_JP'], }); ``` > \[!IMPORTANT]\ > To use AI auto-generation, you need to fill in the [OpenAI Token](https://platform.openai.com/account/api-keys) in the settings. ```shell # Translate Locale files $ lobe-i18n # or $ lobe-i18n locale # Translate Markdown files $ lobe-i18n md # Lint translation files for language correctness $ lobe-i18n lint # or $ lobe-i18n --lint # Run i18n translation and markdown translation simultaneously $ lobe-i18n --with-md # Specify the configuration file $ lobe-i18n -c './custom-config.js' # or use the full flag --config ``` <br/> ### Configuration You can choose any configuration method in [cosmiconfig](https://github.com/cosmiconfig/cosmiconfig) format - `i18n` property in `package.json` - `.i18nrc` file in JSON or YAML format - `.i18nrc.json`, `.i18nrc.yaml`, `.i18nrc.yml`, `.i18nrc.js`, `.i18nrc.cjs` - `defineConfig` provides a secure definition method that can be imported from `@lobehub/i18n-cli` > \[!TIP] > > This project provides a secure definition method `defineConfig` that can be imported from `@lobehub/i18n-cli` <br/> ### Environment Variable This project provides some additional configuration items set with environment variables: | Environment Variable | Required | Description | Example | | -------------------- | -------- | --------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------- | | `OPENAI_API_KEY` | Yes | This is the API key you apply on the OpenAI account page | `sk-xxxxxx...xxxxxx` | | `OPENAI_PROXY_URL` | No | If you manually configure the OpenAI interface proxy, you can use this configuration item to override the default OpenAI API request base URL | `https://api.chatanywhere.cn/v1`<br/>The default value is<br/>`https://api.openai.com/v1` | <div align="right"> [![][back-to-top]](#readme-top) </div> ## 🌏 Locale Configuration | Property Name | Required | Type | Default Value | Description | | --------------- | -------- | -------------- | --------------- | ---------------------------------------------------------------------------- | | entry | `*` | `string` | - | Entry file or folder | | entryLocale | `*` | `string` | - | Language to use as translation reference | | modelName | | `string` | `gpt-3.5-turbo` | Model to use | | output | `*` | `string` | - | Location to store localized files | | outputLocales | `*` | `string[] ` | `[]` | All the languages to be translated | | reference | | `string` | - | Provide some context for more accurate translations | | saveImmediately | | `boolean` | `false` | Save translation results immediately after each chunk is completed | | splitToken | | `number` | - | Split the localized JSON file by tokens, automatically calculated by default | | temperature | | `number` | `0` | Sampling temperature to use | | topP | | `number` | `1` | Nucleus sampling threshold, controls the diversity of generated text | | concurrency | | `number` | `5` | Number of concurrently pending promises returned | | experimental | | `experimental` | `{}` | Experimental features, see below | | markdown | | `markdown` | `{}` | See `markdown` configuration below | #### `experimental` | Property Name | Required | Type | Default Value | Description | | ------------- | -------- | --------- | ------------- | --------------------------------------------------------------------------------------------- | | jsonMode | | `boolean` | `false` | Enable gpt force JSON output for stability (only supported by new models after November 2023) | <br/> #### Example 1 `.i18nrc.js` ```js const { defineConfig } = require('@lobehub/i18n-cli'); module.exports = defineConfig({ entry: 'locales/en_US.json', entryLocale: 'en_US', output: 'locales', outputLocales: ['zh_CN', 'ja_JP'], }); ``` #### Example 2 `.i18nrc.json` ```json { "entry": "locales/en_US.json", "entryLocale": "en_US", "output": "locales", "outputLocales": ["zh_CN", "ja_JP"] } ``` #### Example 3 `package.json` ```json { "...": "...", "i18n": { "entry": "locales/en_US.json", "entryLocale": "en_US", "output": "locales", "outputLocales": ["zh_CN", "ja_JP"] } } ``` #### Example 4 - Enable Immediate Save ```js const { defineConfig } = require('@lobehub/i18n-cli'); module.exports = defineConfig({ entry: 'locales/en_US.json', entryLocale: 'en_US', output: 'locales', outputLocales: ['zh_CN', 'ja_JP'], saveImmediately: true, // Save results immediately after each chunk completes concurrency: 5, temperature: 0.3, }); ``` > \[!TIP] > > Enabling `saveImmediately` allows you to see translation progress in real-time and provides better fault tolerance. If the translation process is interrupted, completed chunks won't be lost. <div align="right"> [![][back-to-top]](#readme-top) </div> ### File Structure Selection There are two types of file structures supported: `flat` and `tree`. #### Flat Structure A flat structure means that all translations for different languages are stored in a single file, as shown below: ``` - locales - en_US.json - ja_JP.json - zh_CN.json - ... ``` > \[!TIP] > > The `flat structure` requires configuring the `entry` property in the configuration file to the corresponding JSON file [Example](./examples/locale/flat/.i18nrc.cjs) ```json { "entry": "locales/en.json", "entryLocale": "en_US", "output": "locales", "outputLocales": ["zh_CN", "ja_JP"] } ``` #### Tree Structure A tree structure means that the translations for each language are stored in separate language folders, as shown below: ``` - locales - en_US - common.json - header.json - subfolder - ... - ja_JP - common.json - header.json - subfolder - ... - zh_CN - common.json - header.json - subfolder - ... ``` > \[!TIP] > > The `tree structure` requires configuring the `entry` property in the configuration file to the corresponding folder [Example](./examples/locale/tree/.i18nrc.cjs) ```json { "entry": "locales/en_US", "entryLocale": "en_US", "output": "locales", "outputLocales": ["zh_CN", "ja_JP"] } ``` ### Running Use the `lobe-i18n` command to generate i18n files automatically: ```shell $ lobe-i18n ``` <div align="right"> [![][back-to-top]](#readme-top) </div> ## 📝 Markdown Configuration | Property Name | Required | Type | Default | Description | | ---------------- | -------- | --------------------------- | ---------------------------- | ----------------------------------------------------------------------- | | entry | `*` | `string[]` | `[]` | Entry file or folder, supports `glob` | | entryLocale | | `string` | _Inherit parent locale_ | Reference language for translation | | entryExtension | | `string` | `.md` | Entry file extension | | exclude | | `string[]` | `[]` | Files to be filtered, supports `glob` | | outputLocales | | `string[]` | _Inherit parent locale_ | All languages to be translated | | outputExtensions | | `function` | `(locale) => '.{locale}.md'` | Output file extension generation | | mode | | `string`,`mdast`,`function` | `string` | Translation mode selection, explained below | | translateCode | | `boolean` | `false` | Whether to translate code blocks under `mdast`, other modes are invalid | | includeMatter | | `boolean` | `false` | Whether to include front matter in the translation | #### `outputExtensions` By default, the translated file names are generated as `.{locale}.md`. You can customize the output file extensions with `outputExtensions`. > \[!NOTE] > > In the example below, the entry file extension is `.zh-CN.md`, but we want the output file extension for the `en-US` translation to be `.md`, while other languages keep the default extensions. ```js module.exports = { markdown: { entry: ['./README.zh-CN.md', './docs/**/*.zh-CN.md'], entryLocale: 'zh-CN', entryExtension: '.zh-CN.md', outputLocales: ['en-US', 'ja-JP'], outputExtensions: (locale, { getDefaultExtension }) => { if (locale === 'en-US') return '.md'; return getDefaultExtension(locale); }, }, }; ``` > `outputExtensions` supports the following `props`: ```ts interface OutputExtensionsProps { /** * @description The locale of the translated file to output */ locale: string; config: { /** * @description The content of the translated file to input */ fileContent: string; /** * @description The path of the translated file to input */ filePath: string; /** * @description The default method for generating extensions */ getDefaultExtension: (locale: string) => string; }; } ``` #### `mode` `mode` is used to specify the translation mode, which supports two modes and custom generation modes. - `string` - Translates the complete `markdown` content. - `mdast` - Parses the text with `mdast` structure and translates the `text value` content. To translate code blocks, you need to enable `translateCode`. > \[!WARNING] > > In `mdast` mode, the content to be translated will be reduced to a minimum, removing most markdown syntax structures and links. > This mode can greatly reduce token consumption, but it may result in inaccurate translation results. <div align="right"> [![][back-to-top]](#readme-top) </div> ### File Structure The translated files will be generated in the same directory as the entry file, with the corresponding language suffix added to the extension: ``` - README.md - README.zh-CN.md - docs - usage.md - usage.zh-CN.md - subfolder - ... ``` > \[!TIP] > > [Example](./examples/markdown/.i18nrc.cjs) ### Running Use the `lobe-i18n md` command to automate the generation of i18n files: ```shell $ lobe-i18n md ``` <div align="right"> [![][back-to-top]](#readme-top) </div> ## ⌨️ Local Development You can use Github Codespaces for online development: [![][github-codespace-shield]][github-codespace-link] Alternatively, you can clone the repository and run the following command for local development: [![][bun-shield]][bun-link] ```bash $ git clone https://github.com/lobehub/lobe-cli-toolbox.git $ cd lobe-cli-toolbox $ bun install $ cd packages/lobe-i18n $ bun dev ``` <div align="right"> [![][back-to-top]](#readme-top) </div> ## 🤝 Contributing We welcome contributions in all forms. If you're interested in contributing code, you can check our GitHub [Issues][github-issues-link], show off your skills, and demonstrate your ideas. [![][pr-welcome-shield]][pr-welcome-link] [![][github-contrib-shield]][github-contrib-link] <div align="right"> [![][back-to-top]](#readme-top) </div> ## 🔗 Links ### More Products - **[🤖 Lobe Chat][lobe-chat]** - An open-source, extensible (Function Calling), high-performance chatbot framework. It supports one-click free deployment of your private ChatGPT/LLM web application. - **[🤯 Lobe Theme][lobe-theme]** - The modern theme for stable diffusion webui, exquisite interface design, highly customizable UI, and efficiency boosting features. ### Credits - **OpenAI** - <https://github.com/openai/openai-node> - **ink** - <https://github.com/vadimdemedes/ink> - **transmart** - <https://github.com/Quilljou/transmart> <div align="right"> [![][back-to-top]](#readme-top) </div> --- #### 📝 License Copyright © 2023 [LobeHub][profile-link]. <br /> This project is licensed under the [MIT](./LICENSE) license.  [back-to-top]: https://img.shields.io/badge/-BACK_TO_TOP-151515?style=flat-square [bun-link]: https://bun.sh [bun-shield]: https://img.shields.io/badge/-speedup%20with%20bun-black?logo=bun&style=for-the-badge [github-action-release-link]: https://github.com/lobehub/lobe-cli-toolbox/actions/workflows/release.yml [github-action-release-shield]: https://img.shields.io/github/actions/workflow/status/lobehub/lobe-cli-toolbox/release.yml?label=release&labelColor=black&logo=githubactions&logoColor=white&style=flat-square [github-action-test-link]: https://github.com/lobehub/lobe-cli-toolbox/actions/workflows/test.yml [github-action-test-shield]: https://img.shields.io/github/actions/workflow/status/lobehub/lobe-cli-toolbox/test.yml?label=test&labelColor=black&logo=githubactions&logoColor=white&style=flat-square [github-codespace-link]: https://codespaces.new/lobehub/lobe-cli-toolbox [github-codespace-shield]: https://github.com/codespaces/badge.svg [github-contrib-link]: https://github.com/lobehub/lobe-cli-toolbox/graphs/contributors [github-contrib-shield]: https://contrib.rocks/image?repo=lobehub%2Flobe-cli-toolbox [github-contributors-link]: https://github.com/lobehub/lobe-cli-toolbox/graphs/contributors [github-contributors-shield]: https://img.shields.io/github/contributors/lobehub/lobe-cli-toolbox?color=c4f042&labelColor=black&style=flat-square [github-forks-link]: https://github.com/lobehub/lobe-cli-toolbox/network/members [github-forks-shield]: https://img.shields.io/github/forks/lobehub/lobe-cli-toolbox?color=8ae8ff&labelColor=black&style=flat-square [github-issues-link]: https://github.com/lobehub/lobe-cli-toolbox/issues [github-issues-shield]: https://img.shields.io/github/issues/lobehub/lobe-cli-toolbox?color=ff80eb&labelColor=black&style=flat-square [github-license-link]: https://github.com/lobehub/lobe-cli-toolbox/blob/master/LICENSE [github-license-shield]: https://img.shields.io/github/license/lobehub/lobe-cli-toolbox?color=white&labelColor=black&style=flat-square [github-releasedate-link]: https://github.com/lobehub/lobe-cli-toolbox/releases [github-releasedate-shield]: https://img.shields.io/github/release-date/lobehub/lobe-cli-toolbox?labelColor=black&style=flat-square [github-stars-link]: https://github.com/lobehub/lobe-cli-toolbox/network/stargazers [github-stars-shield]: https://img.shields.io/github/stars/lobehub/lobe-cli-toolbox?color=ffcb47&labelColor=black&style=flat-square [issues-url]: https://github.com/canisminor1990/lobe-commit/issues/new/choose [lobe-chat]: https://github.com/lobehub/lobe-chat [lobe-theme]: https://github.com/lobehub/sd-webui-lobe-theme [npm-release-link]: https://www.npmjs.com/package/@lobehub/i18n-cli [npm-release-shield]: https://img.shields.io/npm/v/@lobehub/i18n-cli?color=369eff&labelColor=black&logo=npm&logoColor=white&style=flat-square [pr-welcome-link]: https://github.com/lobehub/lobe-cli-toolbox/pulls [pr-welcome-shield]: https://img.shields.io/badge/%F0%9F%A4%AF%20PR%20WELCOME-%E2%86%92-ffcb47?labelColor=black&style=for-the-badge [profile-link]: https://github.com/lobehub