encoding-japanese
Version:
Convert and detect character encodings in JavaScript
868 lines (630 loc) • 32.3 kB
Markdown
encoding.js
===========
[](https://www.npmjs.com/package/encoding-japanese)
[](https://github.com/polygonplanet/encoding.js/actions)
[](https://github.com/polygonplanet/encoding.js/blob/master/LICENSE)
Convert and detect character encodings in JavaScript.
[**README (日本語)**](README_ja.md)
## Table of contents
- [Features](#features)
* [How to Handle Encoded Data as Strings?](#how-to-handle-encoded-data-as-strings)
- [Installation](#installation)
* [npm](#npm)
+ [TypeScript](#typescript)
* [Browser (standalone)](#browser-standalone)
* [CDN](#cdn)
- [Supported encodings](#supported-encodings)
* [About `SJIS`](#about-sjis)
* [About `UNICODE`](#about-unicode)
- [Example usage](#example-usage)
- [Demo](#demo)
* [Playground](#playground)
* [Sample file encoding conversion demo](#sample-file-encoding-conversion-demo)
* [User-selected file encoding conversion and detection demo](#user-selected-file-encoding-conversion-and-detection-demo)
- [API](#api)
* [`detect` : Detects character encoding](#encodingdetect-data-encodings)
* [`convert` : Converts character encoding](#encodingconvert-data-to-from)
+ [Specifying conversion options with an object](#specifying-conversion-options-with-an-object)
+ [Specify the return type by the `type` option](#specify-the-return-type-by-the-type-option)
+ [Specify handling for unrepresentable characters](#specify-handling-for-unrepresentable-characters)
+ [Replacing characters with HTML entities when they cannot be represented](#replacing-characters-with-html-entities-when-they-cannot-be-represented)
+ [Ignoring characters when they cannot be represented](#ignoring-characters-when-they-cannot-be-represented)
+ [Throwing an Error when they cannot be represented](#throwing-an-error-when-they-cannot-be-represented)
+ [Specify BOM in UTF-16](#specify-bom-in-utf-16)
* [`urlEncode` : Encodes to percent-encoded string](#encodingurlencode-data)
* [`urlDecode` : Decodes from percent-encoded string](#encodingurldecode-string)
* [`base64Encode` : Encodes to Base64 formatted string](#encodingbase64encode-data)
* [`base64Decode` : Decodes from Base64 formatted string](#encodingbase64decode-string)
* [`codeToString` : Converts character code array to string](#encodingcodetostring-code)
* [`stringToCode` : Converts string to character code array](#encodingstringtocode-string)
* [Japanese Zenkaku/Hankaku conversion](#japanese-zenkakuhankaku-conversion)
- [Other examples](#other-examples)
* [Example using the `Fetch API` and Typed Arrays (Uint8Array)](#example-using-the-fetch-api-and-typed-arrays-uint8array)
* [Convert encoding for file using the File APIs](#convert-encoding-for-file-using-the-file-apis)
- [Contributing](#contributing)
- [License](#license)
## Features
encoding.js is a JavaScript library for converting and detecting character encodings,
supporting both Japanese character encodings (`Shift_JIS`, `EUC-JP`, `ISO-2022-JP`) and Unicode formats (`UTF-8`, `UTF-16`).
Since JavaScript string values are internally encoded as UTF-16 code units
([ref: ECMAScript® 2019 Language Specification - 6.1.4 The String Type](https://www.ecma-international.org/ecma-262/10.0/index.html#sec-ecmascript-language-types-string-type)),
data in other encodings cannot be handled directly as JavaScript strings.
encoding.js works around this limitation by representing encoded data as arrays of numeric code values instead of strings, enabling conversion between different character encodings.
For example, `[130, 160]` represents "あ" in Shift_JIS.
These character code arrays can also be passed as typed arrays (such as `Uint8Array`) or as Node.js `Buffer` objects.
### How to Handle Encoded Data as Strings?
Numeric arrays of character codes can be converted to strings using methods such as [`Encoding.codeToString`](#encodingcodetostring-code).
However, due to the JavaScript specifications mentioned above, some character encodings may not be handled properly when converted directly to strings.
If you need a string representation, [`Encoding.urlEncode`](#encodingurlencode-data) and
[`Encoding.urlDecode`](#encodingurldecode-string) can convert numeric arrays to and from
percent-encoded strings such as `'%82%A0'`.
Similarly, [`Encoding.base64Encode`](#encodingbase64encode-data) and
[`Encoding.base64Decode`](#encodingbase64decode-string) can convert numeric arrays to and from Base64 strings.
## Installation
### npm
encoding.js is published under the package name `encoding-japanese` on npm.
```bash
npm install --save encoding-japanese
```
#### Using ES6 `import`
```javascript
import Encoding from 'encoding-japanese';
```
#### Using CommonJS `require`
```javascript
const Encoding = require('encoding-japanese');
```
#### TypeScript
TypeScript type definitions for encoding.js are available at [@types/encoding-japanese](https://www.npmjs.com/package/@types/encoding-japanese) (thanks to [@rhysd](https://github.com/rhysd)).
```bash
npm install --save-dev @types/encoding-japanese
```
### Browser (standalone)
To use encoding.js in a browser environment, you can either install it via npm or download it directly from the [release list](https://github.com/polygonplanet/encoding.js/tags).
The package includes both `encoding.js` and `encoding.min.js`.
Note: Cloning the repository via `git clone` might give you access to the *master* (or *main*) branch, which could still be in a development state.
```html
<!-- To include the full version -->
<script src="encoding.js"></script>
<!-- Or, to include the minified version for production -->
<script src="encoding.min.js"></script>
```
When the script is loaded, the object `Encoding` is defined in the global scope (i.e., `window.Encoding`).
### CDN
You can use encoding.js (package name: `encoding-japanese`) directly from a CDN via a script tag:
```html
<script src="https://unpkg.com/encoding-japanese@2.4.0/encoding.min.js"></script>
```
In this example we use [unpkg](https://unpkg.com/encoding-japanese/), but you can use any CDN that provides npm packages,
for example [cdnjs](https://cdnjs.com/libraries/encoding-japanese) or [jsDelivr](https://www.jsdelivr.com/package/npm/encoding-japanese).
## Supported encodings
|Value in encoding.js|[`detect()`](#encodingdetect-data-encodings)|[`convert()`](#encodingconvert-data-to-from)|MIME Name (Note)|
|:------:|:----:|:-----:|:---|
|ASCII |✓ | |US-ASCII (Code point range: `0-127`)|
|BINARY |✓ | |(Binary string. Code point range: `0-255`)|
|EUCJP |✓ |✓ |EUC-JP|
|JIS |✓ |✓ |ISO-2022-JP|
|SJIS |✓ |✓ |Shift_JIS, Windows-31J (*See [About `SJIS`](#about-sjis) below)|
|UTF8 |✓ |✓ |UTF-8|
|UTF16 |✓ |✓ |UTF-16|
|UTF16BE |✓ |✓ |UTF-16BE (big-endian)|
|UTF16LE |✓ |✓ |UTF-16LE (little-endian)|
|UTF32 |✓ | |UTF-32|
|UNICODE |✓ |✓ |(JavaScript string. *See [About `UNICODE`](#about-unicode) below) |
### About `SJIS`
Strictly speaking, Shift_JIS and CP932 (also known as Windows-31J, MS932, SJIS-win, and other names) are different character encodings.
Following common usage, encoding.js defines encodings in the Shift_JIS family as `SJIS`.
`SJIS` covers CP932, an extension of Shift_JIS.
In addition to JIS X 0201 and JIS X 0208, it supports the following CP932 extension areas:
|Area |Code range |[`detect()`](#encodingdetect-data-encodings)|[`convert()`](#encodingconvert-data-to-from)|
|:-----------------------------------|:------------------|:----:|:----:|
|NEC special characters |`0x8740` - `0x879C`|✓ |✓ |
|NEC-selected IBM extended characters|`0xED40` - `0xEEFC`|✓ |✓ |
|User-defined area (外字 / Gaiji) |`0xF040` - `0xF9FC`|✓ | |
|IBM extended characters |`0xFA40` - `0xFC4B`|✓ |✓ |
The user-defined area is detected as `SJIS`, but it cannot be converted because its character assignments are not defined by any standard.
During conversion, characters in this area are replaced with `?` (U+003F).
### About `UNICODE`
In encoding.js, `UNICODE` represents JavaScript strings (e.g., `"hello"` or `"こんにちは"`)
and is defined as the internal character representation used by JavaScript.
As mentioned in the [Features](#features) section, JavaScript strings are internally encoded using UTF-16 code units.
This means other character encodings cannot be directly handled without conversion.
Therefore, specify `UNICODE` when converting data to a form that can be properly represented as a JavaScript string.
(Note: Even if your HTML file is UTF-8 encoded, you should specify `UNICODE` instead of `UTF8` when handling data as JavaScript strings.)
When using [`Encoding.convert`](#encodingconvert-data-to-from),
if you specify a character encoding other than `UNICODE` (such as `UTF8` or `SJIS`),
the values in the returned character code array will range from `0` to `255` (byte array).
However, if you specify `UNICODE`, the values will range from `0` to `65535`,
which corresponds to the range of values returned by `String.prototype.charCodeAt()` (UTF-16 code units).
## Example usage
Convert character encoding from JavaScript string (`UNICODE`) to `SJIS`.
```javascript
const unicodeArray = Encoding.stringToCode('こんにちは'); // Convert string to code array
const sjisArray = Encoding.convert(unicodeArray, {
to: 'SJIS',
from: 'UNICODE'
});
console.log(sjisArray);
// [130, 177, 130, 241, 130, 201, 130, 191, 130, 205] ('こんにちは' array in SJIS)
```
Convert character encoding from `SJIS` to `UNICODE`.
```javascript
const sjisArray = [
130, 177, 130, 241, 130, 201, 130, 191, 130, 205
]; // 'こんにちは' array in SJIS
const unicodeArray = Encoding.convert(sjisArray, {
to: 'UNICODE',
from: 'SJIS'
});
const str = Encoding.codeToString(unicodeArray); // Convert code array to string
console.log(str); // 'こんにちは'
```
Detect character encoding.
```javascript
const data = [
227, 129, 147, 227, 130, 147, 227, 129, 171, 227, 129, 161, 227, 129, 175
]; // 'こんにちは' array in UTF-8
const detectedEncoding = Encoding.detect(data);
console.log(`Character encoding is ${detectedEncoding}`); // 'Character encoding is UTF8'
```
(Node.js) Example of reading a text file written in `SJIS`.
```javascript
const fs = require('fs');
const Encoding = require('encoding-japanese');
const sjisBuffer = fs.readFileSync('./sjis.txt');
const unicodeArray = Encoding.convert(sjisBuffer, {
to: 'UNICODE',
from: 'SJIS'
});
console.log(Encoding.codeToString(unicodeArray));
```
## Demo
### Playground
Try character encoding conversion and detection directly in your browser.
[Open the Playground](https://polygonplanet.github.io/encoding.js/tests/playground.html)
### Sample file encoding conversion demo
[Try converting sample files encoded in Shift_JIS, EUC-JP, and other encodings](https://polygonplanet.github.io/encoding.js/tests/encoding-test.html)
### User-selected file encoding conversion and detection demo
[Try converting and detecting the character encoding of a selected file](https://polygonplanet.github.io/encoding.js/tests/detect-file-encoding.html)
----
## API
* [detect](#encodingdetect-data-encodings)
* [convert](#encodingconvert-data-to-from)
* [urlEncode](#encodingurlencode-data)
* [urlDecode](#encodingurldecode-string)
* [base64Encode](#encodingbase64encode-data)
* [base64Decode](#encodingbase64decode-string)
* [codeToString](#encodingcodetostring-code)
* [stringToCode](#encodingstringtocode-string)
* [Japanese Zenkaku/Hankaku conversion](#japanese-zenkakuhankaku-conversion)
----
### Encoding.detect (data, [encodings])
Detects the character encoding of the given data.
#### Parameters
* **data** *(Array\<number\>|TypedArray|Buffer|string)* : The code array or string to detect character encoding.
* **\[encodings\]** *(string|Array\<string\>|Object)* : (Optional) Specifies a specific character encoding,
or an array of encodings to limit the detection. Detects automatically if this argument is omitted or `AUTO` is specified.
Supported encoding values can be found in the "[Supported encodings](#supported-encodings)" section.
#### Return value
*(string|boolean)*: Returns a string representing the detected encoding (e.g., `SJIS`, `UTF8`) listed in the "[Supported encodings](#supported-encodings)" section, or `false` if the encoding cannot be detected.
If the `encodings` argument is provided, it returns the name of the detected encoding if the `data` matches any of the specified encodings, or `false` otherwise.
#### Examples
Example of detecting character encoding.
```javascript
const sjisArray = [130, 168, 130, 205, 130, 230]; // 'おはよ' array in SJIS
const detectedEncoding = Encoding.detect(sjisArray);
console.log(`Encoding is ${detectedEncoding}`); // 'Encoding is SJIS'
```
Example of using the `encodings` argument to specify the character encoding to be detected.
This returns the detected encoding name if the specified encoding matches, or `false` otherwise:
```javascript
const sjisArray = [130, 168, 130, 205, 130, 230]; // 'おはよ' array in SJIS
const detectedEncoding = Encoding.detect(sjisArray, 'SJIS');
if (detectedEncoding) {
console.log('Encoding is SJIS');
} else {
console.log('Encoding does not match SJIS');
}
```
Example of specifying multiple encodings:
```javascript
const sjisArray = [130, 168, 130, 205, 130, 230]; // 'おはよ' array in SJIS
const detectedEncoding = Encoding.detect(sjisArray, ['UTF8', 'SJIS']);
if (detectedEncoding) {
console.log(`Encoding is ${detectedEncoding}`); // 'Encoding is SJIS'
} else {
console.log('Encoding does not match UTF8 and SJIS');
}
```
----
### Encoding.convert (data, to[, from])
Converts the character encoding of the given data.
#### Parameters
* **data** *(Array\<number\>|TypedArray|Buffer|string)* : The code array or string to be converted.
* **to** *(string|Object)* : The character encoding name of the conversion destination as a string, or conversion options as an object.
* **\[from\]** *(string|Array\<string\>)* : (Optional) The character encoding name of the conversion source as a string,
or an array of encoding names. Detects automatically if this argument is omitted or `AUTO` is specified.
Supported encoding values can be found in the "[Supported encodings](#supported-encodings)" section.
#### Return value
*(Array\<number\>|TypedArray|string)* : Returns a numeric character code array of the converted character encoding if `data` is an array or a buffer,
or returns the converted string if `data` is a string.
#### Examples
Example of converting a character code array to Shift_JIS from UTF-8:
```javascript
const utf8Array = [227, 129, 130]; // 'あ' in UTF-8
const sjisArray = Encoding.convert(utf8Array, 'SJIS', 'UTF8');
console.log(sjisArray); // [130, 160] ('あ' in SJIS)
```
Typed arrays, such as `Uint8Array`, and Node.js `Buffer` objects can be passed in the same way:
```javascript
const utf8Array = new Uint8Array([227, 129, 130]);
const sjisArray = Encoding.convert(utf8Array, 'SJIS', 'UTF8');
```
Converts character encoding by auto-detecting the encoding name of the source:
```javascript
// The character encoding is automatically detected when the argument `from` is omitted
const utf8Array = [227, 129, 130];
let sjisArray = Encoding.convert(utf8Array, 'SJIS');
// You can also explicitly specify 'AUTO' to enable auto-detection
sjisArray = Encoding.convert(utf8Array, 'SJIS', 'AUTO');
```
#### Specifying conversion options with an object
For readability, the second argument `to` can be passed as an options object.
Options such as `type`, `fallback`, and `bom` must be specified this way.
```javascript
const utf8Array = [227, 129, 130];
const sjisArray = Encoding.convert(utf8Array, {
to: 'SJIS',
from: 'UTF8'
});
```
#### Specify the return type by the `type` option
`convert` returns an array by default, but you can change the return type by specifying the `type` option.
If `data` is a string and `type` is omitted, `type: 'string'` is assumed and the result is returned as a string.
```javascript
const sjisArray = [130, 168, 130, 205, 130, 230]; // 'おはよ' array in SJIS
const unicodeString = Encoding.convert(sjisArray, {
to: 'UNICODE',
from: 'SJIS',
type: 'string' // Specify 'string' to return as string
});
console.log(unicodeString); // 'おはよ'
```
The following `type` options are supported.
* **string** : Return as a string.
* **arraybuffer** : Return a `Uint16Array` (the option name is retained for historical reasons).
* **array** : Return as an Array. (*default*)
`type: 'string'` can be used as a shorthand for converting a code array to a string,
as performed by [`Encoding.codeToString`](#encodingcodetostring-code).
Note: Specifying `type: 'string'` may not handle conversions properly, except when converting to `UNICODE`.
#### Specify handling for unrepresentable characters
With the `fallback` option, you can specify how to handle characters that cannot be represented in the target encoding.
The `fallback` option supports the following values:
* **html-entity**: Replace characters with HTML entities (decimal HTML numeric character references).
* **html-entity-hex**: Replace characters with HTML entities (hexadecimal HTML numeric character references).
* **ignore**: Ignore characters that cannot be represented.
* **error**: Throw an error if any character cannot be represented.
#### Replacing characters with HTML entities when they cannot be represented
Characters that cannot be represented in the target character set are replaced with '?' (U+003F) by default,
but by specifying `html-entity` as the `fallback` option, you can replace them with HTML entities (Numeric character references), such as `🍣`.
Example of specifying `{ fallback: 'html-entity' }` option:
```javascript
const unicodeArray = Encoding.stringToCode('寿司🍣ビール🍺');
// No fallback specified
let sjisArray = Encoding.convert(unicodeArray, {
to: 'SJIS',
from: 'UNICODE'
});
console.log(sjisArray); // Converted to a code array of '寿司?ビール?'
// Specify `fallback: html-entity`
sjisArray = Encoding.convert(unicodeArray, {
to: 'SJIS',
from: 'UNICODE',
fallback: 'html-entity'
});
console.log(sjisArray); // Converted to a code array of '寿司🍣ビール🍺'
```
Example of specifying `{ fallback: 'html-entity-hex' }` option:
```javascript
const unicodeArray = Encoding.stringToCode('ホッケの漢字は𩸽');
const sjisArray = Encoding.convert(unicodeArray, {
to: 'SJIS',
from: 'UNICODE',
fallback: 'html-entity-hex'
});
console.log(sjisArray); // Converted to a code array of 'ホッケの漢字は𩸽'
```
#### Ignoring characters when they cannot be represented
By specifying `ignore` as a `fallback` option, characters that cannot be represented in the target encoding format can be ignored.
Example of specifying `{ fallback: 'ignore' }` option:
```javascript
const unicodeArray = Encoding.stringToCode('寿司🍣ビール🍺');
// No fallback specified
let sjisArray = Encoding.convert(unicodeArray, {
to: 'SJIS',
from: 'UNICODE'
});
console.log(sjisArray); // Converted to a code array of '寿司?ビール?'
// Specify `fallback: ignore`
sjisArray = Encoding.convert(unicodeArray, {
to: 'SJIS',
from: 'UNICODE',
fallback: 'ignore'
});
console.log(sjisArray); // Converted to a code array of '寿司ビール'
```
#### Throwing an Error when they cannot be represented
If you need to throw an error when a character cannot be represented in the target character encoding,
specify `error` as a `fallback` option. This will cause an exception to be thrown.
Example of specifying `{ fallback: 'error' }` option:
```javascript
const unicodeArray = Encoding.stringToCode('おにぎり🍙ラーメン🍜');
try {
const sjisArray = Encoding.convert(unicodeArray, {
to: 'SJIS',
from: 'UNICODE',
fallback: 'error' // Specify 'error' to throw an exception
});
} catch (e) {
console.error(e); // Error: Character cannot be represented: [240, 159, 141, 153]
}
```
#### Specify BOM in UTF-16
You can add a BOM (byte order mark) by specifying the `bom` option when converting to `UTF16`.
The default is no BOM.
```javascript
const utf16Array = Encoding.convert(utf8Array, {
to: 'UTF16',
from: 'UTF8',
bom: true // Specify to add the BOM
});
```
The default byte order for `UTF16` is big-endian.
If you want to convert as little-endian, specify the `{ bom: 'LE' }` option.
```javascript
const utf16leArray = Encoding.convert(utf8Array, {
to: 'UTF16',
from: 'UTF8',
bom: 'LE' // Specify to add the BOM as little-endian
});
```
If you do not need BOM, use `UTF16BE` or `UTF16LE`.
`UTF16BE` is big-endian, and `UTF16LE` is little-endian, and both have no BOM.
```javascript
const utf16beArray = Encoding.convert(utf8Array, {
to: 'UTF16BE',
from: 'UTF8'
});
```
----
### Encoding.urlEncode (data)
Encodes a numeric character code array into a percent-encoded string formatted as a URI component in `%xx` format.
urlEncode escapes all characters except the following, just like [`encodeURIComponent()`](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/encodeURIComponent).
```
A-Z a-z 0-9 - _ . ! ~ * ' ( )
```
#### Parameters
* **data** *(Array\<number\>|TypedArray|Buffer|string)* : The numeric character code array or string that will be encoded into a percent-encoded URI component.
#### Return value
*(string)* : Returns a percent-encoded string formatted as a URI component in `%xx` format.
#### Examples
Example of URL encoding a Shift_JIS array:
```javascript
const sjisArray = [130, 168, 130, 205, 130, 230]; // 'おはよ' array in SJIS
const encoded = Encoding.urlEncode(sjisArray);
console.log(encoded); // '%82%A8%82%CD%82%E6'
```
----
### Encoding.urlDecode (string)
Decodes a percent-encoded string formatted as a URI component in `%xx` format to a numeric character code array.
#### Parameters
* **string** *(string)* : The string to decode.
#### Return value
*(Array\<number\>)* : Returns a numeric character code array.
#### Examples
Example of decoding a percent-encoded Shift_JIS string:
```javascript
const encoded = '%82%A8%82%CD%82%E6'; // 'おはよ' encoded as percent-encoded SJIS string
const sjisArray = Encoding.urlDecode(encoded);
console.log(sjisArray); // [130, 168, 130, 205, 130, 230]
```
----
### Encoding.base64Encode (data)
Encodes a numeric character code array into a Base64 encoded string.
#### Parameters
* **data** *(Array\<number\>|TypedArray|Buffer|string)* : The numeric character code array or string to encode.
#### Return value
*(string)* : Returns a Base64 encoded string.
#### Examples
Example of Base64 encoding a Shift_JIS array:
```javascript
const sjisArray = [130, 168, 130, 205, 130, 230]; // 'おはよ' array in SJIS
const encodedStr = Encoding.base64Encode(sjisArray);
console.log(encodedStr); // 'gqiCzYLm'
```
----
### Encoding.base64Decode (string)
Decodes a Base64 encoded string to a numeric character code array.
#### Parameters
* **string** *(string)* : The Base64 encoded string to decode.
#### Return value
*(Array\<number\>)* : Returns the decoded numeric character code array.
#### Examples
Example of `base64Encode` and `base64Decode`:
```javascript
const sjisArray = [130, 177, 130, 241, 130, 201, 130, 191, 130, 205]; // 'こんにちは' array in SJIS
const encodedStr = Encoding.base64Encode(sjisArray);
console.log(encodedStr); // 'grGC8YLJgr+CzQ=='
const decodedArray = Encoding.base64Decode(encodedStr);
console.log(decodedArray); // [130, 177, 130, 241, 130, 201, 130, 191, 130, 205]
```
----
### Encoding.codeToString (code)
Converts a numeric character code array to a string.
#### Parameters
* **code** *(Array\<number\>|TypedArray|Buffer)* : The numeric character code array to convert.
#### Return value
*(string)* : Returns a converted string.
#### Examples
Example of converting a character code array to a string:
```javascript
const sjisArray = [130, 168, 130, 205, 130, 230]; // 'おはよ' array in SJIS
const unicodeArray = Encoding.convert(sjisArray, {
to: 'UNICODE',
from: 'SJIS'
});
const unicodeStr = Encoding.codeToString(unicodeArray);
console.log(unicodeStr); // 'おはよ'
```
----
### Encoding.stringToCode (string)
Converts a string to a numeric character code array.
#### Parameters
* **string** *(string)* : The string to convert.
#### Return value
*(Array\<number\>)* : Returns a numeric character code array converted from the string.
#### Examples
Example of converting a string to a character code array:
```javascript
const unicodeArray = Encoding.stringToCode('おはよ');
console.log(unicodeArray); // [12362, 12399, 12424]
```
----
### Japanese Zenkaku/Hankaku conversion
The following methods convert Japanese full-width (zenkaku) and half-width (hankaku) characters.
They accept either `UNICODE` strings or numeric arrays of `UNICODE` code units.
Returns a converted string if the argument `data` is a string.
Returns a numeric character code array if the argument `data` is a code array.
- **Encoding.toHankakuCase (data)** : Converts full-width (zenkaku) symbols and alphanumeric characters to their half-width (hankaku) equivalents.
- **Encoding.toZenkakuCase (data)** : Converts half-width (hankaku) symbols and alphanumeric characters to their full-width (zenkaku) equivalents.
- **Encoding.toHiraganaCase (data)** : Converts full-width katakana to full-width hiragana.
- **Encoding.toKatakanaCase (data)** : Converts full-width hiragana to full-width katakana.
- **Encoding.toHankanaCase (data)** : Converts full-width katakana to half-width katakana.
- **Encoding.toZenkanaCase (data)** : Converts half-width katakana to full-width katakana.
- **Encoding.toHankakuSpace (data)** : Converts the ideographic space (U+3000) to an ASCII space (U+0020).
- **Encoding.toZenkakuSpace (data)** : Converts an ASCII space (U+0020) to the ideographic space (U+3000).
#### Parameters
- **data** *(Array\<number\>|TypedArray|Buffer|string)* : The string or numeric character code array to convert.
#### Return value
*(Array\<number\>|string)* : Returns a converted string or numeric character code array.
#### Examples
Example of converting zenkaku and hankaku strings:
```javascript
console.log(Encoding.toHankakuCase('ａｂｃＤＥＦ１２３＠！＃＊＝')); // 'abcDEF123@!#*='
console.log(Encoding.toZenkakuCase('abcDEF123@!#*=')); // 'ａｂｃＤＥＦ１２３＠！＃＊＝'
console.log(Encoding.toHiraganaCase('アイウエオァィゥェォヴボポ')); // 'あいうえおぁぃぅぇぉゔぼぽ'
console.log(Encoding.toKatakanaCase('あいうえおぁぃぅぇぉゔぼぽ')); // 'アイウエオァィゥェォヴボポ'
console.log(Encoding.toHankanaCase('アイウエオァィゥェォヴボポ')); // 'ｱｲｳｴｵｧｨｩｪｫｳﾞﾎﾞﾎﾟ'
console.log(Encoding.toZenkanaCase('ｱｲｳｴｵｧｨｩｪｫｳﾞﾎﾞﾎﾟ')); // 'アイウエオァィゥェォヴボポ'
console.log(Encoding.toHankakuSpace('あいうえお abc 123')); // 'あいうえお abc 123'
console.log(Encoding.toZenkakuSpace('あいうえお abc 123')); // 'あいうえお abc 123'
```
Example of converting zenkaku and hankaku code arrays:
```javascript
const unicodeArray = Encoding.stringToCode('ａｂｃ１２３！＃ あいうアイウ ABCｱｲｳ');
console.log(Encoding.codeToString(Encoding.toHankakuCase(unicodeArray)));
// 'abc123!# あいうアイウ ABCｱｲｳ'
console.log(Encoding.codeToString(Encoding.toZenkakuCase(unicodeArray)));
// 'ａｂｃ１２３！＃ あいうアイウ ＡＢＣｱｲｳ'
console.log(Encoding.codeToString(Encoding.toHiraganaCase(unicodeArray)));
// 'ａｂｃ１２３！＃ あいうあいう ABCｱｲｳ'
console.log(Encoding.codeToString(Encoding.toKatakanaCase(unicodeArray)));
// 'ａｂｃ１２３！＃ アイウアイウ ABCｱｲｳ'
console.log(Encoding.codeToString(Encoding.toHankanaCase(unicodeArray)));
// 'ａｂｃ１２３！＃ あいうｱｲｳ ABCｱｲｳ'
console.log(Encoding.codeToString(Encoding.toZenkanaCase(unicodeArray)));
// 'ａｂｃ１２３！＃ あいうアイウ ABCアイウ'
console.log(Encoding.codeToString(Encoding.toHankakuSpace(unicodeArray)));
// 'ａｂｃ１２３！＃ あいうアイウ ABCｱｲｳ'
console.log(Encoding.codeToString(Encoding.toZenkakuSpace(unicodeArray)));
// 'ａｂｃ１２３！＃ あいうアイウ ABCｱｲｳ'
```
----
## Other examples
### Example using the `Fetch API` and Typed Arrays (Uint8Array)
This example reads a text file encoded in Shift_JIS as binary data,
and displays it as a string after converting it to Unicode using [Encoding.convert](#encodingconvert-data-to-from).
```javascript
(async () => {
try {
const response = await fetch('shift_jis.txt');
const buffer = await response.arrayBuffer();
// Code array with Shift_JIS file contents
const sjisArray = new Uint8Array(buffer);
// Convert encoding to UNICODE (JavaScript Code Units) from Shift_JIS
const unicodeArray = Encoding.convert(sjisArray, {
to: 'UNICODE',
from: 'SJIS'
});
// Convert to string from code array for display
const unicodeString = Encoding.codeToString(unicodeArray);
console.log(unicodeString);
} catch (error) {
console.error('Error loading the file:', error);
}
})();
```
<details>
<summary>XMLHttpRequest version of this example</summary>
```javascript
const req = new XMLHttpRequest();
req.open('GET', 'shift_jis.txt', true);
req.responseType = 'arraybuffer';
req.onload = (event) => {
const buffer = req.response;
if (buffer) {
// Code array with Shift_JIS file contents
const sjisArray = new Uint8Array(buffer);
// Convert encoding to UNICODE (JavaScript Code Units) from Shift_JIS
const unicodeArray = Encoding.convert(sjisArray, {
to: 'UNICODE',
from: 'SJIS'
});
// Convert to string from code array for display
const unicodeString = Encoding.codeToString(unicodeArray);
console.log(unicodeString);
}
};
req.send(null);
```
</details>
### Convert encoding for file using the File APIs
This example uses the File API to read the content of a selected file, detects its character encoding,
and converts the file content to UNICODE from any character encoding such as `Shift_JIS` or `EUC-JP`.
The converted content is then displayed in a textarea.
```html
<input type="file" id="file">
<div id="encoding"></div>
<textarea id="content" rows="5" cols="80"></textarea>
<script>
function onFileSelect(event) {
const file = event.target.files[0];
const reader = new FileReader();
reader.onload = function(e) {
const codes = new Uint8Array(e.target.result);
const detectedEncoding = Encoding.detect(codes);
const encoding = document.getElementById('encoding');
encoding.textContent = `Detected encoding: ${detectedEncoding}`;
// Convert encoding to UNICODE
const unicodeString = Encoding.convert(codes, {
to: 'UNICODE',
from: detectedEncoding,
type: 'string'
});
document.getElementById('content').value = unicodeString;
};
reader.readAsArrayBuffer(file);
}
document.getElementById('file').addEventListener('change', onFileSelect);
</script>
```
[**Demo**](https://polygonplanet.github.io/encoding.js/tests/detect-file-encoding.html)
## Contributing
We welcome contributions from everyone.
For bug reports and feature requests, please [create an issue on GitHub](https://github.com/polygonplanet/encoding.js/issues).
### Pull requests
Before submitting a pull request, please run `npm run test` to ensure there are no errors.
We only accept pull requests that pass all tests.
## License
This project is licensed under the terms of the MIT license.
See the [LICENSE](LICENSE) file for details.