@giorgi-g/csv-parser
Version:
CSV parser for migrations with Sequelizer
90 lines (74 loc) • 3.42 kB
Markdown
## CSV PARSER ON STEROIDS
__CSV files parser on TypeScript with Sequelizer. For easier migration.__
___
### Install npm modules
```shell
npm install
```
### Then configure a database according to the [.env.example](.env.example) file
#### In [config.ts](config.ts) you will also see sample default values...
___
### We can define different options upon reading a file e.g:
```javascript
interface CSVParserOptions {
mapKeyIndexes?: number[];
mapKeySeparator?: string;
classPath?: string;
classObjectGetterName?: string;
delimiter?: string;
rootDir?: string;
rootEnv?: string;
fileExtension?: string;
mergeMapKeyValues?: boolean;
}
```
___
1. ***mapKeyIndexes***
* After reading a file a `Map<any|number, any>` is generated by default it has numeric index as a key, but we can assign a key generated by the properties.
* If we want records to have a custom key from [CSV](files/sample_profiles_dev01.csv) file e.g: **email** we can set `mapKeyIndexes: [3]` where **3** is the index in [CSV](files/sample_profiles_dev01.csv) for **email**.
* We can also have combined key e.g: `[0, 3]` which will give us `${ID}-${EMAIL}`
* **Note**: the keys are separated by the **Dash** symbol if you want to override the default you can change the value of `mapKeySeparator` key.
2. ***classPath*** **&&** ***classObjectGetterName***
* After parsing the data from CSV you can cast it to a certain **Class** which by default can be created inside [entities'](src/entities) folder. e.g: `{ classPath: '../entities/Profile' }`
* If you want value of the map to be some certain property or getter inside the Class you can provide a value for **classObjectGetterName** e.g: `{ classObjectGetterName: 'profile' }` which is the [profile](src/entities/Profile.ts) property inside the class
3. ***delimiter***
* As a delimiter your CSV file can contain **[`comma: ,`]** symbol if that is not the default for your CSV you can provide a different value defined inside.
4. ***mergeMapKeyValues***
* While setting a map to the key if you want to receive the same key values in an array make this value true
___
### Example:
```typescript
const fileName = "sample_profiles"; // file name inside /files/FILE_NAME.csv
const options: CSVParserOptions = {
classPath: '../entities/Profile', // Class name if you want to cast result into a class
classObjectGetterName: 'profile', // Name of the property inside the Class which will be the value inside the Map
mapKeyIndexes: [3] // Email as a key of the Map
};
// Initialize the parser with properties
const csvParser = new CSVParser(
fileName,
options,
);
// Read the data inside the parser
csvParser.Read().then((data) => {
data.forEach((profile) => {
console.log('>>> profile', profile);
})
});
const connection = dbConnection("profile", 100, 0).then((response) => {
console.log('>>> response from db:', response);
}).catch((error) => {
console.log('>>> error: ', error);
});
dbConnection = async (schema?: string, limit: number = 0, offset: number = 0) => {
const DB = Sequelizer(schema);
return DB.query(`SELECT uuid, brand_id FROM
profile.profiles
WHERE brand_id IS NOT NULL
ORDER BY uuid DESC LIMIT ${limit} OFFSET ${offset}`,
{
type: QueryTypes.SELECT,
logging: false,
});
}
```