@jsmanifest/content-combiner
Version:
Combine content from multiple sources and transform them into one unified data structure to work with
181 lines (148 loc) • 5.04 kB
Markdown
Grab data from multiple sources by providing a fetcher and a keymapping that you provide for each source which will translate their data structures to a unified one that you define by using the **Keymapper** API from this library.
```bash
npm install --save @jsmanifest/content-combiner
```
```js
import api from '../../reporter-api'
import Keymapper from '../Keymapper'
import ContentCombiner from '../Combiner'
const mockData2 = [
{
title: 'ARSON AT HALL OF JUSTICE',
details:
"<p>Seeking the public's assistance in identifying the individuals who participated in an arson that occurred at the Louisville Hall of Justice,</p>",
dates_of_birth_used: ['January 21st, 2001', 'January 22nd, 2001'],
description: 'Louisville, Kentucky\r\nSeptember 23, 2020',
url: 'https://www.hojustice.com/at-hall-of-justice',
uid: '061f36e9ed009amk387988c38459dd2',
sex: null,
modified: '2020-09-25T14:00:55+00:00',
published: '2020-09-25T14:00:55+00:00',
relatives: {
sisters: ['Jeanie Mo', 'Jane Nurse'],
brothers: ['Michael Mo', 'Solomon Mo', { firstName: 'Bennie', lastName: 'Mo' }],
},
},
{
title: 'HOMICIDE AT QUEENSVILLE',
details: 'last seen walking at a park in queensland',
description: 'Louisville, Kentucky\r\nSeptember 23, 2020',
url: 'https://cardiac-homeland.com/homicide-at-queensville/',
uid: '061f36e9a3d9w387988c38459dd2',
sex: null,
modified: '2020-09-28T14:00:55+00:00',
published: '2020-09-26T14:00:55+00:00',
relatives: null,
},
]
const combiner = new ContentCombiner()
const getSiblings = (type) => (item) => {
if (item && item[type]) {
return {
[]: item[type].map((fullName) => {
if (typeof fullName === 'string') {
const [firstName, lastName] = fullName.split(' ')
return {
firstName,
lastName,
}
} else if (fullName && typeof fullName === 'object') {
return {
firstName: fullName.firstName,
lastName: fullName.lastName,
}
}
}),
}
}
}
const getBrothers = getSiblings('brothers')
const getSisters = getSiblings('sisters')
const composeSiblingMappers = (...fns) => {
return (item) =>
fns.reduceRight(
(acc, fn) => (acc ? Object.assign(acc, fn(item)) : fn(item) || acc),
undefined,
)
}
// prettier-ignore
combiner
keymappers: {
subtitle: 'description',
description: 'details',
updatedAt: 'modified',
siblings: (item) => composeSiblingMappers(
getBrothers,
getSisters
)(get(item, 'relatives')),
},
})
combiner
.execute()
.then((results) => {
console.log(JSON.stringify(results, null, 2))
})
.catch(console.error)
```
**Result**:
```json
[
{
"subtitle": "Louisville, Kentucky\r\nSeptember 23, 2020",
"description": "<p>Seeking the public's assistance in identifying the individuals who participated in an arson that occurred at the Louisville Hall of Justice,</p>",
"updatedAt": "2020-09-25T14:00:55+00:00",
"siblings": {
"sisters": [
{
"firstName": "Jeanie",
"lastName": "Mo"
},
{
"firstName": "Jane",
"lastName": "Nurse"
}
],
"brothers": [
{
"firstName": "Michael",
"lastName": "Mo"
},
{
"firstName": "Solomon",
"lastName": "Mo"
},
{
"firstName": "Bennie",
"lastName": "Mo"
}
]
}
},
{
"subtitle": "Louisville, Kentucky\r\nSeptember 23, 2020",
"description": "last seen walking at a park in queensland",
"updatedAt": "2020-09-28T14:00:55+00:00"
}
]
```
Apply the previous steps starting with `createFetcher`. Once `execute` is called the fetcher will be called and each returned result of each fetcher will be used to apply the keymappings. If the result you return from the fetch function is an array, it will map over each item and return an array of the mapped results. If the fetch function returns an object, it will return a single object with the mapped result.
When you have lets say 10 different fetchers and call `execute`, you will receive an array of 10 items (each of which will be an array or the single object from the corresponding fetcher)
You have now a unified data structure and what you want to do with that data is all up to you.