@csermet/multiprovider
Version:
cloud-graph provider plugin for AWS used to fetch AWS cloud data.
324 lines (236 loc) • 10.2 kB
Markdown
# Contribution Guidelines
<!-- contributionguidelines -->
<!-- toc -->
- [Getting Started](#getting-started)
- [Creating A new provider](#creating-a-new-provider)
- [Adding A new entity to an existing provider](#Adding-a-new-entity-to-an-existing-provider)
- [Adding new data to an existing entity](#Adding-new-data-to-an-existing-entity)
<!-- tocstop -->
## Getting Started
To setup `CloudGraph` in development mode, first clone the CLI repo.
**TODO:** update to correct url
```bash
git clone https://github.com/cloudgraphdev/cli.git
```
Next, if you are doing updates to an **existing** provider module, clone that as well. For example `cg-provider-aws`
```bash
git clone https://github.com/cloudgraphdev/cloudgraph-provider-aws.git
```
`cd` into the provider repo and run the repos build command. For `cg-provider-aws` this would be:
```
yarn build
```
In order to have the `CLI` pick up changes you have made locally, you must link the two repos. In the provider repo, run:
```bash
yarn link
```
The output of `yarn link` will tell you what command to run within the CLI repo. For example:
```bash
yarn link cg-provider-aws
```
Next, make your changes within the provider repo and run `yarn build` again. And that's it! Now the `CLI` will pick up your changes when it pulls in the provider client.
## Creating A New Provider
To create a new provider, you must create a new NPM module that is publicly available within the NPM registry and conforms to the naming convention `@${yourOrgName}/cg-provider-${providerName}`. For example `@myOrg/cg-provider-pivotal`. The module must export a client for your provider that extends the `Client` class found in `/sdk` shown below and defines the functions `configure`, `getSchema`, and `getData` . We will describe what each function should do below.
```
export default abstract class Provider {
constructor(config: any) {
this.logger = config.logger
this.config = config.provider
}
interface = inquirer
logger: Logger
config: any
async configure(flags: any): Promise<any> {
throw new Error('Function configure has not been defined')
}
getSchema(): string {
throw new Error('Function getSchema has not been defined')
}
async getData({ opts }: { opts: Opts }): Promise<any> {
throw new Error('Function getData has not been defined')
}
}
```
To add a custom naming for the resources' schemas, export a schemasMap property at the provider. Following the structure:
```
{
[serviceName]: "schemaName",
}
```
### Configure
The `configure` function is called by `/cli` in the `INIT` command to allow each provider to control its own configuration. This configuration will then be passed to the provider client's `constructor` as `config.provider`. The provider client must call `super(config)` within its `constructor` to allow the `@cloudgraph/sdk` client to set the `this.config` which can then be consumed within the provider. The `configure` function should return an `Object` containing all the properties and values the provider wants to allow the end user to set. Here is an example configuration for `aws`
```
{
"regions": "us-east-1,us-east-2,us-west-1",
"resources": "alb,lambda,ebs"
}
```
You may prompt the user to enter values using `this.interface` which is an instance of `Inquirer.js` https://github.com/SBoudrias/Inquirer.js
### getSchema
The `getSchema` function should return the stringified GraphQL schema that will be used by your provider. You can add any valid [Dgraph directive](https://dgraph.io/docs/graphql/directives/) to your schema in order to control the results of the schema generated by Dgraph. Below is an example implementation of `getSchema` used in `@cloudgraph/cg-provider-aws`.
**NOTE**: You will only need to define the GraphQL **types** that describe your schema and Dgraph will automatically generate the queries and mutations to access those types.
```
/**
* getSchema is used to get the schema for provider
* @returns A string of graphql sub schemas
*/
getSchema(): string {
const typesArray = loadFilesSync(path.join(__dirname), {
recursive: true,
extensions: ['graphql'],
})
return print(mergeTypeDefs(typesArray))
}
```
## getData
The `getData` function is responsible for collecting and returning all the provider data that you would like to be query-able by the end user. `/cli` creates **nodes** in the graph through the concept of `entities` and **edges** in the graph through the concept of `connections`. `entities` are the provider data objects themselves as described by the defined GraphQL schema for the provider. `connections` are objects that describe how the tool should make connections **between** entities in the provider data. The data structure returned by the `getData` function should match the `ProviderData` interface below:
**Note**: Please see the [`/cg-template-provider`](https://github.com/cloudgraphdev/cloudgraph-provider-aws.git) (**TODO**: update with link to actual template) for an example on how to create entities and connections for a provider
```
export interface ServiceConnection {
id: string // The id of the entity to make a connection to
resourceType?: string // [Optional] The name of the connection
relation?: string // [Optional] The relation beteen the entity and its connection
field: string // The property on the parent schema this connected entity should be added to
}
export interface Entity {
name: string, // The name of the entity
mutation: string, // The GraphQL mutation that should be called to push this entity to Dgraph
/**
* An array of the entity data supplied by the provider
* that matches the GraphQL schema of that entity
* (except for connections)
*/
data: any[]
}
export interface ProviderData {
entities: Entity[], // An array of objects matching the Entity interface
/**
* An object where the keys are the ids of parent entities
* to make connections to and where the values are an array of ServiceConnection
* objects denoting which child entities the parent is connected to.
*/
connections: {[key: string]: ServiceConnection[]}
}
```
## Adding a new entity to an existing provider
To add a new entity (i.e. adding RDS to AWS) to an existing provider, (i.e. `/cg-provider-aws`), you must create a new GraphQL sub-schema for that entity. This GraphQL schema should define the **type(s)** for the new entity and add any directives wanted. You must then define the functions the provider requires to query, format, and form connections for the new entity. In the case of **officially** supported providers under the `` org, this would be done by creating the functions defined in the `Service` interface below.
**NOTE**: community supported providers could handle entities differently, consult with the creators of those providers if the way to add new entities is unclear.
```
export interface Service {
/**
* function that formats an entity to match the GraphQL schema for that entity
*/
format: ({
service,
region,
account,
}: {
service: any
region: string
account: string
}) => any
/**
* [Optional] function that returns the connections for an entity
*/
getConnections?: ({
service,
region,
account,
data,
}: {
service: any
region: string
account: string
data: any
}) => {[key: string]: ServiceConnection[],
mutation: string, // GraphQL mutation used to insert this entity into the DB
/**
* Function to get the RAW entity data from the provider (such as the aws-sdk)
*/
getData: ({
regions,
credentials,
opts,
}: {
regions: string
credentials: any
opts: Opts
}) => any
}
```
You then must ensure that the `getData` function for the provider client knows about the new entity. In the case of `/cg-provider-aws` this would be done by updating the `ServiceMap` Object and `services.js` file to include the new entity. For example, if you created a new entity `MyEntity`, you would first update the `services.js` file to include your new entity.
```
export default {
alb: 'alb',
cloudwatch: 'cloudwatch',
ebs: 'ebs',
...
myEntity: 'myEntity', // The new entity you are adding
...
subnet: 'subnet',
vpc: 'vpc',
}
```
You would then update the `ServiceMap` to point to the new entity's class as seen below:
```
export const ServiceMap = {
[services.alb]: ALB,
[services.cloudwatch]: CloudWatch,
...
[services.myEntity]: MyEntity, // The new entity class you have created
[services.vpc]: VPC,
...
[services.ebs]: EBS,
}
```
## Adding new data to an existing entity
In order to add new data to an existing entity for **Officially** supported providers, you must update the entity's `schema`, `format` function, and `getData` function. Lets say you have an entity called `MyEntity` with the following schema, `getData` and `format`.
```
type MyEntity {
id: String!
name: String!
someDataPoint: String
}
function getData() => {
return {
id: 'fakeId',
name: 'fakeName',
someDataFieldToChange: 'isADataPoint'
}
}
function format(rawData) => {
return {
id: rawData.id,
name: rawData.name,
someDataPoint: rawData.someDataFieldToChange
}
}
```
and you wanted to add a new attribute called `myNewData`. You would update the entity like so:
```
type MyEntity {
id: String!
name: String!
someDataPoint: String
myNewData: String // or whatever type the new data is
}
function getData() => {
return {
id: 'fakeId',
name: 'fakeName',
someDataFieldToChange: 'isADataPoint',
myNewData: 'myNewDataToAdd'
}
}
function format(rawData) => {
return {
id: rawData.id,
name: rawData.name,
someDataPoint: rawData.someDataFieldToChange,
myNewData: rawData.myNewData
}
}
```
And that's it! The CLI will now pick up the new data point and push it to the DB.
If you have any ideas for how to make this contribution guide more effective or easier to work with please let us know, we would love to hear your feedback.
<!-- contributionguidelinesstop -->