pshregistry-parser

# Platform.sh Registry Parser This is a simple library for parsing the Platform.sh registry. It provides access to a subset of Registry properties for each image, and it can be used to generate valid and up-to-date configuration YAML files for Platform.sh written in Node.js. It accesses data in a `registry.json` to generate those files. > **Note:** > At this stage, the library requires a local copy of `registry.json` to be defined in order to work. Ultimately, there will be no local registry file it reads from, instead reading from a remote, regularly-updating source in GitLab. ## Usage Example This is an early stage of the registry project. At this point, common actions like adding a newly supported version to an existing image requires you to manually add that new version to the `<image>.versions.supported` attribute in the `registry.json` file. You can find the current up-to-date version of this file in the deployed [public documentation](https://docs.platform.sh/registry/images/registry.json). ### Generating example configuration files The library creates example YAML files that fall into three main categories: * `commented`: Full configuration file used in a project, including heavy commenting. For example, for Elasticsearch, ```yaml # The name given to the Elasticsearch service (lowercase alphanumeric only). mysearch: # The type of your service (elasticsearch), which uses the format # 'type:version'. Be sure to consult the Elasticsearch documentation # (https://docs.platform.sh/configuration/services/elasticsearch.html#supported-versions) # when choosing a version. If you specify a version number which is not available, # the CLI will return an error. type: elasticsearch:7.2 # The disk attribute is the size of the persistent disk (in MB) allocated to the service. disk: 256 ``` * `full`: Full configuration file used in a project. For the Elasticsearch `.platform.app.yaml`, ```yaml relationships: elasticsearch: "mysearch:elasticsearch" ``` * `snippet`: Partial configuration file. Single-line for that image so that it could be used to append to another file if we ever want to have tools that generate complete configuration files for the user. For the Elasticsearch `.platform.app.yaml` snippet, ```yaml elasticsearch: "mysearch:elasticsearch" ``` Each of these files for every image in a Registry object is generated from the `write()` method: ```node // updateConfigs.js const psh = require("pshregistry-parser"); var registryLocation = "src/registry/images/registry.json"; var registry = new RegistryParser(registryLocation); registry.write(); ``` but you can also write files for individual images (i.e. Elasticsearch) with ``` cg.write("elasticsearch"); ``` Filenames use the convention `<image_name>.<config_file>.yaml`. Therefore, the command `cg.write("elasticsearch")` would generate ```bash saveDir/ examples/ commented/ elasticsearch.app.yaml elasticsearch.services.yaml full/ elasticsearch.app.yaml elasticsearch.services.yaml snippet/ elasticsearch.app.yaml elasticsearch.services.yaml ``` #### `content.json` Depending on your use case, you may not want to install this library as a dependency for your project, and instead point to another repository that serves the generated example configuration YAMLs as a resource. In order to facilitate your ability to do so, each examples subdirectory contains a file called `content.json` which lists all of the example files in that subdirectory. For example, `examples/commented/content.json` could include the object: ```json { "files": [ "elasticsearch.app.yaml", "elasticsearch.services.yaml", "golang.app.yaml", "kafka.app.yaml", "kafka.services.yaml", "redis.app.yaml", "redis.services.yaml", "mariadb.app.yaml", "mariadb.services.yaml", "network-storage.app.yaml", "network-storage.services.yaml", "varnish.app.yaml", "varnish.routes.yaml", "varnish.services.yaml", "mysql.app.yaml", "mysql.services.yaml", "redis-persistent.app.yaml", "redis-persistent.services.yaml" ] } ``` `content.json` files are not created when single images are passed to `write()`, only when files are written for every image in the Registry. ### Save Location You can specify the save location for the generated YAML files with a second parameter to `RegistryParser()` ```node const psh = require("pshregistry-parser"); var registryLocation = "src/registry/images/registry.json"; var saveLocation = "myfiles/examples/" var registry = new RegistryParser(registryLocation, saveLocation); registry.write(); ``` If a directory is not specfied, the library will assume that all files should be generated in the same directory as `registry.json`. If `examples` or any of its subdirectories do not exist, they will be created. ## Accessing images ### Accessing all images Each Registry image is an instance of a child of an `Image` object, and is accessible directly from the config generator instance through the `images` property. For example, `registry.images["php"]` returns ```nodejs Runtime { indent: ' ', description: 'PHP service for Platform.sh.', repo_name: 'php', disk: false, docs: { relationship_name: null, service_name: null, url: '/languages/php.html' }, endpoint: null, min_disk_size: null, name: 'PHP', runtime: true, type: 'php', supported: [ '7.1', '7.2', '7.3' ], deprecated: [ '5.4', '5.5', '5.6', '7.0' ], recommended: '7.3', supportedHTML: '<ul><li>7.1</li><li>7.2</li><li>7.3</li></ul>', supportedString: '7.1, 7.2, 7.3', deprecatedHTML: '<ul><li>5.4</li><li>5.5</li><li>5.6</li><li>7.0</li></ul>', deprecatedString: '5.4, 5.5, 5.6, 7.0', config: { app: { commented: '# The runtime the application uses. The \'type\' key defines the base container\n# image that will be used to run the application. There is a separate base\n# container image for each primary language for the application,\n# in multiple versions. Check the PHP documentation\n# (https://docs.platform.sh/languages/php.html#supported-versions)\n# to find the supported versions for the \'php\' type.\ntype: php:7.3', full: 'type: php:7.3', snippet: 'type: php:7.3' }, routes: { commented: '', full: '', snippet: '' }, services: { commented: '', full: '', snippet: '' } } } ``` PHP is a `Runtime` instance, whereas MariaDB is a `Service` instance. They each contain the same properties, some of which are built by `pshregistry-parser` and others will be a public subset of properties in the generated Registry. They only differ in the way they construct the template strings that are placed in their example configuration files. > **Note:** > In the above example, `registry.images.php` is also valid, but not recommended generally. The keys of image objects use their `type` by default, and for certain images (i.e, "chrome-headless", "network-storage", "oracle-mysql", "persistent-redis") their `type` key includes a hyphen, which is not allowed for directly accessing object properties with `registry.images.<image_type>`. > > The generated Registry will likely continue to use 'type' for image keys, so no workaround has been included to convert these examples into snakeCase. Feel free to use the other form if your use is restricted to a few images that you know will not run into this problem, but if you are going to be using every image that will include the examples above, stick to `registry.images["image_type"].property`. There are certain special cases where configuration files for a service are very different than others, and in those cases the property for that image is an instance of its own class (e.g., `Varnish`, `NetworkStorage`). #### Recommended versions For each example configuration YAML file, a "recommended" supported version is chosen and used in each file where a version must be specified. In this case, the _newest_ supported release is considered the recommended version, and is accessible with `registry.images["elasticsearch"].recommended`. ### Accessing subsets of images It is also possible to access a subset of the Registry's images, namely by accessing objects that include only those that are service or runtime images, with the properties `registry.services` and `registry.runtimes`. Individual runtime and service images can be accessed identically to `images` (e.g. `registry.services["elasticsearch"]`), as well as that individual image's properties (e.g. `registry.services["elasticsearch"].recommended`). ## Future work ### Use in the documentation `pshregistry-parser` generates up-to-date example configuration YAML files that are currently pulled into the public documentation. Since 1.1.0, it generates additional markdown files that contain "Supported Version" tables for both service and runtime images that can also be included, that forego the necessity to create them during the doc's build process (aside from calling `write()`). Currently, each image has the associated `supportedHTML` and `deprecatedHTML` properties meant to be pulled into the documentation at some point, specifically on each image's primary documentation page (e.g. Elasticsearch's [supported](https://docs.platform.sh/configuration/services/elasticsearch.html#supported-versions) and [deprecated](https://docs.platform.sh/configuration/services/elasticsearch.html#deprecated-versions) versions). The adoption of pre-generated tables in 1.1.0 may influence how these supported/deprecated sections are handled moving forward. Namely, that another subdirectory is added (i.e. `<saveDir>/versions` or `<saveDir>/lists`) to the output of `write()` that writes files that contain these strings for each image (e.g. `elasticsearch_supported.md`), and that they are similarly included into the documentation. They could include section headers (so that reference to an image's deprecated version's that does not have any will not generate that header despite the shortcode being included in the documentation) as well as descriptive text (I'm thinking specifically about the block of text commonly included in the "Deprecated" regarding upstreams). That way, each image's documentation page would look very similar: ```markdown # Elasticsearch < Short description of Elasticsearch > {% include "../../registry/images/lists/elasticsearch_supported.md" %} {% include "../../registry/images/lists/elasticsearch_deprecated.md" %} ## Relationship The format exposed in the `$PLATFORM_RELATIONSHIPS` [environment variable](/development/variables.md#platformsh-provided-variables): {% codesnippet "https://examples.docs.platform.sh/relationships/elasticsearch", language="json" %}{% endcodesnippet %} ## Usage example In your `.platform/services.yaml`: {% codesnippet "/registry/images/examples/full/elasticsearch.services.yaml", language="yaml" %}{% endcodesnippet %} In your `.platform.app.yaml`: {% codesnippet "/registry/images/examples/full/elasticsearch.app.yaml", language="yaml" %}{% endcodesnippet %} You can then use the service in a configuration file of your application with something like: {% codetabs name="Java", type="java", url="https://examples.docs.platform.sh/java/elasticsearch" -%} {%- language name="Node.js", type="js", url="https://examples.docs.platform.sh/nodejs/elasticsearch" -%} {%- language name="PHP", type="php", url="https://examples.docs.platform.sh/php/elasticsearch" -%} {%- language name="Python", type="py", url="https://examples.docs.platform.sh/python/elasticsearch" -%} {%- endcodetabs %} ``` ### Generating the registry Right now we will have to edit the `registry.json` by hand, but ideally this library will point at a "ground truth" repo somewhere else (that updates regularly) to get its data. When that happens, the `registry.json` file will be eliminated from docs altogether, and calling `write()` will also call an `update()` command to the ground truth `registry.json` in its final repo, sync up, and write the files using that data rather than a local source. The decision that remains - that will influence how the final Registry is designed - is whether a project using `pshregistry-parser` places a call to: * The `registry.json` within that repository (GitLab API call, requires a token) to make the request. * A `registry.json` that is served by an application built by that repository connected to a Platform.sh project. In that case, keeping the Registry private (as it goes on to include more sensitive information about `images/` repository locations) would require some other kind of authentication for that request. #### Consequences Currently, `pshregistry-parser` requires a local copy of the Registry and for that to be provided to the `RegistryParser` instance. The location of that file is used to determine the save locations of the generated files, unless an alternative `saveDir` is also passed to it. Removing the ability to define a local `registrySource` at all is a logical option once this change is made, but then this save directory logic will need to be reworked. At this stage, I'm thinking the following: * `registrySource` as a parameter is disabled entirely, and instead the request to the generated source elsewhere is added to the `constructor` to define `images`. * `saveDir` becomes the only parameter to `RegistryParser`, with a default value of `null`. From there, it wouldn't be necessary for projects that are interested in just accessing `images`. * If generating files is the purpose of that project (i.e. the public docs), `saveDir` can be either passed to the `constructor` or with a new `set` method.