auspice/DEV_DOCS.md

Web app for visualizing pathogen evolution

# Contributing to Auspice development
<!-- WARNING -->
<!-- Do not edit this file from within the docs.nextstrain.org repository. -->
<!-- It is fetched from another repository to be included in the docs.nextstrain.org build. -->
<!-- So, if you edit it after it is fetched into docs.nextstrain.org, your changes will be lost. -->
<!-- Instead, edit this file in its own repository and commit your changes there. -->
<!-- For more details on this (temporary) implementation, see https://github.com/nextstrain/docs.nextstrain.org#fetching-of-documents-from-other-repositories -->
<!-- This file is located at (fetched from): https://github.com/nextstrain/auspice/blob/master/DEV_DOCS.md -->
<!-- WARNING -->
<!-- WARNING -->
<!-- WARNING -->


Thank you for helping us to improve Nextstrain!

> **To get started, please see [the contributing guide](https://github.com/nextstrain/.github/blob/master/CONTRIBUTING.md) for useful information about how to pick an issue, submit your contributions, and so on.**

This project strictly adheres to the [Contributor Covenant Code of Conduct](https://github.com/nextstrain/.github/blob/master/CODE_OF_CONDUCT.md).

Please see the [project boards](https://github.com/orgs/nextstrain/projects) for currently available issues.

## Developer Installation

This is useful for debugging, modifying the source code, or using an unpublished feature branch.

You can install using your system `npm` or [within a Conda environment](https://docs.nextstrain.org/projects/auspice/en/stable/introduction/install.html#install-dependencies) to keep things isolated.

```sh
# grab the GitHub auspice repo
git clone https://github.com/nextstrain/auspice.git
cd auspice

# install dependencies and build auspice
npm ci

# make `auspice` available globally
npm install --global .
```

Updating Auspice should only require pulling the new version from GitHub - it shouldn't require any `npm` commands. You will, however, have to re-build Auspice whenever the client-related code has changed, via `auspice build`.

## Contributing code

Please comment on an open issue if you are working on it.
For changes unrelated to an open issue, please make an issue outlining what you would like to change/add.

Where possible, **please rebase** your work onto master rather than merging changes from master into your PR.

From a fork: `git pull --rebase upstream master`

### Make sure tests are passing

We use the following libraries for all kinds of testing, so it'd help to read the docs to get familiar with their APIs and features:

1. [`Jest`](https://github.com/facebook/jest)
2. [`Playwright`](https://playwright.dev)

When you submit a pull request to the auspice repository, certain tests will need to pass before it can be merged.

You will likely want to run these tests locally before submitting:

First, install the dependencies with `npm i`, then:

#### For unit tests

Run `npm test`.

#### For linting

Run `npm run lint`. If there are issues run `npm run lint:fix`.

#### For integration tests

Auspice used to have integration testing using jest + puppeteer, however this was removed in [PR 1672](https://github.com/nextstrain/auspice/pull/1672).
Ideally this functionality will be brought back, and that PR would be a good place to start.
If embarking on this journey, consider using Playwright since it is already used for smoke tests.

#### For smoke tests

1. Fetch the datasets with `npm run get-data`.
2. Install the testing browser with `npx playwright install chromium`.
3. Run `npm run smoke-test`.


#### Test Tips

1. Run a single `describe()`, `it()`, or `test()` **within a file**, add `.only()`:

    E.g., `describe.only()`, `it.only()`, or `test.only()`

2. You can run a single test file like so:

    E.g., `npx jest test/dates.test.js`



#### Manual testing

A Heroku pipeline for this repository is connected to GitHub under the nextstrain-bot user account. The Review Apps feature facilitates manual review of changes by automatically creating a test instance from the PR source branch and adding a link to it on the GitHub PR page. These apps are based on configuration in [app.json](./app.json).

#### Test on downstream repositories

Additionally, a GitHub Actions workflow has been set up to generate PRs in downstream repositories that reflect the new changes in Auspice. To use it, add the label [preview on auspice.us](https://github.com/nextstrain/auspice/labels/preview%20on%20auspice.us) and/or [preview on nextstrain.org](https://github.com/nextstrain/auspice/labels/preview%20on%20nextstrain.org).

## git-lfs

[Git Large File Storage](https://github.com/git-lfs/git-lfs) was removed in [PR 1672](https://github.com/nextstrain/auspice/pull/1672).
You may uninstall it via `git lfs uninstall` if needed.


## Typescript

Auspice is in the process of moving to typescript, and currently supports both `.js(x)` and `.ts(x)` files.
[This guide](https://www.typescriptlang.org/docs/handbook/migrating-from-javascript.html) is helpful to understand the bigger picture of migrating a JS project to TS. 
Typescript is supposed to help us, so there's no problem using `any` types as we move code from JS to TS, however new code would ideally be typed.

You can check the types via `npm type-check` or run `npm type-check:watch`; alternatively your editor should be able to show this.

Our CI (GitHub action) will type-check the project.

The various moving parts involved are:

* `typescript` (`tsc` command) is used by `npm run type-check` as well as other plugins/libraries
* `@typescript-eslint/eslint-plugin`, `@typescript-eslint/parser` allow ESLint to parse TypeScript syntax
* `@babel/preset-typescript` is used by babel-loader (via webpack) to parse `.ts(x)` files

## Contributing to Documentation

The main Nextstrain documentation is available at [docs.nextstrain.org](https://docs.nextstrain.org/en/latest/index.html). That documentation is built from the [docs.nextstrain.org GitHub repo](https://github.com/nextstrain/docs.nextstrain.org/) -- see the [contributing page for docs.nextstrain.org](https://docs.nextstrain.org/en/latest/guides/contribute/documentation.html) for more details on how to contribute to that documentation.

The Auspice technical reference guide is available at [docs.nextstrain.org/projects/auspice](https://docs.nextstrain.org/projects/auspice/en/stable/index.html). That documentation is built from the files in this repo in the `./docs` folder.

To preview the Auspice documentation locally,

1. Create and activate a Conda environment from `docs/conda.yml`. Example:

    ```bash
    conda env create --file docs/conda.yml
    conda activate auspice-docs
    ```

2. Build the docs from a clean slate.

    ```bash
    make -C docs clean livehtml
    ```

3. Open the preview link (http://127.0.0.1:8000) in a browser.

Changes to documentation source files (`.md` and `.rst` files under `docs/`) should automatically be reflected upon page refresh.

## Contributing to Internationalization and Localization (i18n/l18n)

If you can assist in efforts to translate the Auspice interface to more languages your assistance would be very much appreciated.
The currently available languages are displayed via a drop-down at the bottom of the sidebar.

## Adding a new language

  1) Add the language to the `getlanguageOptions` function in [this file](https://github.com/nextstrain/auspice/blob/master/src/components/controls/language.js#L24)
  2) If this is a new language, copy the folder (and the JSONs within it) `src/locales/en` and name it to match the language code for the new translation -- e.g. for Spanish this would be `src/locales/es`
  3) For each key-value in the JSONs, translate the english phrase to the new locale. (Do not modify the parts formatted as `{{...}}` or `<...>...</...>`.)

For example, a spanish translation would change the English:

```json
  "sampled between {{from}} and {{to}}": "sampled between {{from}} and {{to}}",
  "and comprising": "and comprising",
```

to

```json
  "sampled between {{from}} and {{to}}": "aislados entre {{from}} y {{to}}",
  "and comprising": "y compuesto de",
```

## Helper script to check what parts of a translation are out-of-date or missing

Run `npm run diff-lang -- X`, where `X` is the language you wish to check, for instance `es`.
This will display the strings which:

* need to be added to the translation
* are present but should be removed as they are no longer used
* are present but are simply a copy of the English version & need to be translated

> Running `npm run diff-lang` will check all available languages.

## Improving an existing translation

If a translation of a particular string is not yet available, then auspice will fall-back to the english version.

  1) Find the relevant key in the (EN) JSONs [in this directory](https://github.com/nextstrain/auspice/tree/master/src/locales/en)
  2) Add the key to the JSON with the same name, but in the directory corresponding to the language you are translating into (see above for an example).

## Releases & versioning

Releasing Auspice requires a few manual steps as there are so many downstream targets which are each to be updated with the new version.

1. Compare the git history against the previous release (e.g. [view master vs release branch on GitHub](https://github.com/nextstrain/auspice/compare/release...master)) and ensure all changes are reflected in `CHANGELOG.md`.
   If new entries are needed, commit these and push them to the remote (GitHub).
1. Ensure you are on the master branch and up to date with the remote, have `node` and `npm` in your environment, and are logged into the GitHub CLI (`gh auth status`).
1. Run the `./releaseNewVersion.sh` script.
    This will prompt you for the version number increase and push changes to both the master and release branches.
    * Auspice's [CI Action](https://github.com/nextstrain/auspice/actions/workflows/ci.yaml) will publish the new version to npm. 
    It takes some time for the new version to appear in the npm registry.
    To check if it has been released run `npm view auspice versions --json | tail`; you may wish to wait for the email to the slack channel `#nextstrain-admin` but it's unclear if there is still a lag after the email arrives before it's available to `npm` APIs.
    None of the following numbered steps will work until the new version is available on npm. 
    * The release script will automatically create an entry on [github.com/nextstrain/auspice/releases](https://github.com/nextstrain/auspice/releases/) containing information from the changelog.
    * Ensure the [docker-base CI action triggered by nextstrain-bot](https://github.com/nextstrain/docker-base/actions/workflows/ci.yml?query=branch%3Amaster+actor%3Anextstrain-bot) runs successfully.
    (This action in nextstrain/docker-base is triggered by Auspice's CI action and installs Auspice from the release branch such that it's not dependant on availability in npm.)
1. Updates to nextstrain sites which depend on Auspice:
    1. Nextstrain.org will check daily for a new version of Auspice and [automatically merge the resulting PR](https://github.com/nextstrain/nextstrain.org/blob/175171e0e1c1b331538729a1168598227d08698d/.github/workflows/dependabot-automation.yml), which results in [the canary site](https://next.nextstrain.org) being updated.
        * You can [trigger this check](https://github.com/nextstrain/nextstrain.org/blob/175171e0e1c1b331538729a1168598227d08698d/.github/dependabot.yml#L16-L17) as soon as the new version is available on npm if you wish.
        * To see the updated Auspice on [nextstrain.org](https://nextstrain.org) you'll need to promote the canary server.
    1. Auspice.us will similarly [check daily for a new version](https://github.com/nextstrain/auspice.us/blob/master/.github/dependabot.yml) however the resulting PR must be merged manually.
    1. Nextclade uses the same approach as auspice.us and is usually merged by Ivan.
1. Update the version on Bioconda:
    1. [Click to edit the Bioconda recipe](https://github.com/bioconda/bioconda-recipes/edit/master/recipes/auspice/meta.yaml). Or, consider using [the Nextstrain fork](https://github.com/nextstrain/bioconda-recipes) to create PRs that other team members can contribute to.
    2. Make the following changes:
        1. Update the version number in the first line.
        2. Change `source.sha256` to the `sha256` hash of the file at `source.url`. You can get the value using the following command:

            ```sh
            curl -s https://registry.npmjs.com/auspice/latest \
                | jq -r ".dist.tarball" \
                | xargs curl -s \
                | shasum -a 256 \
                | cut -f 1 -d " "
            ```
    3. Click through to create a Pull Request to the Bioconda GitHub repository.
1. When the new version of Auspice is available on Bioconda, manually run the [conda-base CI workflow](https://github.com/nextstrain/conda-base/actions/workflows/ci.yaml) on the `main` branch.