Infrastructureļ
Domainsļ
nextstrain.org, next.nextstrain.org, and dev.nextstrain.org are hosted on Heroku.
data.nextstrain.org is an AWS CloudFronted S3 bucket, nextstrain-data
.
staging.nextstrain.org is an AWS CloudFronted S3 bucket, nextstrain-staging
.
login.nextstrain.org is used by our AWS Cognito user pool.
clades.nextstrain.org is used by Nextclade.
docs.nextstrain.org is a documentation site with the Nextstrain Read the Docs project and sub-projects.
support.nextstrain.org is an email ticketing system powered by Zoho Desk.
discussion.nextstrain.org is a discussion forum powered by Discourse.
Herokuļ
We use a Heroku pipeline named nextstrain-server
to manage multiple related apps.
The production app serving nextstrain.org is nextstrain-server
.
The canary app serving next.nextstrain.org is nextstrain-canary
.
Deploys of master
to the canary app happen automatically after GitHub Actions CI tests are successful.
Deploys to the production app are performed by manually promoting the canaryās current release to production.
Environment (or config) variablesļ
NODE_ENV
is used to condition behaviour between production and non-production environments, following the widely-used convention set by Express. Its value affects not just the explicit conditionals in this repo but also Express and other layers in our dependencies. All our Heroku apps run underNODE_ENV=production
.SESSION_SECRET
must be set to a long, securely generated string. It protects the session data stored in browser cookies. Changing this will invalidate all existing sessions and forcibly logout people.SESSION_ENCRYPTION_KEYS
must be set to a URL query param string encoding pairs of key names and base64-encoded key material. These keys protect sensitive data in the session (such as authn tokens) when session data is āat restā (such as in Redis). You may prepend new keys to use for new sessions (i.e. key rotation) but do not drop old keys or old sessions will be unusable and people will be forcibly logged out. Keys must be 256 bits in length.SESSION_COOKIE_DOMAIN
is set when necessary to allow sharing of session cookies between parent and subdomains, e.g. nextstrain.org and next.nextstrain.org.AWS_ACCESS_KEY_ID
andAWS_SECRET_ACCESS_KEY
are tied to thenextstrain.org
AWS IAM user. These credentials allow the backend web server limited access to private S3 buckets.AWS_REGION
should be set to the region containing S3 buckets. This isus-east-1
for production and testing.REDIS_URL
is provided by the Heroku Redis add-on. It should not be modified directly. Our authentication handlers rewrite it at server start to use a secure TLS connection.FETCH_CACHE
is not currently used, but can be set to change the location of the on-disk cache used by (some) serverfetch()
-es. The default location is/tmp/fetch-cache
.PAPERTRAIL_API_TOKEN
is used for logging through papertrail.GITHUB_TOKEN
is used to make authenticated requests to the GitHub API so that we get increased rate limits and can download workflow artifacts. The token should be a fine-grained personal access token with public access only, i.e. no specific respositories or permissions granted. If not provided, requests to the GitHub API are unauthenticated. The token we use in production is associated with thenextstrain-bot
GitHub user.CANARY_ORIGIN
is the origin of a canary deployment to redirect to for users who are opted-in to thecanary
flag (the!flags/canary
Cognito group). Redirection does not happen if this variable is not defined or if the requestās origin already matches this variableās value. In production we set this tohttps://next.nextstrain.org
.CONFIG_FILE
is the path to a JSON file defining the defaults for the required variables below. If not provided, the checked-in filesenv/production/config.json
andenv/testing/config.json
are used (depending on the value ofNODE_ENV
).USE_PREBUILT_STATIC_SITE
is useful for development purposes and allows you to run the static site (sometimes referred to as the āfrontendā or the āNext.js siteā) using pre-built assets, which is how it behaves in production mode. In the absence of this variable theNODE_ENV
is used to decide whether to use pre-built assets (āproduction modeā) or compile assets as requested (ādevelopment modeā). IfNODE_ENV="production"
thenUSE_PREBUILT_STATIC_SITE
has no effect. (P.S. to use prebuilt assets they must first be built, typically vianpm run build
.)
Several variables are required but obtain defaults from a config file (e.g. env/production/config.json
):
COGNITO_USER_POOL_ID
must be set to the id of the Cognito user pool to use for authentication.OIDC_IDP_URL
must be set to the URL of the Cognito user poolās IdP endpoint. This is something likehttps://cognito-idp.{REGION}.amazonaws.com/{REGION}_{ID}
.OAUTH2_CLIENT_ID
must be set to the OAuth2 client id for the nextstrain.org client registered with the Cognito user pool.OAUTH2_CLI_CLIENT_ID
must be set to the OAuth2 client id for the Nextstrain CLI client registered with the Cognito user pool.OAUTH2_LOGOUT_URL
overrides any value discovered via IdP metadata. For Cognito, which doesnāt provide a value via metadata, this must be set to the logout URL of the Cognito user poolās hosted UI. In production, this ishttps://login.nextstrain.org/logout
. In development and testing, this would be something likehttps://nextstrain-testing.auth.us-east-1.amazoncognito.com/logout
.OIDC_USERNAME_CLAIM
must be set to the field in the id token claims which contains the username for a user. For Cognito, this iscognito:username
.OIDC_GROUPS_CLAIM
must be set to the field in the id token claims which contains the list of group names for a user. For Cognito, this iscognito:groups
.GROUPS_DATA_FILE
is the path to a JSON file defining the known Groups. Our config files point to the checked-in filesenv/production/groups.json
andenv/testing/groups.json
.
Variables in the environment override defaults from the config file.
Redis add-onļ
The Heroku Redis add-on is attached to our nextstrain-server
and nextstrain-dev
apps.
Redis is used to persistently store login sessions after authentication via AWS Cognito.
A persistent data store is important for preserving sessions across deploys and regular dyno restarts.
The maintenance window is set to Friday at 22:00 UTC to Saturday at 02:00 UTC. This tries to optimize for being outside/on the fringes of business hours in relevant places around the world while being in US/Pacific business hours so the Seattle team can respond to any issues arising.
If our Redis instance reaches its maximum memory limit, existing keys will be evicted using the volatile-ttl
policy to make space for new keys.
This should preserve the most active logged in sessions and avoid throwing errors if we hit the limit.
If we regularly start hitting the memory limit, we should bump up to the next add-on plan, but I donāt expect this to happen anytime soon with current usage.
Logsļ
Server logs are available via the papertrail web app (requires heroku login).
The dev server does not have papertrail enabled, but logs may be viewed using the heroku CLI via heroku logs --app=nextstrain-dev --tail
.
Development serverļ
A testing app, nextstrain-dev
, is also used, available at dev.nextstrain.org.
Deploys to it are manual, via the dashboard or git pushes to the Heroku remote, e.g. git push -f heroku-dev <branch>:master
, where the heroku-dev
remote is https://git.heroku.com/nextstrain-dev.git.
Note that the dev server runs in production mode (NODE_ENV=production
), and also uses the nextstrain.org
AWS IAM user.
Review appsļ
We use Heroku Review Apps to create ephemeral apps for PRs to the GitHub repo. These are automatically created for PRs submitted by Nextstrain team members. To recreate an inactivated app, or create one for a PR from a fork, you can use the heroku dashboard. (Make sure to review code for security purposes before creating such an app.)
It is not currently possible to login/logout of these apps due to our AWS Cognito setup; thus private datasets cannot be accessed.
Rolling back deploymentsļ
Normal Heroku deployments, which require our GitHub Actions CI tests to pass and are subsequently built on Heroku, can take upwards of 10 minutes.
Heroku allows us to immediately return to a previous version using rollbacks.
Rollbacks can be performed via the Heroku dashboard or with heroku rollback --app=nextstrain-server vX
, where X is the version number (available via heroku releases --app=nextstrain-server
).
AWSļ
All resources are in the us-east-1
region.
If you donāt see them in the AWS Console, make sure to check the region youāre looking at.
S3 bucketsļ
nextstrain-dataļ
Public.
CloudFronted.
Contains JSONs for our core builds, as well as the nextstrain.yml
conda environment definition.
Fetches by the server happen over unauthenticated HTTP.
nextstrain-stagingļ
Public. CloudFronted. Contains JSONs for staging copies of our core builds. Fetches by the server happen over unauthenticated HTTP.
nextstrain-groupsļ
Private. Contains data for all groups. Fetches by the server happen via the S3 HTTP API using signed URLs.
EC2 instancesļ
rethink.nextstrain.org hosts the labās fauna instance, used to maintain data for the core builds.
Ephemeral instances are automatically managed by AWS Batch for nextstrain build --aws-batch
jobs.
Cognitoļ
A user pool called nextstrain.org
provides authentication for Nextstrain logins.
Cognito is integrated with the nextstrain.org server using the OAuth2 support from PassportJS in our authn/index.js
file.
We donāt use Cognitoās identity pools.
DNSļ
Nameservers for the nextstrain.org zone are hosted by DNSimple.
GitHubļ
nextstrain/nextstrain.org is the GitHub repo for the Nextstrain website.
Core and staging narratives are sourced from the nextstrain/narratives repo (the master
and staging
branches, respectively).
CIļ
CI tests and deployments are run via GitHub Actions using our CI workflow.
Any push or PR will trigger CI tests.
Pushes to the master
branch will trigger a CI deployment after tests pass.