Skip to main content

DevNets

note

This page is only relevant to you if you are part of the core protocol team at Grove.

Make sure to follow the instructions here if you are unsure how to set up your access to GCP.

Table of Contents ​

GCP Console​

As an example, this GCP link links to devnet-issue-498 created from PR #498.

GCP console

Grafana logs​

As an example, this Grafana link links to the logs for devnet-issue-477 from PR #477

Grafana logs explorer

Infrastructure Provisioning​

The following is a list of details to know how our DevNet infrastructure is provisioned:

  • Grove & Kubernetes: The Kubernetes cluster is provisioned using Grove's internal tooling.
  • Main Cluster: We set up ArgoCD on the cluster and configure it to sync the main/root application on the cluster.
  • App of Apps: ArgoCD provisions all the necessary resources and other ArgoCD Applications included in that Application, following the ArgoCD App of Apps pattern.
  • PR devnet label: One of the manifests provisioned is an ArgoCD ApplicationSet, which monitors our GitHub labels and provisions a network for each GitHub issue tagged with the devnet label.
  • PR devnet-test-e2e label: As part of our CI process, when a GitHub issue is labeled devnet-test-e2e, we execute a script that creates a Kubernetes Job to for that DevNet.
  • Ephemeral DevNet: The DevNet deplyed using the labels described above are considered to be ephemeral.
  • Closing PRs: When the PR is closed or the label is removed, the infrastructure is cleaned up.
  • Persistent DevNet: We have an option to provision persistent DevNets by creating a DevNet yaml file in this directory (ArgoCD Application that monitors this directory for reference).

Configuration​

Each DevNet ArgoCD App (following the App of Apps pattern) provisions a Helm chart called full-network.

Each full-network includes other ArgoCD applications that deploy Validators and off-chain actors.

Each Helm chart receives a list of configuration files. For example, see the relayminer configuration. All possible values can be found in the values.yaml of the Helm chart, such as the relayminer Helm chart.

Interaction with DevNet​

note

Devnets are provisioned with the same mnemonc phrases as LocalNet, so it is possible to reuse the same keys from the keybase - the user just needs to change the --node= flag to point to the DevNet RPC endpoint when using the poktrolld CLI.

The DevNet RPC endpoint is exposed on https://devnet-**NETWORK_NAME**-validator-rpc.poktroll.com.

It is possible to reuse the LocalNet make targets with DevNet endpoints. For example

POCKET_NODE=https://devnet-issue-420-validator-rpc.poktroll.com make supplier_list

DevNet types​

We have two types of DevNets:

  1. Automatically provisioned by devnet-test-e2e GitHub label.
  • Their main purpose is to automatically run e2e tests for each PR this label has been assigned to.
  • They are not easily modified and configured. E.g. we automatically provision single instance of each actor only.
  1. Manually provisioned by creating a file in protocol-infra repo under devnet-configs directory.
  • Their main purpose is for testing new features or configurations that require more complex setups than the first type can provide.
  • They are easily modified and configured, allowing you to add multiple instances of each actor.

Manually provisioned DevNets​

How to create​

Commit a new YAML file in the devnets-configs directory on the main branch. Use this template as a reference. If you are not a member of the protocol team, open a pull request (PR) to add the new YAML file to the main branch. Protocol team members can bypass the main branch protection to commit directly without going through the PR flow.

How to delete​

Remove the devnet config file by committing the deletion to the main branch. If you are not a member of the protocol team, open a pull request (PR) to delete the YAML file from the main branch. Protocol team members can bypass the main branch protection to commit the deletion directly without going through the PR flow.

Configuring and updating DevNets​

New container images are automatically built on each merge to the main branch and each push to non-main branches for which there exists a PR with the push-image label. Images named according to the following format:

ghcr.io/pokt-network/poktrolld:sha-7042be3
ghcr.io/pokt-network/poktrolld:sha-7042be3922245fb4313ee90e1f28d0f402b700a0

You can update the version of DevNet by changing this parameter in the devnet config file:

image:
  tag: sha-7042be3

All parameters in this config are passed to the downstream helm chart via an ArgoCD Application. Here is a list of all options which can be modified, in addition to the image tag.

Scaling actors up and down​

You can modify the number of each actor by changing the devnet config file:

info

We use the same ignite config.yaml to provision genesis in devnet as is used in localnet. Because localnet supports a max of 3 of each actor type, any devnet deployment with more actors would also require additional corresponding genesis state to be included in config.yaml, or on-demand account funding and staking. General rule of thumb: don't go over 3.

appgateservers:
  count: 1
gateways:
  count: 1
relayminers:
  count: 1

Run e2e tests on a manually provisioned DevNet​

To run e2e tests on a manually provisioned DevNet, use the following command from the root of the poktroll repo:

IMAGE_TAG=**IMAGE TAG NAME FROM DEVNET CONFIG** NAMESPACE=devnet-**NETWORK NAME** JOB_NAME=e2e-test-**GITSHA FROM IMAGE TAG** POCKET_NODE=tcp://devnet-**NETWORK NAME**-validator-poktrolld:26657 bash .github/workflows-helpers/run-e2e-test.sh

Environment Variables:

  • IMAGE_TAG: The tag of the image from the devnet config YAML file. This tag must match the tag of the image used in the DevNet configuration.
  • NAMESPACE: The name of the devnet. This should be specified in the environment variables and follow the format devnet-NETWORK NAME.
  • JOB_NAME: A unique identifier for the e2e test job. It follows the format e2e-test-GITSHA FROM IMAGE TAG.
  • POCKET_NODE: The address of the pocket node, following the format tcp://devnet-NETWORK NAME-validator-poktrolld:26657.

Example

IMAGE_TAG=sha-7042be3 NAMESPACE=devnet-sophon JOB_NAME=e2e-test-7042be3 POCKET_NODE=tcp://devnet-sophon-validator-poktrolld:26657 bash .github/workflows-helpers/run-e2e-test.sh
info
  • The IMAGE_TAG must match the tag of the image from the devnet config YAML file.
  • The name of the devnet in the environment variables must be specified.
  • The Kubernetes context must be pointed to the protocol cluster. This command can be used to change context: 
    kubectl config set-context gke_protocol-us-central1-d505_us-central1_protocol-us-central1

Staking actors​

Since the genesis actor keys are the same as LocalNet, we can use the same commands (and keyring) to stake actors on DevNet. Additionally, DevNets match the hostnames with LocalNet, making it possible for the same stake configs to work on different networks. For example, this command stakes supplier2 on DevNet devnet-sophon:

POCKET_NODE=https://devnet-sophon-validator-rpc.poktroll.com make supplier2_stake
note

Only manually provisioned DevNets have more than one actor to stake.