.. Copyright (c) 2024 Digital Asset (Switzerland) GmbH and/or its affiliates. All rights reserved. .. SPDX-License-Identifier: Apache-2.0 .. _k8s_validator: Kubernetes-Based Deployment of a Validator node =============================================== This section describes how to deploy a standalone validator node in Kubernetes using Helm charts. The Helm charts deploy a validator node along with associated wallet and CNS UIs, and connect it to a global synchronizer. .. _validator_prerequisites: Requirements ------------ 1) A running Kubernetes cluster in which you have administrator access to create and manage namespaces. 2) A development workstation with the following: a. ``kubectl`` - At least v1.26.1 b. ``helm`` - At least v3.11.1 3) Your cluster needs a static egress IP. After acquiring that, provide it to your SV sponsor who will propose adding it to the IP allowlist to the other SVs. 4) Please download the release artifacts containing the sample Helm value files, from here: |bundle_download_link|, and extract the bundle: .. parsed-literal:: tar xzvf |version|\_splice-node.tar.gz .. include:: ../common/backup_suggestion.rst .. include:: required_network_parameters.rst TRUSTED_SCAN_URL The scan URL of an SV that you trust and that is reachable by your validator, often your SV sponsor. This should be of the form |generic_scan_url|, e.g., for the Global Synchronizer Foundation SV it is |gsf_scan_url|. Additional parameters describing your own setup as opposed to the connection to the network are described below. .. _validator-network-diagram: Validator Network Diagram ------------------------- .. _LucidChart link: https://lucid.app/lucidchart/46af6ed2-25b8-4608-88f7-66fc19674fce/edit?viewport_loc=-5217%2C-3874%2C6157%2C3382%2C0_0&invitationId=inv_fc3a3c02-e9fd-4ca9-9388-8defdfaa7153 .. image:: images/validator-network-diagram.png :width: 600 :alt: Validator Network Diagram .. _validator-identity-token: Preparing a Cluster for Installation ------------------------------------ Create the application namespace within Kubernetes. .. code-block:: bash kubectl create ns validator .. _validator-postgres-auth: Configuring PostgreSQL authentication ------------------------------------- The PostgreSQL instance that the helm charts create, and all apps that depend on it, require the user's password to be set through Kubernetes secrets. Currently, all apps use the Postgres user ``cnadmin``. The password can be setup with the following command, assuming you set the environment variable ``POSTGRES_PASSWORD`` to a secure value: .. todo:: call out the option of using a managed postgres instance .. code-block:: bash kubectl create secret generic postgres-secrets \ --from-literal=postgresPassword=${POSTGRES_PASSWORD} \ -n validator .. _validator-onboarding: Preparing for Validator Onboarding ---------------------------------- Ensure that your validator onboarding secret ``ONBOARDING_SECRET`` is set in the namespace you created earlier. .. code-block:: bash kubectl create secret generic splice-app-validator-onboarding-validator \ "--from-literal=secret=${ONBOARDING_SECRET}" \ -n validator .. _helm-validator-auth: Configuring Authentication -------------------------- For security, the various components that comprise your Validator node need to be able to authenticate themselves to each other, as well as be able to authenticate external UI and API users. We use JWT access tokens for authentication and expect these tokens to be issued by an (external) `OpenID Connect `_ (OIDC) provider. You must: 1. Set up an OIDC provider in such a way that both backends and web UI users are able to obtain JWTs in a supported form. 2. Configure your backends to use that OIDC provider. .. _helm-validator-auth-requirements: OIDC Provider Requirements ++++++++++++++++++++++++++ This section provides pointers for setting up an OIDC provider for use with your Validator node. Feel free to skip directly to :ref:`helm-validator-auth0` if you plan to use `Auth0 `_ for your Validator node's authentication needs. Your OIDC provider must be reachable [#reach]_ at a well known (HTTPS) URL. In the following, we will refer to this URL as ``OIDC_AUTHORITY_URL``. Both your Validator node and any users that wish to authenticate to a web UI connected to your Validator node must be able to reach the ``OIDC_AUTHORITY_URL``. We require your OIDC provider to provide a `discovery document `_ at ``OIDC_AUTHORITY_URL/.well-known/openid-configuration``. We furthermore require that your OIDC provider exposes a `JWK Set `_ document. In this documentation, we assume that this document is available at ``OIDC_AUTHORITY_URL/.well-known/jwks.json``. For machine-to-machine (Validator node component to Validator node component) authentication, your OIDC provider must support the `OAuth 2.0 Client Credentials Grant `_ flow. This means that you must be able to configure (`CLIENT_ID`, `CLIENT_SECRET`) pairs for all Validator node components that need to authenticate themselves to other components. Currently, this is the validator app backend - which needs to authenticate to the Validator node's Canton participant. The `sub` field of JWTs issued through this flow must match the user ID configured as ``ledger-api-user`` in :ref:`helm-validator-auth-secrets-config`. In this documentation, we assume that the `sub` field of these JWTs is formed as ``CLIENT_ID@clients``. If this is not true for your OIDC provider, pay extra attention when configuring ``ledger-api-user`` values below. For user-facing authentication - allowing users to access the various web UIs hosted on your Validator node, your OIDC provider must support the `OAuth 2.0 Authorization Code Grant `_ flow and allow you to obtain client identifiers for the web UIs your Validator node will be hosting. Currently, these are the Wallet web UI and the CNS web UI. You might be required to whitelist a range of URLs on your OIDC provider, such as "Allowed Callback URLs", "Allowed Logout URLs", "Allowed Web Origins", and "Allowed Origins (CORS)". If you are using the ingress configuration of this runbook, the correct URLs to configure here are ``https://wallet.validator.YOUR_HOSTNAME`` (for the Wallet web UI) and ``https://cns.validator.YOUR_HOSTNAME`` (for the CNS web UI). ``YOUR_HOSTNAME`` is a placeholder that you need to replace with the actual domain name or IP address of the server hosting your services. An identifier that is unique to the user must be set via the `sub` field of the issued JWT. On some occasions, this identifier will be used as a user name for that user on your Validator node's Canton participant. In :ref:`helm-validator-install`, you will be required to configure a user identifier as the ``validatorWalletUser`` - make sure that whatever you configure there matches the contents of the `sub` field of JWTs issued for that user. *All* JWTs issued for use with your Validator node: - must be signed using the RS256 signing algorithm In the future, your OIDC provider might additionally be required to issue JWTs with a ``scope`` explicitly set to ``daml_ledger_api`` (when requested to do so as part of the OAuth 2.0 authorization code flow). Summing up, your OIDC provider setup must provide you with the following configuration values: ======================= =========================================================================== Name Value ----------------------- --------------------------------------------------------------------------- OIDC_AUTHORITY_URL The URL of your OIDC provider for obtaining the ``openid-configuration`` and ``jwks.json``. VALIDATOR_CLIENT_ID The client id of your OIDC provider for the validator app backend. VALIDATOR_CLIENT_SECRET The client secret of your OIDC provider for the validator app backend. WALLET_UI_CLIENT_ID The client id of your OIDC provider for the wallet UI. CNS_UI_CLIENT_ID The client id of your OIDC provider for the CNS UI. ======================= =========================================================================== We are going to use these values, exported to environment variables named as per the `Name` column, in :ref:`helm-validator-auth-secrets-config` and :ref:`helm-validator-install`. When first starting out, it is suggested to configure both JWT token audiences below to the same value: ``https://canton.network.global``. Once you can confirm that your setup is working correctly using this (simple) default, we recommend that you configure dedicated audience values that match your deployment and URLs. This is important for security to avoid tokens for your validators on one network be usable for your validators on another network. You can configure audiences of your choice for the participant ledger API and the validator backend API. We will refer to these using the following configuration values: ==================================== =========================================================================== Name Value ------------------------------------ --------------------------------------------------------------------------- OIDC_AUTHORITY_LEDGER_API_AUDIENCE The audience for the participant ledger API. e.g. ``https://ledger_api.example.com`` OIDC_AUTHORITY_VALIDATOR_AUDIENCE The audience for the validator backend API. e.g. ``https://validator.example.com/api`` ==================================== =========================================================================== In case you are facing trouble with setting up your (non-Auth0) OIDC provider, it can be beneficial to skim the instructions in :ref:`helm-validator-auth0` as well, to check for functionality or configuration details that your OIDC provider setup might be missing. .. [#reach] The URL must be reachable from the Canton participant and validator app running in your cluster, as well as from all web browsers that should be able to interact with the wallet and CNS UIs. .. TODO(#2052) use a unique audience for each app .. _helm-validator-auth0: Configuring an Auth0 Tenant +++++++++++++++++++++++++++ To configure `Auth0 `_ as your validator's OIDC provider, perform the following: 1. Create an Auth0 tenant for your validator 2. Create an Auth0 API that controls access to the ledger API: a. Navigate to Applications > APIs and click "Create API". Set name to ``Daml Ledger API``, set identifier to ``https://canton.network.global``. Alternatively, if you would like to configure your own audience, you can set the identifier here. e.g. ``https://ledger_api.example.com``. b. Under the Permissions tab in the new API, add a permission with scope ``daml_ledger_api``, and a description of your choice. c. On the Settings tab, scroll down to "Access Settings" and enable "Allow Offline Access", for automatic token refreshing. 3. (Optional) If you want to configure a different audience to your APIs, you can do so by creating new Auth0 APIs with an identifier set to the audience of your choice. For example, a. Create another API by setting name to ``Validator App API``, set identifier for the Validator backend app e.g. ``https://validator.example.com/api``. 4. Create an Auth0 Application for the validator backend: a. In Auth0, navigate to Applications -> Applications, and click the "Create Application" button. b. Name it ``Validator app backend``, choose "Machine to Machine Applications", and click Create. c. Choose the ``Daml Ledger API`` API you created in step 2 in the "Authorize Machine to Machine Application" dialog and click Authorize. 5. Create an Auth0 Application for the wallet web UI. a. In Auth0, navigate to Applications -> Applications, and click the "Create Application" button. b. Choose "Single Page Web Applications", call it ``Wallet web UI``, and click Create. c. Determine the URL for your validator's wallet UI. If you're using the ingress configuration of this runbook, that would be ``https://wallet.validator.YOUR_HOSTNAME``. d. In the Auth0 application settings, add the URL of the validator wallet to the following: - "Allowed Callback URLs" - "Allowed Logout URLs" - "Allowed Web Origins" - "Allowed Origins (CORS)" e. Save your application settings. 6. Create an Auth0 Application for the CNS web UI. Repeat all steps described in step 5, with following modifications: - In step b, use ``CNS web UI`` as the name of your application. - In steps c and d, use the URL for your validator's *CNS* UI. If you're using the ingress configuration of this runbook, that would be ``https://cns.validator.YOUR_HOSTNAME``. Please refer to Auth0's `own documentation on user management `_ for pointers on how to set up end-user accounts for the two web UI applications you created. Note that you will need to create at least one such user account for completing the steps in :ref:`helm-validator-install` - for being able to log in as your Validator node's administrator. You will be asked to obtain the user identifier for this user account. It can be found in the Auth0 interface under User Management -> Users -> your user's name -> user_id (a field right under the user's name at the top). We will use the environment variables listed in the table below to refer to aspects of your Auth0 configuration: ================================== =========================================================================== Name Value ---------------------------------- --------------------------------------------------------------------------- OIDC_AUTHORITY_URL ``https://AUTH0_TENANT_NAME.us.auth0.com`` OIDC_AUTHORITY_LEDGER_API_AUDIENCE The optional audience of your choice for Ledger API. e.g. ``https://ledger_api.example.com`` VALIDATOR_CLIENT_ID The client id of the Auth0 app for the validator app backend VALIDATOR_CLIENT_SECRET The client secret of the Auth0 app for the validator app backend WALLET_UI_CLIENT_ID The client id of the Auth0 app for the wallet UI CNS_UI_CLIENT_ID The client id of the Auth0 app for the CNS UI ================================== =========================================================================== The ``AUTH0_TENANT_NAME`` is the name of your Auth0 tenant as shown at the top left of your Auth0 project. You can obtain the client ID and secret of each Auth0 app from the settings pages of that app. .. _helm-validator-auth-secrets-config: Configuring Authentication on your Validator ++++++++++++++++++++++++++++++++++++++++++++ We are now going to configure your Validator node software based on the OIDC provider configuration values you exported to environment variables at the end of either :ref:`helm-validator-auth-requirements` or :ref:`helm-validator-auth0`. (Note that some authentication-related configuration steps are also included in :ref:`helm-validator-install`) The validator app backend requires the following secret. .. code-block:: bash kubectl create --namespace validator secret generic splice-app-validator-ledger-api-auth \ "--from-literal=ledger-api-user=${VALIDATOR_CLIENT_ID}@clients" \ "--from-literal=url=${OIDC_AUTHORITY_URL}/.well-known/openid-configuration" \ "--from-literal=client-id=${VALIDATOR_CLIENT_ID}" \ "--from-literal=client-secret=${VALIDATOR_CLIENT_SECRET}" \ "--from-literal=audience=${OIDC_AUTHORITY_LEDGER_API_AUDIENCE}" To setup the wallet and CNS UI, create the following two secrets. .. code-block:: bash kubectl create --namespace validator secret generic splice-app-wallet-ui-auth \ "--from-literal=url=${OIDC_AUTHORITY_URL}" \ "--from-literal=client-id=${WALLET_UI_CLIENT_ID}" kubectl create --namespace validator secret generic splice-app-cns-ui-auth \ "--from-literal=url=${OIDC_AUTHORITY_URL}" \ "--from-literal=client-id=${CNS_UI_CLIENT_ID}" .. _helm-validator-install: Installing the Software ----------------------- Configuring the Helm Charts +++++++++++++++++++++++++++ To install the Helm charts needed to start a Validator node connected to the cluster, you will need to meet a few preconditions. The first is that there needs to be an environment variable defined to refer to the version of the Helm charts necessary to connect to this environment: |chart_version_set| Please modify the file ``splice-node/examples/sv-helm/participant-values.yaml`` as follows: - Replace ``OIDC_AUTHORITY_LEDGER_API_AUDIENCE`` in the `auth.targetAudience` entry with audience for the ledger API. e.g. ``https://ledger_api.example.com``. If you are not ready to use a custom audience, you can use the suggested default ``https://canton.network.global``. - Update the `auth.jwksUrl` entry to point to your auth provider's JWK set document by replacing ``OIDC_AUTHORITY_URL`` with your auth provider's OIDC URL, as explained above. - If you are running on a version of Kubernetes earlier than 1.24, set `enableHealthProbes` to `false` to disable the gRPC liveness and readiness probes. If you are using the provided postgres helm chart, modify ``splice-node/examples/sv-helm/postgres-values-validator-participant.yaml`` as follows: - Add ``db.volumeSize`` and ``db.volumeStorageClass`` to the values file adjust persistant storage size and storage class if necessary. (These values default to 20GiB and ``standard-rwo``) Additionally, please modify the file ``splice-node/examples/sv-helm/standalone-participant-values.yaml`` as follows: - Replace ``MIGRATION_ID`` with the migration ID of the global synchronizer on the network you are connecting to (devnet/testnet/mainnet). To configure the validator app, please modify the file ``splice-node/examples/sv-helm/validator-values.yaml`` as follows: - Replace ``TRUSTED_SCAN_URL`` with a URL of a Scan you host or trust that is reachable by your Validator. For example, the GSF scan URL, |gsf_scan_url| (This Scan instance will be used for obtaining additional Scan URLs for BFT Scan reads.) - If you want to configure the audience for the Validator app backend API, replace ``OIDC_AUTHORITY_VALIDATOR_AUDIENCE`` in the `auth.audience` entry with audience for the Validator app backend API. e.g. ``https://validator.example.com/api``. - If you want to configure the audience for the Ledger API, set the ``audience`` field in the `splice-app-validator-ledger-api-auth` k8s secret with the audience for the Ledger API. e.g. ``https://ledger_api.example.com``. - Replace ``OPERATOR_WALLET_USER_ID`` with the user ID in your IAM that you want to use to log into the wallet as the validator operator party. Note that this should be the full user id, e.g., ``auth0|43b68e1e4978b000cefba352``, *not* only the suffix ``43b68e1e4978b000cefba352`` - Replace ``YOUR_CONTACT_POINT`` by a slack user name or email address that can be used by node operators to contact you in case there are issues with your node. Note that this contact information will be publicly visible. If you do not want to share contact information, you can put an empty string. - Update the `auth.jwksUrl` entry to point to your auth provider's JWK set document by replacing ``OIDC_AUTHORITY_URL`` with your auth provider's OIDC URL, as explained above. If you want to only connect to a single trusted scan at ``TRUSTED_SCAN_URL`` but not obtain additional Scan URLs for BFT Scan reads, you can uncomment the following and set ``nonSvValidatorTrustSingleScan`` to ``true``. This does mean that you depend on that single SV and if it is broken or malicious you will be unable to use the network so usually you want to default to not enabling this. .. literalinclude:: ../../../apps/app/src/pack/examples/sv-helm/validator-values.yaml :language: yaml :start-after: TRUSTED_SINGLE_SCAN_START :end-before: TRUSTED_SINGLE_SCAN_END If you want to connect to the decentralized synchronizer via only a single trusted sequencer, you can uncomment the following and set ``useSequencerConnectionsFromScan`` to ``false``. Also replace ``TRUSTED_SYNCHRONIZER_SEQUENCER_URL`` with the publicly accessible URL of the trusted sequencer, e.g., |gsf_sequencer_url| for the sequencer operated by the GSF. This does mean that you depend on that single SV and if it is broken or malicious you will be unable to use the network so usually you want to default to not enabling this. .. literalinclude:: ../../../apps/app/src/pack/examples/sv-helm/validator-values.yaml :language: yaml :start-after: TRUSTED_SINGLE_SEQUENCER_START :end-before: TRUSTED_SINGLE_SEQUENCER_END Additionally, please modify the file ``splice-node/examples/sv-helm/standalone-validator-values.yaml`` as follows: - Replace ``MIGRATION_ID`` with the migration ID of the global synchronizer on the network you are connecting to. - Replace ``SPONSOR_SV_URL`` with the URL of the SV that provided you your secret. - Replace ``YOUR_VALIDATOR_PARTY_HINT`` with the desired name for your validator operator party. It must be of the format ``--``. - Replace ``YOUR_VALIDATOR_NODE_NAME`` with the name you want your validator node to be represented as on the network. Usually you can use the same value as for your ``validatorPartyHint``. If you are redeploying the validator app as part of a :ref:`synchronizer migration `, you will also need to set ``migrating`` to ``true`` in your ``standalone-validator-values.yaml``: .. literalinclude:: ../../../apps/app/src/pack/examples/sv-helm/standalone-validator-values.yaml :language: yaml :start-after: MIGRATION_START :end-before: MIGRATION_END Finally, please add the following values to your ``standalone-validator-values.yaml`` file: .. literalinclude:: ../../../cluster/configs/configs/ui-config-values.yaml :language: yaml .. _validator-helm-charts-install: Installing the Helm Charts ++++++++++++++++++++++++++ With these files in place, you can execute the following helm commands in sequence. It's generally a good idea to wait until each deployment reaches a stable state prior to moving on to the next step. .. parsed-literal:: helm install postgres |helm_repo_prefix|/splice-postgres -n validator --version ${CHART_VERSION} -f splice-node/examples/sv-helm/postgres-values-validator-participant.yaml --wait helm install participant |helm_repo_prefix|/splice-participant -n validator --version ${CHART_VERSION} -f splice-node/examples/sv-helm/participant-values.yaml -f splice-node/examples/sv-helm/standalone-participant-values.yaml --wait helm install validator |helm_repo_prefix|/splice-validator -n validator --version ${CHART_VERSION} -f splice-node/examples/sv-helm/validator-values.yaml -f splice-node/examples/sv-helm/standalone-validator-values.yaml --wait Once this is running, you should be able to inspect the state of the cluster and observe pods running in the new namespace. A typical query might look as follows: .. code-block:: bash $ kubectl get pods -n validator NAMESPACE NAME READY STATUS RESTARTS AGE validator ans-web-ui-5bf489db78-bdn2j 1/1 Running 0 24m validator participant-8988dfb54-m9655 1/1 Running 0 26m validator postgres-0 1/1 Running 0 37m validator validator-app-f8c74d5dd-zf9j4 1/1 Running 0 24m validator wallet-web-ui-69d85cdb99-fnj7q 1/1 Running 0 24m Note also that ``Pod`` restarts may happen during bringup, particularly if all helm charts are deployed at the same time. For example, the ``participant`` cannot start until ``postgres`` is running. .. _helm-validator-ingress: Configuring the Cluster Ingress ------------------------------- The following routes should be configured in your cluster ingress controller. ====================== ============ =========================================================================== Services Port Routes ---------------------- ------------ --------------------------------------------------------------------------- ``wallet-web-ui`` ``https://wallet.validator.`` ``validator-app`` 5003 ``https://wallet.validator./api/validator`` ``ans-web-ui`` ``https://cns.validator.`` ``validator-app`` 5003 ``https://cns.validator./api/validator`` ====================== ============ =========================================================================== * ``https://wallet.validator.`` should be routed to service ``wallet-web-ui`` in the ``validator`` namespace * ``https://wallet.validator./api/validator`` should be routed to ``/api/validator`` at port 5003 of service ``validator-app`` in the ``validator`` namespace * ``https://cns.validator.`` should be routed to service ``ans-web-ui`` in the ``validator`` namespace * ``https://cns.validator./api/validator`` should be routed to ``/api/validator`` at port 5003 of service ``validator-app`` in the ``validator`` namespace Internet ingress configuration is often specific to the network configuration and scenario of the cluster being configured. To illustrate the basic requirements of a Validator node ingress, we have provided a Helm chart that configures ingress according to the routes above using Istio, as detailed in the sections below. Requirements ++++++++++++ In order to install the reference charts, the following must be satisfied in your cluster: * *cert-manager* must be available in the cluster (See `cert-manager documentation `_) * *istio* should be installed in the cluster (See `istio documentation `_) *Note that their deployments are often platform-dependent and good documentations on how to set them up can be found online.* **Example of Istio installation:** .. code-block:: bash helm repo add istio https://istio-release.storage.googleapis.com/charts helm repo update helm install istio-base istio/base -n istio-system --set defaults.global.istioNamespace=cluster-ingress --wait helm install istiod istio/istiod -n cluster-ingress --set global.istioNamespace="cluster-ingress" --set meshConfig.accessLogFile="/dev/stdout" --wait Installation Instructions +++++++++++++++++++++++++ Create a `cluster-ingress` namespace: .. code-block:: bash kubectl create ns cluster-ingress Ensure that there is a cert-manager certificate available in a secret named ``cn-net-tls``. An example of a suitable certificate definition: .. code-block:: yaml apiVersion: cert-manager.io/v1 kind: Certificate metadata: name: cn-certificate namespace: cluster-ingress spec: dnsNames: - '*.validator.YOUR_HOSTNAME' issuerRef: name: letsencrypt-production secretName: cn-net-tls Create a file named ``istio-gateway-values.yaml`` with the following content (Tip: on GCP you can get the cluster IP from ``gcloud compute addresses list``): .. code-block:: yaml service: loadBalancerIP: "YOUR_CLUSTER_IP" loadBalancerSourceRanges: - "35.194.81.56/32" - "35.198.147.95/32" - "35.189.40.124/32" - "34.132.91.75/32" And install it to your cluster: .. code-block:: bash helm install istio-ingress istio/gateway -n cluster-ingress -f istio-gateway-values.yaml A reference Helm chart installing a gateway that uses this service is also provided. To install it, run the following (assuming the environment variable `YOUR_HOSTNAME` is set to your hostname): .. parsed-literal:: helm install cluster-gateway |helm_repo_prefix|/splice-istio-gateway -n cluster-ingress --version ${CHART_VERSION} --set cluster.daHostname=${YOUR_HOSTNAME} --set cluster.cantonHostname=${YOUR_HOSTNAME} This gateway terminates tls using the secret that you configured above, and exposes raw http traffic in its outbound port 443. Istio VirtualServices can now be created to route traffic from there to the required pods within the cluster. Another reference Helm chart is provided for that, which can be installed after 1. replacing ``YOUR_HOSTNAME`` in ``splice-node/examples/sv-helm/validator-cluster-ingress-values.yaml`` and 2. setting ``nameServiceDomain`` in the same file to ``"cns"`` using: .. parsed-literal:: helm install cluster-ingress-validator |helm_repo_prefix|/splice-cluster-ingress-runbook -n validator --version ${CHART_VERSION} -f splice-node/examples/sv-helm/validator-cluster-ingress-values.yaml .. _helm-validator-wallet-ui: Logging into the wallet UI -------------------------- After you deploy your ingress, open your browser at https://wallet.validator.YOUR_HOSTNAME and login using the credentials for the user that you configured as ``validatorWalletUser`` earlier. Once logged in one should see the transactions page. .. image:: images/wallet_home.png :width: 600 :alt: After logged in into the wallet UI .. todo:: explain the config sections below in a way that makes them also accessible to the Docker compose users .. _helm_validator_testnet_cc_grant: Granting coin to the validator for traffic purchases ---------------------------------------------------- On DevNet, your validator will automatically tap enough coin to purchase :ref:`traffic `. However, on TestNet and MainNet, your validator party will need an initial coin grant to be able to purchase traffic. After logging into the wallet UI in the previous section, note that the party ID is displayed in the top right of the UI (e.g. ``validator_validator_service_user::12204f9f94b7369e027544927703efcdf0f03cb15bd26ac53c784c627b63bdf8f041``). An SV (say your sponsor) will need to transfer coin to this party. They can do this through their wallet UI. .. todo:: * show error message that people will see while the traffic purchase fails due to insufficient funds; it is currentlye here: :ref:`error-insufficient-funds` * link to the option to disable automatic top-ups, and call out the option of using third-party traffic providers * explain liveness rewards being an alternative .. _helm_validator_topup: Configuring automatic traffic purchases --------------------------------------- Optionally you may want to configure your validator to :ref:`automatically purchase traffic ` on a pay-as-you-go basis with rate limiting. This is currently enabled by default! To disable or tune to your needs, edit the following section in the validator-values.yaml file: .. literalinclude:: ../../../apps/app/src/pack/examples/sv-helm/standalone-validator-values.yaml :language: yaml :start-after: CONFIGURING_TOPUP_START :end-before: CONFIGURING_TOPUP_END On each successful top-up, the validator app purchases a `top-up amount` of roughly ``targetThroughput * minTopupInterval`` bytes of traffic (specific amount can vary due to rounding-up). The ``minTopupInterval`` allows validator operators to control the upper-bound frequency at which automated top-ups happen. If the top-up amount is below the synchronizer-wide ``minTopupAmount`` (see :ref:`traffic_parameters`), ``minTopupInterval`` is automatically stretched so that at least ``minTopupAmount`` bytes of traffic are purchased while respecting the configured ``targetThroughput``. The next top-up gets triggered when all of the following conditions are met: - The available :ref:`extra traffic balance ` drops below the configured top-up amount (i.e., below ``targetThroughput * minTopupInterval``). - At least ``minTopupInterval`` has elapsed since the last top-up. - The validator has sufficient CC in its wallet to buy the top-up amount worth on traffic (except on DevNet, where the validator app will automatically tap enough coin to purchase traffic). Configuring sweeps and auto-accepts of transfer offers ------------------------------------------------------ You can optionally configure the validator to automatically create transfer offers to other parties on the network whenever the balance of certain parties that it hosts exceeds a certain threshold. To do so, uncomment and fill in the following section in the ``validator-values.yaml`` file: .. literalinclude:: ../../../apps/app/src/pack/examples/sv-helm/validator-values.yaml :language: yaml :start-after: SWEEP_START :end-before: SWEEP_END Whenever the balance of `` exceeds `maxBalanceUSD`, the validator will automatically create a transfer offer to ``, for an amount that leaves `minBalanceUSD` in the sender's wallet. Note that you will need to know the party IDs of both the sender and receiver, which can be copied from the wallet UIs of the respective users (in the top right corner). This therefore needs to be applied to the Helm chart in a second step after the initial deployment, once the party IDs are known. Similarly, you can configure the validator to automatically accept transfer offers from certain parties on the network. To do so, uncomment and fill in the following section in the ``validator-values.yaml`` file: .. literalinclude:: ../../../apps/app/src/pack/examples/sv-helm/validator-values.yaml :language: yaml :start-after: AUTO_ACCEPT_START :end-before: AUTO_ACCEPT_END Whenever the validator receives a transfer offer from `` to ``, it will automatically accept it. Similarly to sweeps, party IDs must be known in order to apply this configuration. .. _helm-validator-ans-web-ui: Logging into the CNS UI ----------------------------- You can open your browser at https://cns.validator.YOUR_HOSTNAME and login using the credentials for the user that you configured as ``validatorWalletUser`` earlier. You will be able to register a name on the Canton Name Service. .. image:: images/ans_home.png :width: 600 :alt: After logged in into the CNS UI .. _validator_participant_pruning: Participant Pruning ------------------- By default, participants preserve all history (it is not preserved across :ref:`major upgrades ` though). However, this leads to gradually growing databases and can slow down certain queries, in particular, queries for the active contract set on the ledger API. To mitigate that, it is possible to enable participant pruning which will remove all history beyond a specified retention point and only preserve the active contract set. Note that this only affects the participant stores. The CN apps (Validator, SV and Scan) are unaffected by enabling this, so e.g., the history in your wallet will never be pruned. Below you can see an example of the pruning config that you need to add to ``validator-values.yaml`` to retain only the history for the last 48h. Note that if your node is down for longer than the pruning window (48 hours in the example above), your node will most probably get corrupted, as the apps race catching up with the participant's attempts to keep pruning. It is therefore advisable to set the pruning window to a value that you are comfortable with in terms of guaranteeing uptime of your node. Setting it to 30 days is in general a reasonable choice, as the sequencers currently are also pruned after 30 days, so you will not be able to catch up with the network after a longer downtime anyway (see :ref:`Disaster Recovery ` for disaster recovery guidelines). Refer to the Canton documentation for more details on participant pruning: * https://docs.daml.com/ops/pruning.html * https://docs.daml.com/canton/usermanual/pruning.html .. literalinclude:: ../../../apps/app/src/pack/examples/sv-helm/validator-values.yaml :language: yaml :start-after: PARTICIPANT_PRUNING_SCHEDULE_START :end-before: PARTICIPANT_PRUNING_SCHEDULE_END