Testing¶

This document describes how to run tests for Otterdog. You should set up your development environment as described in the Development Setup guide before running tests.

Prerequisites¶

• Skaffold
• tailscale or ngrok (either can expose the dev environment for GitHub Webhooks, see below)
• minikube
• k9s

Development Resources¶

[Optional] Creating an Organization for development¶

On your profile, create an organization

• Organization name: otterdog-<github username> (Ex: otterdog-foobar)
• Contact email: your@email
• This organization belongs to: My personal account

Accept the Terms of Service (if you agree)

Fork otterdog¶

Fork otterdog in your own organization. This will act as a base for modifications and contributions to the project, and will serve as the URL for the otterdog config base_template.

https://github.com/<Github Username>/otterdog

Create a user token¶

Create a Personal Access Token with sufficient permissions to access the organization API.

The GitHub API token needs to have the following scopes enabled: repo, workflow, admin:org, admin:org_hook, delete_repo.

For more about credentials, see Credentials.

Credentials provider configuration¶

For the purpose of this testing section, we use GitHub environment variables, but third-party providers are also supported, such as pass or bitwarden.

Add these environment variables at the GitHub org level: https://github.com/organizations/otterdog-/settings/secrets/actions/new: * OTTER_USERNAME * OTTER_PASSWORD * OTTER_TOTP_SEED * OTTER_API_TOKEN (token previously created)

NOTE: 2FA must be activated on your account.

For other credential providers, see Credentials.

Configure your otterdog (source code) to work with your development organization¶

Create a new repository named otterdog-config and set up the otterdog configuration: Otterdog configuration

Go to your organization https://github.com/otterdog-<github username> > Repositories > New repository

• Repository name: otterdog-config

Click on Create repository.

Then create the file otterdog.json in this repository with all the required settings.

{
    "defaults": {
        "jsonnet": {
            "base_template": "https://github.com/<Github Username>/otterdog#otterdog/resources/defaults/otterdog-defaults.libsonnet@main",
            "config_dir": "orgs"
        }
    },
    "organizations": [
        {
            "name": "otterdog-<Github Username>",
            "github_id": "otterdog-<Github Username>",
            "credentials": {
                "provider": "env",
                "api_token": "OTTER_API_TOKEN",
                "username": "OTTER_USERNAME",
                "password": "OTTER_PASSWORD",
                "twofa_seed": "OTTER_TOTP_SEED"
            }
        }
    ]
}

[Optional] Set up Otterdog GitHub org configuration repository¶

Go to your organization https://github.com/otterdog-<github username> > Repositories > New repository

• Repository name: .otterdog

Click on Create repository.

If the name differs from .otterdog, edit your configuration and include:

    "github": {
      "config_repo": ".<your_otterdog_github_config_repo>"
    },

[Optional] Otterdog WebApp development environment¶

To run the otterdog webapp (without integration with GitHub)

Make sure you have the eclipse-csi Helm chart repository added:

helm repo add eclipse-csi https://eclipse-csi.github.io/helm-charts

make dev-webapp

If you want use the integration with GitHub, you can use tailscale or ngrok to expose your development environment. Pick one:

• Setup a tailscale tunnel
• Setup an ngrok tunnel (alternative to tailscale)

Setup a tailscale to enable GitHub Webhooks to your development environment¶

1. Sign up/Login to tail tailscale

2. Go to tailscale admin console

We use tailscale on Kubernetes (minikube), configuring it:

1. Go to tailscale admin console -> ACL

   Include or update the tagOwners

       "tagOwners": {
           "tag:k8s-operator": [],
           "tag:k8s":          ["tag:k8s-operator"],
       },
   

   Add tag:k8s to the nodeAttrs:

   "nodeAttrs": [
       {
   
           "target": ["autogroup:member", "tag:k8s"],
           "attr":   ["funnel"],
       },
       {
           "target": ["tag:k8s"], // tag that Tailscale Operator uses to tag proxies; defaults to 'tag:k8s'
           "attr":   ["funnel"],
       },
   ],
   

2. Create an OAuth in Settings > OAuth clients (https://login.tailscale.com/admin/settings/oauth)

   Create an OAuth client in the OAuth clients page of the admin console. Create the client with Devices Core and Auth Keys write scopes, and the tag tag:k8s-operator.

   This is well described in the official doc

   Your new OAuth client's Client ID and Client Secret will be displayed—copy these values now as you'll need them in the next step.

3. Save your tailscale DNS

   It might look like tail<some hash>.ts.net

4. Export them in your terminal

   export TS_CLIENT_ID=<client id>
   export TS_CLIENT_SECRET=<client secret>
   export TS_DNS=tail<some hash>.ts.net
   

   NOTE: If you close the terminal, you will need to re-export these values.

5. Test secrets

export TS_ACCESS_TOKEN="$(
  curl -sS \
    -d "client_id=${TS_CLIENT_ID}" \
    -d "client_secret=${TS_CLIENT_SECRET}" \
    "https://api.tailscale.com/api/v2/oauth/token" \
  | jq -r '.access_token'
)"

echo "$TS_ACCESS_TOKEN"

curl -sS \
  -H "Authorization: Bearer ${TS_ACCESS_TOKEN}" \
  "https://api.tailscale.com/api/v2/tailnet/-/devices" \
| jq .

1. Enabling HTTPS

Official documentation: https://tailscale.com/docs/how-to/set-up-https-certificates

Go to the dns configuration page: https://login.tailscale.com/admin/dns

HTTPS Certificates -> Click Enable HTTPS...

Create a tailscale GitHub App¶

NOTES:

• Replace <OTTERDOG-WEBAPP> by your tail<some hash>.ts.net (tailscale) or otterdog.<your reserved domain> (ngrok)
• You can always double check the URL at https://login.tailscale.com/admin/machines as multiple instances of your development environment can generate otterdog-1, otterdog-2 ...

It is required if you are the integration with GitHub (Create a GitHub app)

1. Go to your organization https://github.com/organizations/<ORG>
2. Click Settings.
3. In the left sidebar, select <> Developer settings
4. Click GitHub Apps
5. New GitHub App

Basic Information

• Add GitHub App name: <choose a name>
• Homepage URL: <OTTERDOG-WEBAPP>

Webhook

• [X] Active
• Webhook url: <OTTERDOG-WEBAPP>/github-webhook/receive
• Secret: Choose the secret

Add the following permissions and events:

Repository Permissions

Organization Permissions

The permissions for Codespaces and Dependabot secrets are only used when the default configuration defines the functions that create these secrets, see Dependabot and Codespaces secrets.

Events

The otterdog app should appear in the organization settings > Third-party Access > GitHub Apps: https://github.com/organizations/otterdog-/settings/apps

Install the GitHub App in the organization¶

Once the app is created, you need to install it in your organization:

https://github.com/organizations/otterdog-/settings/apps/otterdog--org-app/installations

Click Install next to your organization and grant the required permissions.

Configure the values.yaml to setup your otterdog webapp¶

This file must be set at the root level of your otterdog fork project.

config:
  configOwner: "otterdog-<github username>"  # GitHub organization hosting the otterdog.json
  configToken: ""  # GitHub token — no specific permissions required, used only for rate-limiting
  dependencyTrackToken: ""  # Dependency Track API token

github:
  webhookSecret: ""  # webhook secret as configured for the GitHub App
  appId: ""  # The App ID created in GitHub
  appPrivateKey: ""  # TApp private key

NOTE: In the dev configuration, the Otterdog webapp uses the following repositories by default:

config:
  configRepo: "otterdog-configs"
  configPath: "otterdog.json"

You can override these defaults in values.yaml

It is recommended to create the repository otterdog-configs and add the otterdog.json file.

Example of otterdog.json

{
    "defaults": {
        "jsonnet": {
            "base_template": "https://github.com/<GitHub Username>/otterdog#otterdog/resources/defaults/otterdog-defaults.libsonnet@main",
            "config_dir": "orgs"
        }
    },
    "github": {
        "config_repo": ".otterdog"
    },
    "organizations": [
        {
            "name": "otterdog-<GitHub Username>",
            "github_id": "otterdog-<GitHub Username>"
        }
    ]
}

Run otterdog with tailscale¶

Make sure you have the eclipse-csi, tailscale, and dependency-track Helm chart repositories added:

helm repo add dependency-track https://dependencytrack.github.io/helm-charts
helm repo add eclipse-csi https://eclipse-csi.github.io/helm-charts
helm repo add tailscale https://pkgs.tailscale.com/helmcharts

Start the otterdog webapp with the tailscale tunnel:

make dev-webapp-tunnel

You can verify the machine is registered at https://login.tailscale.com/admin/machines.

The Otterdog WebApp is reachable at https://otterdog.tail<some hash>.ts.net

Trigger the Otterdog initialization:

curl https://otterdog.tail<some hash>.ts.net/internal/init

Dependency Track is reachable at https://sbom.tail<some hash>.ts.net (it still needs to be configured — see below)

Setup ngrok to enable GitHub Webhooks to your development environment¶

Alternative to the tailscale tunnel above.

1. Sign up/Login to ngrok

2. Go to the ngrok dashboard

We use the ngrok Kubernetes Operator (minikube), configuring it:

1. Get your API Key from https://dashboard.ngrok.com/api and your Authtoken from https://dashboard.ngrok.com/get-started/your-authtoken

2. Reserve a domain in https://dashboard.ngrok.com/domains

   NOTE: Free ngrok domains (e.g. <something>.ngrok-free.dev/.ngrok-free.app) do not support subdomains — only a custom domain you own, with wildcard DNS delegated to ngrok on a paid plan, does (see ngrok custom domains). NGROK_DOMAIN below is therefore used as-is (no otterdog. prefix) for the Otterdog webapp. Dependency Track is not exposed via ngrok in this dev setup — access it with kubectl port-forward instead (see [Optional] Configure dependency track below).

3. Export them in your terminal

   export NGROK_API_KEY=<api key>
   export NGROK_AUTHTOKEN=<authtoken>
   export NGROK_DOMAIN=<your reserved domain>
   

   NOTE: If you close the terminal, you will need to re-export these values.

Run otterdog with ngrok¶

Alternative to #### Run otterdog with tailscale above.

Make sure you have the eclipse-csi, ngrok, and dependency-track Helm chart repositories added:

helm repo add dependency-track https://dependencytrack.github.io/helm-charts
helm repo add eclipse-csi https://eclipse-csi.github.io/helm-charts
helm repo add ngrok https://charts.ngrok.com

Start the otterdog webapp with the ngrok tunnel:

make dev-webapp-ngrok

The Otterdog WebApp is reachable at https://<your reserved domain>

Trigger the Otterdog initialization:

curl https://<your reserved domain>/internal/init

Dependency Track is not exposed via ngrok; reach it with:

kubectl port-forward -n otterdog svc/dependency-track-frontend 8080:8080

then open http://localhost:8080 (it still needs to be configured — see below)

Verify webhook delivery¶

Once the service is running and the GitHub App is installed in your organization, verify that webhooks are correctly reaching the backend by sending a ping event from the app's advanced settings:

https://github.com/organizations/otterdog-/settings/apps/otterdog--org-app/advanced

Click Redeliver on the most recent ping delivery to trigger a test event. Check the otterdog backend logs to confirm the ping was received:

[2026-07-01 09:30:22.093  ] [9] [INFO] ping (3d5b7472-7522-11f1-9353-7f66c710a3db)

[Optional] Test the Playground with the documentation¶

The Playground of the webapp is available at /playground, it lists the examples of the Configuration Guides next to the editor. It evaluates them with the default configuration of the instance, or with the example template of otterdog (otterdog/resources/defaults, /playground?template=example) which the examples are validated with. Complete examples of the documentation link to it with the example preloaded and the example template selected. To make these links target your development environment, serve the documentation with the target that matches the way you started the webapp, in a second terminal:

The webapp started with make dev-webapp is not exposed, forward its port first:

kubectl port-forward -n otterdog deploy/otterdog 5000:5000

Then open the documentation at http://127.0.0.1:8000, e.g. the Configuration Guides, and follow the "Open in the Playground" link below an example.

The webapp started with any make dev-webapp* target links the examples of its Playground to the documentation served with make docs-serve*, at http://127.0.0.1:8000/ by default. Both take this address from the variable DOCS_DEV_ADDR of the Makefile, set it for both to serve the documentation on another port, e.g.:

make dev-webapp-ngrok DOCS_DEV_ADDR=127.0.0.1:8001
make docs-serve-ngrok DOCS_DEV_ADDR=127.0.0.1:8001

The webapp gets this address as value config.documentationUrl of the Helm chart (environment variable DOCUMENTATION_URL), supported from version 1.6.0 of the chart: with an earlier chart, use the -local targets, which deploy the chart of a helm-charts checkout. Outside of development, the Playground links to https://csi.eclipse.dev/otterdog/ unless config.documentationUrl is set.

The Playground of a project starts with the current configuration of its organization, once the webapp fetched it again, e.g. after calling /internal/init. Its button "Propose on GitHub" validates the modified configuration with otterdog, asking to confirm if it has errors, then copies it and opens it in the editor of GitHub, where the user creates the pull request from their own account. It opens the editor on the default branch of the configuration repository, which is also known once the configuration has been fetched again.

[Optional] Configure dependency track¶

1. Access https://sbom.tail<some hash>.ts.net (tailscale) or http://localhost:8080 via kubectl port-forward (ngrok, see above) — default DependencyTrack credentials (First time will ask to change password and re-login)

2. To generate the dependencyTrackToken, go to Administration > Access Management > Teams

3. Create a Team or choose one (i.e. Automation) create and API Key.

4. Use this API Key as dependencyTrackToken in values.yaml

It will reload automatically.

Test otterdog client¶

This section walks through basic end-to-end testing of the otterdog CLI against your development organization.

1. Test GitHub web access¶

Export your credentials and set the path to your local otterdog configuration project, then verify that web-based login works:

export OTTER_API_TOKEN=...
export OTTER_USERNAME=...
export OTTER_PASSWORD=...
export OTTER_TOTP_SEED=...

export OTTERDOG_CONFIG_ROOT="/path/to/your/otterdog-config"

otterdog web-login otterdog-<GitHub Username>

NOTE: These environment variables must be re-exported in every new terminal session.

2. Import and push the organization configuration¶

Import the current state of your GitHub organization and push the generated configuration to the .otterdog repository:

otterdog import otterdog-<GitHub Username>
otterdog push-config

This creates the initial jsonnet configuration file under orgs/otterdog-<GitHub Username>/ in your config project.

3. Fetch the remote configuration¶

Fetch the configuration stored in the remote .otterdog repository to ensure it is in sync with your local state:

otterdog fetch-config otterdog-<GitHub Username>

4. Create a test repository¶

Edit the organization jsonnet configuration file:

/path/to/your/otterdog-config/orgs/otterdog-<GitHub Username>/otterdog-<GitHub Username>.jsonnet

Add a new repository entry:

orgs.newRepo('test-repo') {
  description: "OtterDog test repository",
},

If the file contains org-level settings that conflict with the defaults (e.g. settings already defined in otterdog-defaults.libsonnet), remove the duplicates. For example, the following block can be trimmed down to only the settings you want to override:

orgs.newOrg('otterdog-<GitHub Username>', 'otterdog-<GitHub Username>') {
  settings+: {
    members_can_change_repo_visibility: true,
    members_can_create_private_pages: true,
    members_can_create_private_repositories: true,
    members_can_create_public_repositories: true,
    members_can_create_teams: true,
    members_can_delete_repositories: true,
  },

Apply the changes to GitHub:

otterdog apply otterdog-<GitHub Username>
otterdog push-config otterdog-<GitHub Username>

Test otterdog PR¶

Clone the .otterdog repository:

git clone https://github.com/otterdog-<GitHub Username>/.otterdog

Change description of test-repo:

Edit the organization jsonnet configuration file:

/path/to/your/otterdog-config/orgs/otterdog-<GitHub Username>/otterdog-<GitHub Username>.jsonnet

Change repo-test description:

orgs.newRepo('test-repo') {
  description: "OtterDog test repository from PR",
},

Create a specific branch and commit the modification:

git checkout -b "feat/repo_desc"
git add .
git commit -s -m "feat: change repo-test description"
git push origin "feat/repo_desc"

Then open a PR from this branch: https://github.com/otterdog-<GitHub Username>/.otterdog/pull/new/feat/repo_desc