Backup, Recovery & Data migration

kubriX integrates Velero for Kubernetes-native backup and recovery. This feature is part of the kubriX prime plan.

High-Level Tooling Overview

Velero provides Kubernetes-native backup, restore, and data migration.
Velero UI provides simplified Velero backup and restore workflows through a more intuitive, visual interface. It provides a central place for managing backups, restores, scheduling, alerts, and multi-cluster visibility.

How Velero Works in kubriX

Velero backs up Kubernetes resources and persistent volume data to an S3-compatible object store. kubriX supports three data protection strategies:

Strategy	Description	Data Handling
Filesystem Backup (FSB)	Pod volumes backed up using Kopia.	Kopia uploader copies files.
CSI Snapshot (no Data Mover)	Creates a CSI snapshot reference only.	No data copied; restore depends on snapshot.
CSI Snapshot + Data Mover	Creates CSI snapshot and copies data to object storage.	DataUpload/DataDownload with Kopia.

Key Differences

• With Data Mover

  • Data durability: stored in object storage.
  • Portable: restores work even if snapshot infra is lost.
  • Requires node-agent pods and CR management.

• Without Data Mover

  • Faster, snapshot-only.
  • Restore requires CSI snapshot infrastructure to exist.

The default in local/kind environments uses MinIO as the S3 backend. In production environments you point velero to your real S3 bucket.

Backup Tiers

Applications can opt in to backup by labeling their resources with backup.kubrix.io/tier:

tier	retention	Description
standard	14 days	daily
weekly	12 weeks	every sunday 3 AM
critical	7 days	every 6h (0,6,12,18)

Example: label a Deployment for daily backup:

metadata:
  labels:
    backup.kubrix.io/tier: "standard"

tip

If you can’t label every resource directly, you could label the namespace with backup.kubrix.io/tier=critical → Velero selectors will match all objects in that namespace.

Base Configuration

The velero chart is configured via values files. The key section is backupStorageLocation:

backupStorageLocation:
  config:
    s3Url: https://my-s3.example.com
    bucket: velero
    region: us-east-1
    insecureSkipTLSVerify: false
    publicUrl: https://my-s3.example.com   # optional, for download URLs
    # usecaCert: true                       # enable if your S3 endpoint uses a custom CA cert

The BackupStorageLocation uses a Kubernetes Secret named velero-cloud-credentials containing:

• cloud: the S3 access key ID and secret in AWS credentials file format
• caCert (optional): the PEM-encoded CA certificate for your S3 endpoint

The velero-repo-credentials Secret holds the Kopia repository-password.

Custom CA Certificate for S3 (caCert)

If your S3 backend uses TLS with a certificate signed by a private or corporate CA (e.g. an on-prem MinIO with a self-signed cert), you need to supply that CA certificate to Velero.

Step 1: Disable insecure TLS and enable usecaCert in your values:

backupStorageLocation:
  config:
    s3Url: https://my-internal-s3.corp.example.com
    bucket: velero
    region: us-east-1
    insecureSkipTLSVerify: false  # must be false when using caCert
    usecaCert: true

Step 2: Store the PEM-encoded CA certificate in OpenBao at kubrix-kv/data/velero/<clusterName> with key caCert:

When usecaCert: true is set, the chart automatically:

• adds the caCert key to the velero-cloud-credentials Secret via ExternalSecret
• adds a caCertRef to the BackupStorageLocation pointing to that secret so Velero uses it when connecting to S3

Velero UI (VUI)

This document describes how Velero UI works with kubriX, how to configure it via Helm values (Example: values-k3d.yaml).

Overview

VUI supports different Operation models, kubriX currently uses Single Cluster Mode for simplicity.

Features

• Intuitive dashboard with multi-cluster management
• Real-time notifications and cron heatmaps
• Storage insights and Kopia operations

Architecture

Component	Description
vui-ui	Front-end web interface
vui-api	Backend API integration with Velero/K8s
vui-watchdog	Monitoring & alert service
vui-core	Optional core for multi-cluster orchestration (extra costs from velero-ui)

Secret Management via OpenBao

In prime environments, velero credentials are managed in OpenBao and synced to the cluster via External Secrets Operator.

How it works

velero namespace
  └── SecretStore
        └── authenticates with OpenBao via Kubernetes Auth
              └── uses velero ServiceAccount
              └── token policies allow read on kubrix-kv/velero/<clusterName>

  ExternalSecret
    └── reads from openbao: kubrix-kv/data/velero/<clusterName>
        - aws_access_key_id
        - aws_secret_access_key
        - caCert  (only when usecaCert: true)
    └── creates Secret

  ExternalSecret
    └── reads from openbao: kubrix-kv/data/velero/<clusterName>
        - repositorypassword
    └── creates Secret

Required OpenBao secrets

Store the following in OpenBao at kubrix-kv/data/velero/<clusterName>:

Key	Description
aws_access_key_id	S3 access key
aws_secret_access_key	S3 secret key
repositorypassword	Kopia repository encryption password
caCert	PEM-encoded CA certificate (only when usecaCert: true)

Activating secret management

Enable createSecret in your velero values file:

createSecret:
  enabled: true
  vaultURL: "https://openbao.your-domain.example.com"

Hub & Spoke: Velero on Spoke Clusters

In a hub-and-spoke topology, each spoke cluster runs its own Velero instance but uses the central OpenBao on the hub to retrieve its S3 credentials.

Architecture overview

Hub Cluster
  ├── OpenBao (central secrets manager)
  │     └── kubrix-kv/data/velero/spoke1  (S3 credentials for spoke1)
  │
  └── team-onboarding
        ├── AuthBackendRole
        │    (allows velero SA on spoke1 to authenticate)
        └── Policy
             (grants read on kubrix-kv/data/velero/spoke1)

Spoke Cluster "spoke1"
  └── velero namespace
        ├── SecretStore
        │     (authenticates to hub OpenBao via Kubernetes Auth)
        ├── ExternalSecret → Secret "velero-cloud-credentials"
        └── ExternalSecret → Secret "velero-repo-credentials"

OpenBao Kubernetes Auth for spokes

Each spoke cluster get its own Kubernetes Auth Backend in OpenBao so the velero ServiceAccount can authenticate. This is set up as part of the general spoke registration process. Once the Kubernetes Auth Backend for a spoke exists, the team-onboarding chart automatically creates the velero-specific role and policy.

destinationClusters:
  - name: in-cluster       # hub
  - name: spoke1           # spoke: triggers AuthBackendRole + Policy creation
  - name: spoke2

Summary: what you need per spoke

What	Where
OpenBao secrets	kubrix-kv/data/velero/<spokeName> (keys: aws_access_key_id, aws_secret_access_key, repositorypassword, optionally caCert)
Kubernetes Auth Backend	Hub: configured via spoke registration
AuthBackendRole + Policy	Auto-created by team-onboarding chart when spoke is in destinationClusters
Velero values	createSecret.enabled: true, vaultURL, cluster.name
Kyverno exclusion	Exclude velero namespace from createVaultSecretStore policy

Kargo

Kargo creates freights and promotions. If they are lost you lose your freight line and your promotion state of the pipeline, see https://github.com/akuity/kargo/discussions/3126 .

• Freight.kargo.akuity.io
• Promotion.kargo.akuity.io

Activate KargoBackup

By default this function is disabled, to enable adapt kyverno helm values file:

kyvernoPolicies:
  createKargoBackup:
    enabled: true

Each change per Namespace triggers Velero Backup platform-apps/charts/kyverno/templates/policy-add-kargo-backup.yaml Default TTL for kargo backups is 7 days, if you want to reduce the amount of backups in lower stages please add label to kargo-project namespace:

  labels:
    backup.kubrix.io/kargo-ttl: 10h0m0s

Backup Example

1. Deploy the demo app:

kubectl apply -f https://raw.githubusercontent.com/vmware-tanzu/velero/refs/heads/main/examples/nginx-app/with-pv.yaml

2. Verify the deployment:

kubectl get all,pvc -n nginx-example

Output:

NAME                                    READY   STATUS    RESTARTS   AGE
pod/nginx-deployment-5fdd5585fd-8fzd4   2/2     Running   0          44s

NAME               TYPE           CLUSTER-IP    EXTERNAL-IP     PORT(S)        AGE
service/my-nginx   LoadBalancer   10.248.3.50   154.41.192.85   80:31570/TCP   44s

NAME                               READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/nginx-deployment   1/1     1            1           44s

NAME                                          DESIRED   CURRENT   READY   AGE
replicaset.apps/nginx-deployment-5fdd5585fd   1         1         1       45s

NAME                               STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   VOLUMEATTRIBUTESCLASS   AGE
persistentvolumeclaim/nginx-logs   Bound    pvc-3adb798c-afe0-4df7-9375-e99ff45e6d6f   1Gi        RWO            premium        <unset>                 47s

3. Create a backup (filesystem):

velero backup create nginx-backup --include-namespaces nginx-example

4. Create a backup with Data Mover (uploads data to S3):

velero backup create nginx-backup-dm --include-namespaces nginx-example --snapshot-move-data

Additional Information

Limitations

Kopia may not fully preserve:

• setuid/setgid bits
• Hard links
• ACLs and extended attributes
• Sockets, mount points, etc.

Velero build-in data mover has some limitations

Important Note

do NOT CHANGE repository password after you create velero backup with datamover option!

References

• Volume Snapshot Datamovement
• Unified Repository & Kopia Integration Designed
• Plugin Progress Monitoring
• VUI Official Documentation
• SerioHub VUI GitHub Repos