Destroying a cluster

Destroying a cluster deletes the cluster and everything running on it from your cloud account. It keeps your data: the storage that holds your DAGs, task logs, data lake files, dbt artifacts, and database backups stays in your account unless you choose to delete it.

To destroy a cluster, you need the Admin role or higher in the cluster's workspace. For details, see Roles.

What's deleted and what's keptCopy Anchor Link

A destroy runs terraform destroy for the cluster, then removes the cloud resources that SkaleData created outside Terraform.

Deleted on every cloud:

• The Kubernetes cluster (GKE, EKS, or AKS), its node pools, and its network.
• Every app on the cluster, and the databases on the cluster's disks.
• The cluster's container registry (Artifact Registry, Amazon ECR, or Azure Container Registry) and its images.

Also deleted on Azure:

• The cluster's Key Vault, if Airflow created one. The vault is purged after it's deleted, so a new cluster can reuse its name. If you turned on purge protection for the vault, it can't be purged; the destroy reports that and still succeeds.
• Each Airflow app's managed identity, its federated credentials, and its role assignments.

Kept in your cloud account:

• The cluster's storage, with your DAGs, task logs, data lake files, dbt artifacts, and database backups:
  • Google Cloud: Cloud Storage buckets
  • Amazon Web Services: S3 buckets
  • Azure: the cluster's storage account, unless you delete data too
• Google Cloud and Amazon Web Services: each app's service account or IAM role. After the destroy finishes, you can delete them in your cloud console.

The destroy dialog in the console lists the same items for your cluster's cloud before you confirm.

Delete data tooCopy Anchor Link

On Azure, you can also delete the cluster's storage account and everything in it. This is permanent: unless you've copied the data somewhere else, it's gone.

To destroy a cluster and delete its data:

1. In the console, open the cluster and click Destroy.
2. Select Delete data too. The storage account moves to the list of resources that will be deleted.
3. Type the cluster's name to confirm.
4. Click Destroy and delete data.

The storage account is deleted last, after the cluster, its Key Vault, and the app identities and their role assignments are gone.

To do the same through the API, add delete_data=true to the destroy request:

curl -X DELETE \
  -H "Authorization: Bearer <api-key>" \
  "https://api.skaledata.com/clusters/<cluster-id>?delete_data=true"

Replace <cluster-id> with the cluster's ID, which appears in its console URL. An API key needs the clusters:write scope to destroy a cluster.

For a cluster on Google Cloud or Amazon Web Services, the API refuses delete_data=true with HTTP 422 and destroys nothing. To delete those buckets, delete them in your cloud console after the destroy finishes.

If a destroy failsCopy Anchor Link

If any step fails, the cluster's status changes to Error, and the error message names the resources that are still in your account. To finish the cleanup, destroy the cluster again. Steps that already finished are skipped, and the destroy resumes where it stopped. If you selected Delete data too the first time, select it again; each destroy request makes its own choice.

You can't destroy a cluster while one of its apps is being provisioned, restarted, or deleted. Wait for that operation to finish, then try again.

Remove a destroyed cluster from the consoleCopy Anchor Link

A destroyed cluster stays in the console so you can review its jobs. To remove it, open the cluster and click Permanently Delete. This removes the cluster's record and job history from SkaleData. It doesn't change anything in your cloud account.