Quickstart

This page takes you from a connected cloud to a running Airflow instance. Expect about 20 minutes, most of it waiting for the cluster to provision.

Before you beginCopy Anchor Link

• Install the CLI and connect a cloud. The console's welcome flow walks a new organization through the same two steps.
• Check that your cloud account has room for a cluster; see Cloud quotas. SkaleData checks quota when you submit the cluster and refuses with the increase command if the account is short.
• You need the Admin role or higher to create a cluster, and Operator or higher to add an application. See Roles and permissions.

Create a clusterCopy Anchor Link

If this is your organization's first cluster, the console shows a welcome flow after you connect a cloud. Enter a Cluster Name, choose the Applications to deploy with it (Airflow is selected by default), and click Deploy Cluster. The cluster gets the current default platform version and the applications you selected. Skip to Wait for the cluster.

Otherwise, create the cluster from the Clusters page:

1. In the console, open Clusters and click Create cluster.
2. Choose the Cloud you connected and give the cluster a Name, such as prod-01.
3. Choose a Region close to your data. Leave Version on the default; Versions and upgrades explains what it pins.
4. Leave the node pools as they are. The default pool is always created. Add pools later, when you know what your workloads need.
5. Click Create cluster.

If the form reports a quota shortfall, it lists each quota and the command that raises it. Request the increase, wait for your cloud to grant it, and submit the form again.

Wait for the clusterCopy Anchor Link

Provisioning takes about 15 minutes. The cluster's status moves from provisioning through installing and verifying to ready.

The Jobs page shows the provisioning run and groups the Terraform and Helm tasks it triggered. Expand the run to read each task's log. If the cluster ends in error, find the failing task in the run, fix the cause (quota and cloud permissions are the usual ones), and click Retry on the cluster's page.

Deploy AirflowCopy Anchor Link

If you created the cluster from the welcome flow with Airflow selected, it's already deploying. Skip to Open Airflow.

1. Open Applications and click Add Application.
2. Choose the Cluster, choose Airflow as the Application, and give the instance a name.
3. Leave the sizing settings at their defaults and click Deploy Application.

The application's status reads provisioning while Helm installs it and running once its pods are up.

Open AirflowCopy Anchor Link

Open the application's page and click Open. The console mints a short-lived credential and opens Airflow in a new tab at an address under skaledata.run, the SkaleData data plane. You sign in as yourself: Airflow identifies you from your console session, so the user and role you see in Airflow follow your SkaleData account.

Next stepsCopy Anchor Link

• Develop and deploy DAGs: run Airflow locally with skale airflow init and skale airflow start, then deploy your DAGs to the instance.
• Executors and Worker queues: decide where tasks run before you move real workloads over.
• Roles and permissions: invite teammates and give each the right role.
• App APIs and credentials: call Airflow's REST API from scripts and CI.