databricks

# Databricks

Databricks is a unified data analytics platform built on Apache Spark. It's used by data scientists, data engineers, and analysts to process and analyze large datasets for machine learning and business intelligence.

Official docs: https://docs.databricks.com/

## Databricks Overview

- **Workspace**
  - **SQL Endpoint**
    - Start SQL Endpoint
    - Stop SQL Endpoint
    - Edit SQL Endpoint
    - Get SQL Endpoint
    - List SQL Endpoints
  - **Cluster**
    - Start Cluster
    - Stop Cluster
    - Edit Cluster
    - Get Cluster
    - List Clusters
  - **Job**
    - Run Job
    - Get Job
    - List Jobs
  - **Notebook**
    - Run Notebook

## Working with Databricks

This skill uses the Membrane CLI to interact with Databricks. Membrane handles authentication and credentials refresh automatically — so you can focus on the integration logic rather than auth plumbing.

### Install the CLI

Install the Membrane CLI so you can run `membrane` from the terminal:

```bash
npm install -g @membranehq/cli
```

### First-time setup

```bash
membrane login --tenant
```

A browser window opens for authentication.

**Headless environments:** Run the command, copy the printed URL for the user to open in a browser, then complete with `membrane login complete <code>`.

### Connecting to Databricks

1. **Create a new connection:**
   ```bash
   membrane search databricks --elementType=connector --json
   ```
   Take the connector ID from `output.items[0].element?.id`, then:
   ```bash
   membrane connect --connectorId=CONNECTOR_ID --json
   ```
   The user completes authentication in the browser. The output contains the new connection id.

### Getting list of existing connections
When you are not sure if connection already exists:
1. **Check existing connections:**
   ```bash
   membrane connection list --json
   ```
   If a Databricks connection exists, note its `connectionId`


### Searching for actions

When you know what you want to do but not the exact action ID:...