Self-hosted, multi-user. Every user gets their own isolated environment. One Helm install. Apache-2.0.
Three teams who pick SparkLabX over wiring JupyterHub, Spark, and an IAM layer together themselves.
The pain: Everyone needs PySpark but nobody wants to install Spark on a laptop. A shared Jupyter VM means everyone shares one kernel — one bad df.collect() takes the whole team down.
The pain: Compliance, residency rules, or DPO concerns rule out SaaS notebooks. PII and PHI cannot leave your VPC, full stop.
The pain: Each business unit wants notebooks but you can't trust them not to read each other's data. App-layer ACLs are leaky.
A faithful preview of the real notebook UI. Click Run on any cell.
import org.apache.spark.sql.SparkSession
import org.apache.spark.sql.functions._
val spark = SparkSession.builder
.appName("hello-sparklabx")
.config("spark.jars.packages", "io.delta:delta-spark_2.12:3.0.0")
.getOrCreate()
println(s"Spark ${spark.version}")val df = spark.read
.option("header", "true")
.option("inferSchema", "true")
.csv("s3a://workspace/public/students.csv")
df.groupBy("department")
.agg(avg($"gpa").as("avg_gpa"))
.orderBy($"avg_gpa".desc)
.show()df.filter($"gpa" > 3.5).show(5)Most "self-hosted notebook" setups stop at docker run jupyter. SparkLabX ships the boring infrastructure — the bits between "it runs on my laptop" and "it survives Monday morning."
Each user gets a MinIO IAM account scoped to their own S3 prefix. A misclick or a typo in a notebook path returns AccessDenied from the storage layer — your app code never sees the request.
Every user gets their own kernel container/pod. Someone runs df.collect() on a 100 GB DataFrame? Only their pod OOMs — everyone else keeps coding.
OAuth wired in. Restrict access by domain (@yourcompany.com) or by individual email. Unallowed addresses fail before any DB row gets created.
Drop reference data into s3a://workspace/public/ — every user can read it from PySpark or Scala. Their own work lives at s3a://workspace/users/<you>/, invisible to everyone else.
Container exited overnight? Pod hit CrashLoopBackOff? The next "Connect" detects the corpse, deletes it, and spawns a fresh one. Nobody has to page the on-call engineer at 7am.
Notebook ↔ kernel mappings, idle-reaper state, spawn progress — all in Postgres. Backend pod can crash and restart mid-execution; the running kernels keep going and reconnect transparently.
Same product, three deployment shapes. Start with shared for a demo; move to k8s_per_user when you need real boundaries.
One Jupyter container for everyone. Quick demos and single-user dev. Cross-user Spark reads are possible — don't ship this to production.
One container per user on the host's Docker daemon. True MinIO IAM isolation per kernel. Great for local dev with prod parity.
docker.sock accessOne pod per user, backend runs in-cluster. MinIO IAM plus Kubernetes NetworkPolicy and ResourceQuota. The mode you actually run for paying users.
The script generates strong random secrets, pulls public Docker images, and starts the stack with sensible defaults. The admin password is printed when it's done.
Apache-2.0 licensed. Fork, customize, ship.
quickstart.shGenerates a fresh .env with random secrets and boots the full stack.
http://localhost:3000Log in as admin with the printed password. OAuth is optional.
# Clone $ git clone https://github.com/sparklabx/sparklabx.git $ cd sparklabx # One command, full stack $ ./quickstart.sh ▶ Generating .env with random secrets... ✓ Secrets generated. (Stored in .env — gitignored.) ▶ Pulling images... ▶ Starting stack... ✓ Backend is up. ──────────────────────────────────────── SparkLabX Notebook is running. URL: http://localhost:3000 Admin: admin Password: •••••••••••••••• Kernel: docker_per_user ────────────────────────────────────────
An honest comparison with the OSS notebooks you'd otherwise reach for. SparkLabX is essentially what you'd end up wiring on top of JupyterHub if you went down that road — bundled into one Helm chart.
| Plain Jupyter | JupyterHub | SparkLabX | |
|---|---|---|---|
| PySpark + Scala kernels | DIY install | DIY install | Bundled, configured |
| Multi-user | ✗ | ✓ | ✓ |
| Per-user kernel container | ✗ | ✓ | ✓ |
| Per-user storage isolation | ✗ | Filesystem only | IAM-enforced at the S3 layer |
| OAuth + email allowlist | Plugin | Plugin | Built in |
| Idle kernel reaper | ✗ | Configurable | Built in |
| Auto-respawn on kernel crash | ✗ | ✗ | ✓ |
| Deploy | pip install | Helm chart | Helm chart |
Same notebook engine, with the pieces you can't get out of any other Spark platform: run a classroom of 200 students on a real cluster, time-boxed exams with sealed kernels, and Spark workers that burst across AWS, GCP, or Azure depending on what's cheapest right now.
chart/. Works on EKS, GKE, AKS, k3s, and bare-metal as long as you have a default StorageClass and an ingress controller. Three other deployment modes (shared, docker_per_user, k8s_per_user) cover everything from a laptop demo to multi-tenant production.users/<username>/* plus the shared public/ prefix. The kernel container is launched with that user's credentials baked in. A PySpark line reading s3a://workspace/users/someone-else/... gets AccessDenied from MinIO before it ever hits any app code. That's the design difference vs every "isolated notebook" project that filters paths in the application layer.