Help

Common issues and frequently-asked questions.

Troubleshooting

Backend stays in CrashLoopBackOff

Fixed in v0.1.3. On older versions, the loop self-resolves once postgres becomes DNS-resolvable (~30–90 s) — wait it out or upgrade.

Long Spark job killed when I closed the browser tab

Fixed in v0.1.4. The idle reaper now checks the kernel’s execution_state before deleting an “idle” pod, so a job that’s actively running survives the closed-tab window.

Kernel pod stuck in ContainerCreating >5 min

The kernel image is ~1.2 GB compressed. First pull on a fresh node takes 5–15 min on average broadband. The kernel-prepuller DaemonSet pulls the image on every node at install time so subsequent user pods spawn instantly — let it complete before opening notebooks.

Image pull backoff on kernel pod

The GHCR repo ghcr.io/sparklabx/kernel might still be private. Either change visibility to Public at github.com/orgs/sparklabx/packages/container/kernel/settings, or create an imagePullSecret and reference it in values.

Library install succeeds but cell fails with ClassNotFoundException

Known UX issue tracked in #33 — Coursier resolution failures aren’t surfaced in the UI yet. For now, verify the coordinate exists on Maven Central before adding.

FAQ

How is this different from JupyterHub?

JupyterHub is single-user-per-server multi-tenancy without Spark bundled and without per-user storage isolation. SparkLabX ships Spark, MinIO IAM enforcement, and one-Helm-install convenience. Think “JupyterHub that actually cares about Spark + isolation”.

Can I use this in production?

Yes — small teams already do. Pre-1.0 means breaking changes are possible, so pin to a specific image SHA in production rather than :latest if you need reproducibility.

Does it work with managed Spark (EMR / Dataproc)?

The bundled Spark is embedded (driver in kernel pod). For external clusters, set SPARK_MASTER to your cluster URL in the kernel config — but the per-user IAM model assumes the kernel pod has direct S3 credentials, so it works best when storage is MinIO or any S3-compatible store.

What’s the roadmap?

See open issues tagged enhancement. Highlights for v0.2: per-notebook resource sizing (#41), better library-install error surfacing (#33), and CD automation.