Image

PART 1 — StatefulSet + Headless Service + Persistent Storage

Avatar photoByAdil

Goal: Deploy a stateful MySQL cluster correctly in Kubernetes and understand every moving part.

LAB PREREQUISITES (DO NOT SKIP)

You need one working Kubernetes cluster:

  • Minikube

Verify: 

kubectl get nodes

You must see Ready.

STEP 0 — WHAT WE ARE BUILDING (VISUAL FIRST)

Image

Image

Image

Final architecture

  • 1 Headless Service (identity)
  • 1 StatefulSet (mysql pods)
  • 3 Pods: mysql-0, mysql-1, mysql-2
  • 1 PVC per pod
  • Stable DNS names
  • Data survives restarts

STEP 1 — CREATE A NAMESPACE (CLEAN ISOLATION)

kubectl create namespace mysql-lab
kubectl config set-context --current --namespace=mysql-lab

Verify: 

kubectl get ns

STEP 2 — HEADLESS SERVICE (MOST IMPORTANT FILE)

File: mysql-headless.yaml

apiVersion: v1
kind: Service
metadata:
  name: mysql-headless
spec:
  clusterIP: None
  selector:
    app: mysql
  ports:
    - port: 3306
      name: mysql

Apply: 

kubectl apply -f mysql-headless.yaml

Verify: 

kubectl get svc

You must see: 

mysql-headless   ClusterIP   None

🔍 WHAT THIS DOES (VISUAL)

Image

Image

  • ❌ No virtual IP
  • ✅ DNS returns individual pod IPs
  • ❌ No kube-proxy load balancing
  • ✅ Enables pod-level DNS

⚠️ Alone this is NOT ENOUGH — identity still missing

STEP 3 — STATEFULSET (THE CORE)

File: mysql-statefulset.yaml

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: mysql
spec:
  serviceName: mysql-headless
  replicas: 3
  selector:
    matchLabels:
      app: mysql
  template:
    metadata:
      labels:
        app: mysql
    spec:
      containers:
        - name: mysql
          image: mysql:8.0
          env:
            - name: MYSQL_ROOT_PASSWORD
              value: rootpass
          ports:
            - containerPort: 3306
          volumeMounts:
            - name: data
              mountPath: /var/lib/mysql
  volumeClaimTemplates:
    - metadata:
        name: data
      spec:
        accessModes: ["ReadWriteOnce"]
        resources:
          requests:
            storage: 1Gi

Apply: 

kubectl apply -f mysql-statefulset.yaml

STEP 4 — WATCH POD CREATION (ORDER MATTERS)

kubectl get pods -w

You will see: 

mysql-0  → Running
mysql-1  → Running
mysql-2  → Running

🔍 IMPORTANT

  • Pods start one by one
  • NOT parallel
  • This is guaranteed by StatefulSet

STEP 5 — VERIFY STABLE POD NAMES

kubectl get pods

Expected: 

mysql-0
mysql-1
mysql-2

Restart one pod: 

kubectl delete pod mysql-1

Watch: 

kubectl get pods -w

Result:

  • mysql-1 comes back
  • Same name
  • Same identity

VISUAL: DEPLOYMENT VS STATEFULSET

Image

Image

This is why Deployments are unsafe for databases.

STEP 6 — VERIFY PERSISTENT STORAGE

kubectl get pvc

Expected: 

data-mysql-0
data-mysql-1
data-mysql-2

Each pod has:

  • Its own disk
  • Independent lifecycle
  • Data survives pod deletion

VISUAL: VOLUMECLAIMTEMPLATES

Image

Image

STEP 7 — VERIFY DNS (CRITICAL STEP)

Run a temporary client pod: 

kubectl run dns-test --image=busybox:1.36 -it --rm --restart=Never -- sh

Inside the pod: 

nslookup mysql-0.mysql-headless
nslookup mysql-1.mysql-headless
nslookup mysql-2.mysql-headless

Each resolves to a different IP.

Exit: 

exit

VISUAL: POD DNS NAMES

Image

Image

Format: 

<pod-name>.<headless-service>.<namespace>.svc.cluster.local

STEP 8 — CONNECT TO MYSQL (REAL TEST)

kubectl exec -it mysql-0 -- mysql -uroot -prootpass

Inside MySQL: 

CREATE DATABASE labdb;
USE labdb;
CREATE TABLE test (id INT);
INSERT INTO test VALUES (1);
SELECT * FROM test;

Exit: 

exit

STEP 9 — DELETE POD, DATA MUST SURVIVE

kubectl delete pod mysql-0

Wait until it comes back: 

kubectl get pods -w

Reconnect: 

kubectl exec -it mysql-0 -- mysql -uroot -prootpass

Check: 

USE labdb;
SELECT * FROM test;

✅ Data is still there.

VISUAL: DATA SURVIVES POD DELETION

Image

Image

STEP 10 — WHAT THIS LAB PROVES (LOCK THIS IN)

Kubernetes guarantees:

  • Stable pod identity
  • Ordered startup & shutdown
  • Persistent storage
  • Predictable DNS

Kubernetes does NOT:

  • Replicate MySQL data
  • Promote replicas
  • Manage DB internals

SENIOR DEVOPS MENTAL MODEL (INTERVIEW READY)

“StatefulSets are required for databases because they provide stable pod identities and ordered lifecycle management. Headless services expose pod-level DNS so clients can target specific replicas. VolumeClaimTemplates ensure each pod has persistent storage that survives restarts.”

Similar Posts