Kubernetes in Depth — Storage, Security, and Advanced Features

Storage in Kubernetes

Kubernetes offers robust storage mechanisms to manage persistent application data. From temporary volumes to persistent volumes, Kubernetes ensures reliability and scalability for production clusters. Let’s dive into how storage works and why it’s critical for your apps.

Volumes and Volume Types

A Pod specification defines one or more volumes, specifying their name, type, and mount point. Volumes allow data sharing among containers within a Pod or across multiple Pods with defined access modes. Kubernetes supports various volume types, each suited for specific use cases:

• emptyDir: A temporary directory created when a Pod starts and erased when it terminates. It’s recreated on container restarts, perfect for transient data like caches.

• hostPath: Mounts a resource (directory, file socket, etc.) from the host node’s filesystem. Great for local testing, but avoid in production due to node-specific risks.

• NFS and iSCSI: Support multiple readers, ideal for shared read-only access.

• rbd, CephFS, GlusterFS: Enable multiple writers, suitable for distributed apps needing concurrent write access.

• GCEPersistentDisk, awsElasticBlockStore: Integrate with Google Cloud and AWS storage. Note: These in-tree plugins are being replaced by Container Storage Interface (CSI) drivers in Kubernetes 1.31 for better extensibility (learn more).

Kubernetes Pod Volumes

Pods request volumes with specific access modes. The cluster matches the request, grouping volumes by mode and sorting by size (smallest to largest). The kubelet maps devices, creates mount points, and links storage to containers via the host filesystem. StorageClass plugins handle backend specifics.

The three access modes are:

• ReadWriteOnce: Read-write access by a single node. Two Pods on the same node can write, but a third on another node triggers a FailedAttachVolume error.

• ReadOnlyMany: Read-only access by multiple nodes.

• ReadWriteMany: Read-write access by multiple nodes.

When a volume is requested, the cluster matches the Pod’s access mode, grouping volumes by mode and sorting them by size (smallest to largest). The kubelet, via the kubelet_pods.go script, maps raw devices, creates mount points, and establishes symbolic links on the host filesystem to associate storage with containers. The API server interacts with StorageClass plugins to fulfill requests, with specifics varying by backend storage.

The following YAML creates a Pod named exampleA with two containers sharing an emptyDir volume:

apiVersion: v1  
kind: Pod  
metadata:  
  name: exampleA  
spec:  
  containers:  
  - name: alphacont  
    image: busybox  
    volumeMounts:  
    - mountPath: /alphadir  
      name: sharevol  
  - name: betacont  
    image: busybox  
    volumeMounts:  
    - mountPath: /betadir  
      name: sharevol  
  volumes:  
  - name: sharevol  
    emptyDir: {}  

Verify shared access:

kubectl exec -ti exampleA -c betacont -- touch /betadir/foobar  
kubectl exec -ti exampleA -c alphacont -- ls -l /alphadir  
# Output:  
total 0  
-rw-r--r-- 1 root root 0 foobar  

Persistent Volume Claim

Persistent Volumes (PVs) abstract storage to retain data beyond a Pod’s lifecycle. Pods request storage via Persistent Volume Claims (PVCs), specifying size and StorageClass. The cluster dynamically attaches a matching PV, often using CSI drivers in modern setups.

apiVersion: v1  
kind: PersistentVolumeClaim  
metadata:  
  name: myclaim  
spec:  
  accessModes:  
  - ReadWriteOnce  
  resources:  
    requests:  
      storage: 8Gi  

In the Pod:

spec:  
  containers:  
  - name: myapp  
    image: myapp  
    volumeMounts:  
    - name: test-volume  
      mountPath: "/data"  
  volumes:  
  - name: test-volume  
    persistentVolumeClaim:  
      claimName: myclaim  

Security in Kubernetes

Kubernetes secures clusters through robust access control and data protection mechanisms. Security in production environments relies on safeguarding sensitive data and controlling API access to prevent unauthorized actions and ensure compliance.

Secrets Management

Secrets store sensitive data (e.g., passwords, keys) in base64-encoded format to avoid plain-text exposure. They’re created via kubectl or YAML and used as environment variables or mounted volumes. Note: For demo purposes, we use MySecurePass123. In production, use strong, unique passwords and enable encryption at rest or tools like Sealed Secrets.

Creating Secrets
Secrets can be generated using kubectl:

kubectl create secret generic mysql --from-literal=password=MySecurePass123  

Alternatively, manual creation involves base64-encoding data in a YAML manifest:

echo -n MySecurePass123 | base64  
# Output: TXlTZWN1cmVQYXNzMTIz  

apiVersion: v1  
kind: Secret  
metadata:  
  name: mysql  
data:  
  password: TXlTZWN1cmVQYXNzMTIz  

Secrets are not encrypted by default, only base64-encoded, requiring additional measures like encryption at rest for enhanced security. Each Secret is limited to 1MB, and excessive use can strain host memory, as Secrets are stored as API objects.

Using Secrets as Environment Variables
Secrets can be injected into containers as environment variables:

apiVersion: v1  
kind: Pod  
metadata:  
  name: dbpod  
spec:  
  containers:  
  - image: mysql:8.0
    name: dbpod  
    env:  
    - name: MYSQL_ROOT_PASSWORD  
      valueFrom:  
        secretKeyRef:  
          name: mysql  
          key: password  

This configuration securely passes the password to the MySQL container without exposing it in plain text.

Mounting Secrets as Volumes
Secrets can be mounted as files within a container, providing an alternative access method:

apiVersion: v1  
kind: Pod  
metadata:  
  name: example-pod  
spec:  
  containers:  
  - name: myapp  
    image: busybox  
    command: ["sleep", "3600"]  
    volumeMounts:  
    - mountPath: /mysqlpassword  
      name: mysql  
  volumes:  
  - name: mysql  
    secret:  
      secretName: mysql  

Verify access:

kubectl exec -ti example-pod -- cat /mysqlpassword/password  
# Output: MySecurePass123  

Mounting Secrets as volumes creates files named after the Secret’s keys, readable only by the container, enhancing data isolation.

API Security

Kubernetes secures API access through authentication, authorization, and admission control.

Authentication

Methods include:

• X.509 certificates: For client authentication, often managed by kubeadm.

• Service Accounts: Identities for Pods to interact with the API.

• OpenID Connect: For external identity providers.

Unauthenticated requests are rejected early.

Authorization

Policies evaluate requests via:

• Role-Based Access Control (RBAC): Dominant in 1.31, using Roles and RoleBindings for granular permissions.

• Attribute-Based Access Control (ABAC): Less common, attribute-based.

• Node Authorization: Limits kubelet to node-specific resources.

Admission Controllers

These validate/modify API objects before persistence, enforcing policies like resource limits or image validation. Examples: LimitRanger, PodSecurityAdmission (replacing PodSecurityPolicy in 1.31).

TLS Encryption

All API requests use TLS, configured by kubeadm. Invalid SSL certificates block access, ensuring secure communication.Advanced Features

Kubernetes advanced features enhance the management of complex applications. These capabilities, including configuration management and sophisticated HTTP routing, provide flexibility and scalability for production workloads.

ConfigMaps for Configuration

ConfigMaps decouple configuration data from container images for dynamic application management. ConfigMaps store non-sensitive data as key-value pairs or configuration files, enabling Pods to access settings without hardcoding. They are created via kubectl or YAML manifests and used as environment variables, command arguments, or volume mounts.

Creating ConfigMaps
ConfigMaps can encapsulate configuration files, such as config.js:

kubectl create configmap foobar --from-file=config.js  

The resulting YAML:

apiVersion: v1  
kind: ConfigMap  
metadata:  
  name: foobar  
data:  
  config.js: |  
    {  
      "key": "value"  
    }  

Retrieve with:

kubectl get configmap foobar -o yaml  

ConfigMaps can also be populated from literal values or directories, supporting diverse configuration needs.

Using ConfigMaps

ConfigMaps are consumed in Pods in several ways:

• Environment Variables: Inject values into containers:

apiVersion: v1  
kind: Pod  
metadata:  
  name: example-pod  
spec:  
  containers:  
  - name: myapp  
    image: myapp  
    env:  
    - name: SPECIAL_LEVEL_KEY  
      valueFrom:  
        configMapKeyRef:  
          name: special-config  
          key: special.how  

• Command Arguments: Pass values to container commands (not shown in the provided excerpt but supported).

• Volume Mounts: Mount ConfigMaps as files:

apiVersion: v1  
kind: Pod  
metadata:  
  name: config-pod  
spec:  
  containers:  
  - name: myapp  
    image: myapp  
    volumeMounts:  
    - name: config-volume  
      mountPath: "/etc/config"  
  volumes:  
  - name: config-volume  
    configMap:  
      name: special-config  

ConfigMaps must exist in the same namespace as the Pod before deployment, unless marked optional. They support complex configurations, such as populating volumes with specific paths or access modes, enhancing application portability.

Ingress for HTTP Routing

Ingress enables centralized HTTP and HTTPS routing to cluster services. Unlike Service Types (ClusterIP, NodePort, LoadBalancer, ExternalName), Ingress routes traffic based on request host or path, reducing reliance on external load balancers. It requires an Ingress Controller to process routing rules.

Ingress Controllers
Ingress Controllers, such as Nginx or Traefik, run as Pods and monitor the /ingresses endpoint in the networking.k8s.io/v1 API group. They apply Ingress Rules to direct external traffic to Services, typically over HTTP. Multiple controllers can coexist, with annotations specifying which controller handles a given Ingress:

apiVersion: networking.k8s.io/v1  
kind: Ingress  
metadata:  
  name: example-ingress  
  annotations:  
    nginx.ingress.kubernetes.io/rewrite-target: /  
spec:  
  rules:  
  - host: example.com  
    http:  
      paths:  
      - path: /  
        pathType: Prefix  
        backend:  
          service:  
            name: my-service  
            port:  
              number: 80  

Without a matching annotation, all controllers may attempt to process the Ingress, potentially causing conflicts.

Ingress Rules

Ingress Rules define how traffic is routed based on URL paths or hostnames. The above example routes requests to example.com/ to my-service on port 80. Path types (Prefix, Exact, ImplementationSpecific) provide granular control, while annotations like rewrite-target modify request paths for compatibility.

Ingress centralizes access to multiple Services, improving efficiency in large clusters. It integrates seamlessly with cloud-native environments, supporting production-grade HTTP routing as of Kubernetes 1.30.

Kubernetes streamlines data persistence and access control for robust application deployment. Advanced routing and configuration mechanisms enhance cluster flexibility. Mastery of these capabilities drives efficient management of production systems.

Made by a Human

---

---