This document outlines the security architecture integrated into the operator-go SDK. It adopts a defense-in-depth approach, split into two primary layers:
- Application Security: Focused on safely injecting sensitive data (Secrets, Keys) into workloads.
- Infrastructure Security: Focused on securing the Kubernetes execution environment (RBAC, Service Accounts, Pod Constraints).
The core design philosophy is "Zero-Touch Security". The Product Operator does not direct handle sensitive data but delegates provisioning to a specialized secret-operator.
SecretClass is a namespaced resource managed by secret-operator. It defines "how" to obtain security artifacts, while the workload (Pod) simply declares "what" it needs by referencing a SecretClass.
This mechanism is implemented using the Kubernetes CSI (Container Storage Interface). The secret-operator provides a CSI driver that intercepts volume mount requests, generates or retrieves the required secrets on-the-fly, and injects them into the container file system as files.
- Definition: Admin creates a
SecretClasscontaining the policy (e.g., "Issue certificates using ClusterIssuer 'my-ca'"). - Reference: The Product CR (e.g., HdfsCluster) specifies
secretClass: "hdfs-secret-class". - Mount: The Operator SDK constructs the StatefulSet with a CSI Volume referencing this
SecretClass. - Injection: When a Pod starts, the CSI driver calls the backend to generate artifacts (TLS certs, Keytabs) and mounts them to
/etc/secret-volume.
The SDK and secret-operator support multiple backends to address different security needs:
Calculates and issues TLS certificates for components.
- Scenario: Internal mTLS communication (e.g., DataNode <-> NameNode) or external HTTPS access.
- Mechanism:
- Automatically generates SANs (Subject Alternative Names) based on Pod DNS names (e.g.,
*.hdfs.svc.cluster.local). - Solves the comprehensive trust problem: Components from different products (e.g., Flink connecting to HDFS) can trust each other if they use
SecretClassessigned by the same Root CA.
- Automatically generates SANs (Subject Alternative Names) based on Pod DNS names (e.g.,
Automates Kerberos integration for Hadoop/Big Data ecosystems.
- Scenario: Secure clusters requiring Kerberos authentication.
- Mechanism:
- Dynamic Principal: Supports generating principals based on the Pod's specific hostname (e.g.,
nn/hdfs-namenode-0.hdfs.svc@REALM). This is critical for K8s StatefulSets where Pod names are deterministic but distinct. - Keytab Injection: Generates the keytab on the KDC and securely mounts it to the container.
- Dynamic Principal: Supports generating principals based on the Pod's specific hostname (e.g.,
Searches and injects existing Kubernetes Secrets or ConfigMaps.
- Scenario: Legacy applications or reusing existing static secrets.
- Mechanism:
- Searches for resources matching specific labels or names in the cluster.
- Security Benefit: The Product Operator does not need
LIST/WATCH/GET Secretpermissions for the entire namespace. Only the privilegedsecret-operatoraccesses the data, minimizing the attack surface.
Automates the injection of Identity Provider (IdP) credentials.
- Scenario: Workloads requiring modern authentication (e.g., Trino interacting with external Data Lakes, or Presto Web UI login).
- Mechanism:
- Credential Injection: The
secret-operatormounts the OIDC client credentials (client-id, client-secret) from a reference secret into the container. - Configuration: The Operator SDK automatically configures necessary environmental variables or JVM system properties (e.g.,
-Dsolr.authentication.oidc.client.secret=...) to enable the OIDC module in the application.
- Credential Injection: The
This layer focuses on how the Operator constructs the Kubernetes Pods and Resources to minimize the attack surface and ensure proper isolation.
Every specific Product Cluster managed by the SDK operates with its own distinct identity.
- Automated Provisioning: The SDK automatically creates a dedicated
ServiceAccountfor each Cluster instance (or specific RoleGroup, depending on configuration). - Scope: Pods run as this ServiceAccount, meaning any audit logs in Kubernetes will reflect the specific application identity rather than a generic "default" account.
- Customization: Users can override the generated ServiceAccount name in the CR Spec if integration with external IAM (like AWS IRSA or Google Workload Identity) is required.
Workloads often need to interact with the Kubernetes API (e.g., Flink JobManager creating generic Jobs, Spark driver creating executor pods).
- Dynamic Binding: The SDK allows Products to define the exact RBAC permissions required by their workloads. The Operator then creates
RoleandRoleBindingresources linking the workload'sServiceAccountto these permissions. - Benefit: No manual
kubectl create rolebindingis needed, yet the permissions are scoped strictly to what the application declares it needs, preventing over-privileged pods.
The SDK generates PodSpecs that adhere to modern container security best practices.
- Non-Root Execution:
- By default, the SDK configures
securityContext.runAsUserandrunAsGroupto non-zero values (typically 1001), ensuring processes do not run as root.
- By default, the SDK configures
- Volume Ownership (fsGroup):
- To ensure that the non-root process can write to Persistent Volumes, the SDK automatically sets
securityContext.fsGroup. This instructs Kubernetes to change the ownership of mounted volumes to the correct group ID at startup.
- To ensure that the non-root process can write to Persistent Volumes, the SDK automatically sets
- Capability Dropping:
- Where possible, the SDK encourages dropping unnecessary Linux capabilities (e.g.,
ALL) and only adding required ones (e.g.,NET_BIND_SERVICE).
- Where possible, the SDK encourages dropping unnecessary Linux capabilities (e.g.,
- Access Isolation: Product Operators operate with minimal RBAC privileges, reducing the blast radius if an operator is compromised.
- Lifecycle Management: Certificates are automatically renewed by the
secret-operatorwithout restarting Pods (if the application supports hot-reload) or via simple rolling restarts. - Consistency: Standardizes security configurations across all data products (HDFS, Hive, Trino, etc.).