What the Firebolt Operator enforces
The Firebolt Operator stamps a hardened security posture on the workloads it renders. How much you can change that posture depends on the CRD surface. Engine pods expose a wide pod-template merge layer throughFireboltEngineClass and FireboltEngine. Gateway and metadata primary
containers keep operator-owned hardening and only let you set image and
resources on the primary container.
Per-field allowlists for each template surface live in
api/v1alpha1/operatorauthority.go (PodTemplateRules per component)
and in the CRD reference pages linked in each subsection below.
Engine
When neither a referencedFireboltEngineClass nor the engine’s own
spec.template sets a container securityContext, the Firebolt Operator
applies these defaults on the engine container:
You can replace that entire container
securityContext wholesale through
FireboltEngineClass.spec.template or
FireboltEngine.spec.template.spec.containers[name=="engine"].securityContext.
The engine template wins over the class. There is no merge or floor. A
template value replaces the operator default completely, including the
ability to weaken hardening.
Pod-level spec.template.spec.securityContext is also user-settable. The
Firebolt Operator only stamps fsGroup (3473) and
fsGroupChangePolicy: OnRootMismatch when you leave those fields unset.
Sidecar containers and init containers pass through from your template
without operator hardening.
Writable directories and arbitrary UIDs
The engine image owns two writable directories: the data dir (/var/lib/firebolt, the PVC mount) and the unix socket dir
(/run/firebolt, an emptyDir). Both are group-owned by root (GID 0)
with mode 2770 (g=u plus the setgid bit), not the broad 777 the
image used previously. The read-only application payload at
/opt/firebolt stays firebolt:firebolt 0755.
This follows the OpenShift “arbitrary UID” convention: a platform that
assigns a random non-namespace UID still places the process in group 0,
so the GID-0 group ownership keeps both directories readable and
writable, while other gets nothing. The setgid bit makes files the
engine creates at runtime inherit group 0. The engine runs with
umask 0007 so it does not strip those group bits off new files. Under
the default UID/GID 3473/3473 the user owns the directories outright,
so the same layout works without OpenShift.
Mounted volumes override the image’s directory permissions. When a
platform assigns an arbitrary UID, give the engine a writable data dir
either through the stamped fsGroup (which chowns the PVC to the
group) or by mounting volumes already group-0-writable, and avoid
pinning a runAsUser that is not a member of group 0.
Validating webhooks on FireboltEngine and FireboltEngineClass (and
the engine reconciler when webhooks are off) reject operator-owned paths
on engine templates: command, args, ports, probes, reserved env keys, and
firebolt.io/* labels. They do not reject a weakened securityContext.
See the FireboltEngineClass CRD reference
and FireboltEngine CRD reference.
Gateway (Envoy)
The Envoy primary container is operator-rendered end to end. The validating webhook rejects user input on itssecurityContext. You may
only set image and resources on
spec.gateway.template.spec.containers[name=="envoy"]. The Firebolt
Operator stamps:
Pod-level fields on
spec.gateway.template (node selector, tolerations,
securityContext, sidecars, init containers) pass through when allowed
by the template rules. See the
FireboltInstance CRD reference.
Metadata (Pensieve)
The metadata primary container follows the same pattern as the gateway. Operator-stamped hardening applies tocontainers[name=="metadata"].
You may set only image and resources there:
At the pod level, the Firebolt Operator floors any user-supplied
PodSecurityContext to runAsNonRoot: true with UID/GID 1111 and sets
seccompProfile: RuntimeDefault when you do not supply one. A template
cannot run the metadata pod under a different user without failing image
ownership checks.
PostgreSQL (internal)
Internal PostgreSQL has no user template surface. The Firebolt Operator stamps hardening on every reconcile:
The Firebolt Operator’s own manager RBAC (rendered by the chart from
the canonical
config/rbac/role.yaml) is the minimal set of verbs
needed to manage the three CRDs and their generated resources. The
Firebolt Operator does not request * on any namespaced verb and
does not request nodes, clusterrolebindings, or any other
cluster-scope mutation.
The chart renders the manager rules in one of two shapes, picked at
install time by watchNamespaces:
- Empty list (default): one cluster-scoped
ClusterRoleand oneClusterRoleBinding. The manager cache spans every namespace. - Non-empty list, e.g.
{tenant-a, tenant-b}: aRoleplusRoleBindingpair in each listed namespace, noClusterRole. The manager cache spans only those namespaces. Use this posture when multi-tenant compliance constraints bound the operator’s blast radius.
FireboltInstance.spec.metricScrapeMode=PodIP reaches
engine metrics through pod IPs and does not need pods/proxy: get,
so the chart’s manager RBAC does not include that verb. Setting
metricScrapeMode=ApiserverProxy on any FireboltInstance requires
rbac.apiserverProxyGrant=true on the operator chart, which
renders a dedicated ClusterRole (or per-namespace Role when
watchNamespaces is set) granting only that one verb. Without the
toggle, the metric scrape surfaces as a 403 from the apiserver.
Resource maxima on the engine container’s resources block (set
under FireboltEngine.spec.template.spec.containers[name=="engine"]
or inherited from a referenced FireboltEngineClass) are enforced by
the validating webhook (see “Resource bounds” below). The bounds
protect a namespace from accidentally admitting an engine whose
requests would starve sibling workloads at scheduling time.
Authentication and transport encryption
Both are opt-in and Instance-wide. Neither is configured per Engine: every Engine in an Instance runs the same authentication configuration, because each one both issues and validates JWTs and a token minted by one must validate on another. See architecture for the provisioning model.Authentication
spec.auth layers two authorization servers:
- The local user store, always present when auth is enabled. It holds the Instance admin plus any users created through SQL, and backs password login through the Engine’s embedded authorization server.
- OIDC bearer validation through
spec.auth.oidc, optional. Engines accept JWTs issued by the trusted providers you configure. Engines validate those tokens; they never run a login flow themselves, so no client secret for your identity provider is ever held by the Instance.
spec.auth.local.admin.password. A credential the operator invented and you never saw would not be usable, so the reconcile preflights the Secret’s existence and reports AuthReady=False/AdminSecretMissing rather than inventing one. The password is mounted into each Engine as a file and referenced by path, so no plaintext credential appears in a rendered config.
The shared JWT signing keypair is provisioned through cert-manager against an Issuer or ClusterIssuer you supply. Signing keys are cert-manager-only — unlike TLS, there is no bring-your-own secretRef path — and the private key is mounted by path, never inlined.
Engine and Gateway TLS
spec.tls.engine terminates TLS on each Engine’s query listener; spec.tls.gateway terminates client-facing TLS on the Gateway. Both accept either a cert-manager issuer or a secretRef to a certificate you manage yourself.
Two properties are worth knowing before you enable them:
- TLS replaces plaintext on the same port; it does not add a second listener. An Engine serving TLS is not reachable in plaintext. Enabling or disabling Engine TLS on a fleet already serving traffic therefore has a brief Gateway-to-Engine outage window — see the transient-outage note in architecture.
- Tightening the Gateway’s posture is fail-closed. Going from plaintext to TLS, from one-way to mutual TLS, or swapping the client CA withholds the client-facing listener and drains the old, looser pods before any secure pod serves. The client port refuses connections for that window rather than briefly accepting clients the new posture is meant to reject. Liveness and readiness probes ride the always-plaintext metrics port, so pods stay alive through it.
clientCASecretRef upgrades the Gateway listener to mutual TLS, verifying client certificates against that CA. Loosening transitions carry no fail-open risk and are not staged.
What the Firebolt Operator does not enforce
Pod-template authors can mount other Secrets in the namespace
AFireboltEngine, FireboltEngineClass, or FireboltInstance pod template may declare volumes and sidecar containers, and a volume may reference any Secret in that namespace. Treat edit rights on those templates as equivalent to read rights on the namespace’s Secrets, and scope RBAC accordingly: someone who can edit a template can mount a Secret and read it from a sidecar whose image and command they control, without any get secrets permission.
The operator’s own auth and TLS Secrets are excluded from this. Every route a template has to them is covered, and the checks match the Secret being referenced rather than the name the author gave it, so renaming evades nothing:
- A volume whose source reaches one — including through a
projectedsource — is dropped from the rendered pod and reported on the component’s condition. It is reported rather than refused on purpose: the kubelet re-syncs a Secret volume for the life of the pod, so a volume that named a not-yet-existing Secret starts yielding real key material once the operator creates it. The remedy is to replace the pod, and refusing to render would freeze the Engine on the very generation holding the exposed volume. - A container that reads one into its environment, through
env[].valueFrom.secretKeyReforenvFrom[].secretRef, is refused outright. An env reference resolves once when the pod starts and is never re-synced, so declining to render is a complete remedy.
FireboltEngineClass template as well as the resource’s own.
Network isolation between pods
The Firebolt Operator emits no NetworkPolicy objects. All pod-to-pod and pod-to-external traffic is governed by whatever the cluster’s CNI and NetworkPolicy controller already enforce. In a default Kubernetes install with no NetworkPolicy controller installed, every pod can reach every other pod on every port. This is a deliberate scoping decision: NetworkPolicy semantics depend on the CNI plugin (Calico, Cilium, Antrea, etc.), the cluster’s default-allow-vs-default-deny posture, and the Firebolt Operator-vs-platform ownership boundary for security primitives. Encoding any of those assumptions into Firebolt Operator-emitted NetworkPolicies would either be a no-op (no controller installed) or actively wrong for the deployment target. Platform teams should apply NetworkPolicies covering at least the allowed flows below. Recommended selectors:Allowed flows
Engine-to-engine, engine-to-PostgreSQL, gateway-to-metadata, and
gateway-to-PostgreSQL are not required by any Firebolt Operator-managed
control flow and should be denied.
Example baseline NetworkPolicy
The snippet below denies all ingress and egress by default in the instance’s namespace, then re-allows the flows above. It assumes the gateway’s external clients live in afirebolt-clients namespace. The
selector should be adjusted to match the actual client topology.
Namespace-level resource ceilings
AResourceQuota capping the aggregate of requests.cpu /
requests.memory / pod count / PVC size across all engines in a
namespace is a platform concern. The Firebolt Operator enforces per-engine
upper bounds in admission (see below) but does not emit a
ResourceQuota. The per-namespace budget is a deployment-target
decision (test cluster vs. multi-tenant production).
Image provenance and supply-chain attestation
The Firebolt Operator pulls whatever image the user supplies viaFireboltEngine.spec.template.spec.containers[engine].image, the
referenced FireboltEngineClass.spec.template.spec.containers[engine].image,
or the embedded defaults shipped with the Firebolt Operator binary
(merged in that order, top wins). The Firebolt Operator does not
validate signatures, attestations, or SBOMs. Use a cluster-level
admission controller (Kyverno, Sigstore Policy Controller, etc.) if
image-policy enforcement is required.
Resource bounds
The FireboltEngine validating webhook rejects engine-containerresources entries above Firebolt Operator-configured maxima. The
gate resolves the effective container the same way the reconciler
will: the engine’s own
spec.template.spec.containers[name=="engine"].resources wins
wholesale when set, otherwise the referenced FireboltEngineClass’s
container resources fill in. Both sources are checked, so a class
with oversized requests cannot escape admission by being referenced
from an engine that omits its own resources. The error message names
the source class when the offending value came from class so the
user knows which side to edit.
This is a defense-in-depth control against accidental
over-provisioning. A typoed 100Gi instead of 10Gi is caught at
admission rather than at scheduling time when it would silently
exhaust namespace capacity and block other engines.
The maxima are configurable at Firebolt Operator install time. Defaults are
sized for typical production deployments. Override via Helm values
when running larger or smaller engines.
Secrets handling
The Firebolt Operator generates one Secret: the internal PostgreSQL credentials (<instance>-metadata-postgres-creds). The password is
generated at first reconcile, persisted to a Kubernetes Secret with
owner reference to the FireboltInstance, and never re-rotated by the
Firebolt Operator. The metadata and PostgreSQL Deployments load it via
envFrom.
User-supplied credentials (external PostgreSQL, external object
store) are referenced by name on the FireboltInstance / FireboltEngine
spec and resolved at reconcile time. The Firebolt Operator never reads or
materializes a user-supplied secret’s value into its own status,
events, or logs.
When auth or TLS is enabled, these additional Secrets exist:
The operator reads a signing key’s public-key fingerprint to detect key
replacement, and a Secret’s
resourceVersion to detect an in-place password
change. Neither the private key nor the password value is ever written to
status, events, or logs.
See also
- architecture: Full Firebolt Operator architecture.
- monitoring: Metrics and alerts the Firebolt Operator exposes.
- instance-crd-reference: Firebolt Operator-owned vs. user-owned fields on FireboltInstance.
- fireboltengineclass-crd-reference: Firebolt Operator-owned vs. user-owned fields on FireboltEngineClass.