Working on infrastructure hardening?
I've spent the last several years on production environments where downtime is measured in lost revenue and bad audit findings. If your team is in that zone, I'm happy to look at your setup.
Case Studies
Five production stories
Five production stories I led end-to-end — from threat intel response to infrastructure hardening, architecture evolution, and performance engineering. Each entry is verified against a concrete artifact on production infrastructure, published gists, or public advisory pages.
Threat intel
1 case study
FBI-CISA joint advisory AA25-239A: lawful intercept as adversary entry point
mediumPRC-affiliated Salt Typhoon actor inside US telecom lawful intercept infrastructure demonstrates that a regulated mandatory backdoor does not distinguish who crosses it — the same door hostile nation-states walk through is the door law enforcement uses.
Intervention
I aligned my zero-trust architecture to NIST SP 800-207 logical components (PE, PA, PEP, EIG, MSG, SAG) so that lawful intercept scrutiny applies at the policy layer, not the network layer. Every transport boundary that an intercept request would cross now has an explicit Policy Enforcement Point: the kernel firewall (L4), nginx SSL termination (L7), Next.js route groups (L7 app), Postgres per-tenant grants (L8 data). The Policy Engine (proxy.ts) cannot be bypassed by a privileged-network caller — every request is authenticated and authorized regardless of source IP.
Outcome
Architecture decomposes 6 logical components per NIST SP 800-207; policy decisions enforced at 4 layers (network, transport, application, data); the VPN mesh trust boundary is audited separately from host root trust. The advisory's lesson is operationalized into a design rule: lawful intercept requirements must be modeled as additive PEPs, not as exceptions to authentication.
Stack
- NIST SP 800-207 Zero Trust Architecture
- CISA advisory AA25-239A (Aug 2025)
- VPN mesh + QUIC tunnel 5-tier fallback
- fail2ban + nftables-allports O(1) ban set
- Next.js route groups + proxy.ts middleware
- Postgres GRANT per role + per tenant
Evidence
salt-typhoonlawful-interceptcisa-aa25-239aprc-aptzero-trustnist-800-207defense-in-depth
Infrastructure hardening
1 case study
fail2ban (5 custom jails) + nftables-allports O(1) ban set on Debian 12
highA five-jail fail2ban pipeline emits bans into a single nftables `addr-set-*` IPv4 set family for O(1) drop lookup under credential-stuffing storms. The design point is that a banned IP is rejected at the kernel, not at the application, so the cost of a ban is constant regardless of attack volume.
Intervention
Replaced the legacy iptables with a hand-tuned nftables ruleset exposing one `table inet f2b-table` with named `addr-set-*` sets keyed by jail. fail2ban's `nftables-allports` action drops all ports per banned IP via single set membership check (O(1) instead of O(N)). Five custom jails layered on top of the Debian defaults: sshd hardening, edge transport, trusted-ssh (ignores the VPN mesh + 14 MX residential ranges), stricter-ssh, and the Debian defaults themselves. The recidive jail escalates repeat offenders to 90-day bans with no manual intervention.
Outcome
Ban lookup complexity: O(N) → O(1). Recidive horizon: 7-day findtime, 90-day bantime. Edge-transport coverage: previously unprotected listeners (QUIC, Hysteria, gost) are now under the same ban set as sshd. Trusted-ssh jail provides 14 MX residential ranges + VPN mesh bypass without relaxing the global default. The legacy ipset-persistent set is retained for rollback only.
Stack
- fail2ban 1.x
- nftables (kernel.x)
- systemd unit: fail2ban.service
- ipset-persistent (legacy, rollback only)
- iptables / ip6tables (VPN interface PostUp/PostDown only)
- VPN mesh
fail2bannftablesiptables-replacementrecidive-bangeo-blockcredential-stuffingo1-lookup
Architecture evolution
1 case study
K3s SaaS tier with triple isolation (NetworkPolicy + readonly user + DirsAllowed)
highThree-tier Virtuoso deployment (laptop authoring, host Tier 2, K3s Tier 3 SaaS) with NetworkPolicy default-deny, a sparql-readonly user with explicit SPARQL_UPDATE revoke, and Virtuoso DirsAllowed restricted to the publish volume. The triple isolation guarantees that a breach at any one layer (network, auth, filesystem) does not propagate.
Intervention
Deployed K3s (lightweight Kubernetes) cluster with the lzt-virtuoso-k3s pod in a dedicated namespace (`lzt-vusto` with restricted PodSecurity label). Triple isolation: (1) Network — NetworkPolicy with default-deny ingress, allowlist only ingress-nginx + same-namespace pods + DNS to kube-system + HTTPS to ghcr.io only (RFC1918 excluded). (2) Auth — `sparql-readonly` user with `GRANT SPARQL_SELECT ONLY` and explicit `REVOKE SPARQL_UPDATE, DBA`. Sync engine uses DBA credentials extracted from k8s secret for TTLP_MT bulk load (privileged ingest by design). (3) Filesystem — `virtuoso-publish.ini` with `DirsAllowed = /opt/virtuoso-publish` only (NOT `/opt/virtuoso-opensource/database`). Sync engine implements allowlist defense-in-depth: graphs matching `urn:lzt:graph:.*` allowed, `urn:loust:.*` / `urn:lzt:repo:.*` / `urn:lzt:identity:.*` denied (private data + identity leak protection).
Outcome
Three independent isolation layers (network / auth / filesystem) reduce blast radius if any single layer is breached. K3s tier-3 SaaS deployment staged with manifests validated, NetworkPolicy design approved, image build pending operator gate (E6). Routing layer integrates via `~/.config/lzt-hub/virtuoso-routing.toml` `saas-consumer` intent — single SaaS entry point by design, no fallback. Pre-helm-ization backups (U1 virtuoso.db 71M→4.1M gzip, U2 stalwart-postgres 721K gzip) executed 2026-07-12 as source-of-truth closure before PVC migration.
Stack
- K3s (lightweight Kubernetes)
- openlink/virtuoso-opensource-7 (Docker image base)
- Kubernetes NetworkPolicy (default-deny + allowlist)
- Virtuoso SPARQL ACLs (SPARQL_SELECT only for external clients)
- kustomize (manifest bundling for `kubectl apply -k`)
- cert-manager + letsencrypt (ingress TLS)
- nginx ingress controller
- kubectl cp / kubectl exec (sync engine transport)
Evidence
k3skubernetesvirtuosotriple-isolationnetworkpolicyrbac-sparqlsaas-tier
Performance engineering
1 case study
APQ + BatchHttpLink + CircuitBreaker at 90.9% cache hit rate over 135k-line schema
highApollo Client v4 Automatic Persisted Queries with sha256 hashing, BatchHttpLink, CircuitBreaker, and apollo3-cache-persist — replaces parse+validate overhead AND shrinks DoS attack surface to a known operation set, achieving 90.9% cache hit rate, p95 12 ms.
Intervention
Two-layer cache: (1) server-side Apollo Server v4 APQ with sha256-persisted query hash as cache key, backed by Redis 7 cluster with Lua EVAL atomic counters (replacing 3-RTT pipeline to eliminate TOCTOU races on circuit-breaker counters). (2) Client-side apollo3-cache-persist with IndexedDB + Service Worker layer keyed by `(operationName, variables hash)` for `cacheLife: 'hours'` / `'days'` profiles. BatchHttpLink batches multi-operation requests into single HTTP body. CircuitBreaker isolates backend failures. 5-phase auto-warmup pre-caches 2,130 queries at startup. Schema-hash invalidation handles Prisma model changes via re-introspection. Self-hosted compile-runner isolates schema builds via cgroup v2 slices (CPUWeight=400, MemoryHigh=12G, MemoryMax=16G, IOWeight=200) so build-time spikes do not OOM production traffic.
Outcome
Hit rate: 90.9% in production. p95 latency: 12 ms (vs 25 ms without APQ, 52% reduction). Payload reduction: 75% on cached queries. Throughput: +125% (200 → 450 req/s). Schema size: 135,504 lines, 2,089 query types, 707 Prisma models. Auto-warmup: 2,130 queries pre-cached at boot. External spend: $0/month (no Apollo Studio / Hasura / Stellate dependency). Runner isolation via cgroup v2 prevents build OOM from cascading into production Next.js workers.
Stack
- Apollo Client v4 (BatchHttpLink + InMemoryCache + apollo3-cache-persist)
- Apollo Server v4 (APQ plugin with sha256 hashing)
- Redis 7 (Lua EVAL atomic counters + persisted query cache)
- Prisma 7.x (707 models, schema introspection)
- Next.js 16 cache components ('use cache' + cacheLife + cacheTag + revalidateTag)
- IndexedDB + Service Worker (browser layer)
- cgroup v2 slices (compile-runner isolation)
- Self-hosted compile-runner (replaces GitHub Actions for schema build)
Evidence
- https://gist.github.com/louzt/64715cb9c6ec6ffdd98c5712b8fb7bac (APQ English, 1336 lines, v2.1 EN)
- https://gist.github.com/louzt/0c91771bf6370f0eb47e905934ab47e8 (APQ Español, 1390 lines, v2.1 ES)
- https://www.apollographql.com/docs/apollo-server/performance/apq
- https://github.com/apollographql/apollo-cache-persist
apqpersisted-queriesgraphqlrediscircuit-breakerdos-mitigationsha256cgroup-v2performance-engineering
Compliance
1 case study
Multi-tenant RBAC + zero-trust across socialspheremx, fuerzaparaseguir, loust.pro
highPer-tenant Postgres database + Redis key-prefix isolation + Next.js route groups (authed) + RequireRole component guarding subtrees — three production tenants on shared infrastructure with no cross-tenant data access path.
Intervention
I built three layers of isolation: (1) Postgres per-tenant database with explicit DATABASE_URL per PM2 process. (2) Redis key prefix per tenant via shared client with the `prefix` option. (3) Next.js route groups `(authed)` with a separate layout for authenticated chrome and a `RequireRole` component guarding individual subtrees. Roles: ADMIN, EDITOR, AUTHOR, AGENT, SUPERADMIN. The admin header button shows the role badge plus a dropdown with role-specific quick actions. Tenant is resolved via Host header on direct port access. Zero-trust applies at every layer: PE (Policy Engine) lives in Next.js middleware (proxy.ts); PA (Policy Administrator) is the AdminHeaderButton + RequireRole tree; PEPs at the kernel firewall (L4), nginx SSL termination (L7), Next.js route groups (L7 app), Postgres per-tenant grant (L8 data); EIG at the Cloudflare edge; MSG aggregates the transport stack into a single observable mesh; SAG is the alert-digest pipeline.
Outcome
Three independent isolation layers (data, cache, render) close cross-tenant leakage paths. Zero-trust architecture decomposes to 6 NIST SP 800-207 logical components. Per-tenant Postgres grants prevent cross-tenant data access even on shared infrastructure. The Postgres Shadow Sync replicates per-tenant DB at 300-second intervals for analytics without affecting production. The SSR sidebar flash is eliminated via the dedicated (authed) layout boundary. The login endpoint is named /api/portal/auth (not /api/portal/login) — Host header resolves tenant at the edge.
Stack
- Next.js 16 route groups + (authed) layouts
- Prisma 7.x with per-tenant DATABASE_URL
- NextAuth (session provider in root layout)
- Redis with key prefix isolation per tenant
- bcrypt (Node ABI 141) for password hashing
- PM2 fork-mode processes per tenant (ecosystem.config.js)
- NIST SP 800-207 Zero Trust Architecture
- Cloudflare Universal SSL (EIG tier)
Evidence
rbacmulti-tenantnext.jspostgres-isolationredis-prefixroute-groupszero-trustnist-800-207