{
  "questions": [
    {
      "domain": "Introduction",
      "question": "Give me a 1 minute run down of your background and how you evolved into the role you're in today?",
      "notes": "Helps set length of answer expectations, and also gives the candidate to warm up before you dive in to deeper technical questions.",
      "followup_questions": []
    },
    {
      "domain": "Introduction",
      "question": "What do you understand this role to be, and why does that interest you?",
      "notes": "Take the opportunity to briefly elaborate on areas they may be overlooking to ensure they get an accurate view of the role.",
      "followup_questions": []
    },
    {
      "domain": "Application Development Expertise",
      "question": "You're building a new web application for a customer. Walk me through how you'd choose a framework -- what factors matter, and what are the trade-offs between the options you'd consider?",
      "notes": "Strong answers: Start with requirements (team expertise, performance needs, ecosystem, long-term support), then compare 2-3 frameworks they've actually used (e.g., Django vs Express vs Next.js). Listen for honest trade-offs (e.g., 'Django ORM is fast for internal tools but monolithic'; 'Express is minimal so you assemble your own stack'). Red flag: generic feature lists from docs with no personal experience, or can't name a trade-off. Follow up on production pain points they've hit.",
      "followup_questions": [
        {
          "question": "Tell me about a time you regretted a framework choice, or inherited one that caused problems. What happened?",
          "notes": "Probes real-world experience. Strong: specific story about breaking upgrades, performance issues at scale, or plugin conflicts. Weak: hypothetical or vague 'it was slow'."
        }
      ],
      "level_guidance": {
        "100": "Names frameworks without explaining selection criteria.",
        "200": "Considers team expertise, ecosystem, performance characteristics. Can compare 2-3 frameworks with real trade-offs. Has shipped with at least one.",
        "300": "Framework decision matrix: considers long-term maintenance, community health, hiring market, deployment model (serverless compatibility), testing ecosystem, performance at scale. Discusses when to NOT use a framework (micro-services, static sites).",
        "400": "Architectural reasoning: framework as an organizational decision (standardization vs freedom), migration costs when frameworks age, the build-vs-buy spectrum (framework vs library vs custom), and how framework choice constrains future architectural options."
      }
    },
    {
      "domain": "Network Expertise",
      "question": "What is a CIDR block?",
      "notes": "CIDR (Classless Inter-Domain Routing) replaces the old class A/B/C network-classes model with a prefix-length notation like 10.0.0.0/24. The '/24' says the first 24 bits are the network portion, leaving 8 bits for hosts = 256 addresses (254 usable). Good answers mention: VLSM (variable-length subnet masking) so you can right-size subnets, how to split a /16 into multiple /24s, and why /28 is the smallest practical subnet on AWS (5 reserved addresses per subnet). Red flag: can't compute the number of hosts in a /22 (1022) or explain why contiguous CIDR blocks are easier to route.",
      "followup_questions": [],
      "level_guidance": {
        "100": "Knows CIDR is for IP-address blocks. Can read /24 as 'a network of 256 addresses' (with help).",
        "200": "Computes CIDR ranges from prefix length, knows /32 vs /24 vs /16 by heart for IPv4. Plans VPC subnets to avoid overlap. Mentions IPv6 /64 default.",
        "300": "Plans real address space: non-overlapping VPC ranges across accounts, room for future growth, separate subnets per AZ + tier (public/private/database), NAT-gateway placement. Knows about RFC 1918 spaces and which to pick to avoid corp-network collisions.",
        "400": "Reasons about address-space economics: the cost of running out (forced renumbering is painful), summary routes through a transit gateway, IP exhaustion as a real constraint at megacorp scale (cgNAT, IPv6 strategy), peering address-overlap mitigation (PrivateLink), and the deeper insight that bad addressing decisions in year 1 compound into year-5 architecture pain."
      }
    },
    {
      "domain": "Storage Expertise",
      "question": "I have an application on a server that is I/O bound (i.e. I/O is the bottleneck.)  What can I do to add more I/O capacity to the server?",
      "notes": "Layered answer: (1) First measure — iostat, iotop, fio — confirm whether it's IOPS or throughput limited, sequential vs random, read vs write. (2) Quick wins: add RAM (larger page cache reduces disk reads), tune filesystem (noatime, journal mode), use faster storage tier (NVMe instead of SATA SSD, SSD instead of HDD). (3) Architectural: RAID 0/10 to stripe across multiple disks (more IOPS in parallel), add read replicas, introduce a cache layer (Redis, Memcached), use a CDN for static assets, offload writes to a queue. (4) Cloud-specific: AWS io2 Block Express for highest IOPS, gp3 for tunable IOPS/throughput independent of size, EFS vs FSx for shared storage. Strong answers start with 'measure first' — red flag is jumping straight to 'add more disks'.",
      "followup_questions": [],
      "level_guidance": {
        "100": "Suggests 'add a faster disk'.",
        "200": "Distinguishes IOPS vs throughput (MB/s) vs latency. Picks SSD over HDD for random-IO workloads. Mentions RAID 0/10 for striping, RAID 1 for redundancy.",
        "300": "Diagnoses with iostat / iotop, identifies hot files, picks the right storage class (gp3 / io2 / NVMe instance store), tunes filesystem mount options (noatime, barrier=0 with caveats), considers app-level batching / async I/O / write coalescing.",
        "400": "Reasons about storage as a system design choice: write amplification on copy-on-write filesystems, queue-depth tuning, kernel I/O scheduler choice (mq-deadline / kyber), io_uring for high-throughput async paths, NVMe namespace partitioning, and the deeper insight that adding I/O capacity is often the wrong fix — fixing the I/O pattern (batch, cache, denormalize) usually wins."
      }
    },
    {
      "domain": "Infrastructure Expertise",
      "question": "In a datacenter/on-premise environment: what are the minimum requirements for making a website available on the internet?",
      "notes": "Core checklist: (1) A server (physical or VM) with a web server (Apache, Nginx, IIS), (2) a public IP address (static, routable), (3) a registered domain name with DNS A/AAAA records pointing to that IP, (4) firewall rules allowing inbound 80/443 from the internet, (5) a routable network path — upstream router with a default route to the ISP, (6) for HTTPS: a valid TLS certificate (Let's Encrypt or commercial CA). Bonus: load balancer for HA, CDN for global reach, WAF for protection, monitoring for uptime. Strong candidates walk through the request lifecycle (browser -> DNS -> router -> firewall -> server -> app) to prove they understand the full stack. Red flag: forgets DNS, or thinks a private IP works from the public internet.",
      "followup_questions": [],
      "level_guidance": {
        "100": "Names 'a server' and 'a domain name'. May not connect them.",
        "200": "Walks through it: register domain → set up authoritative DNS → run a web server (nginx/Apache) on a public IP → open port 80/443 in the firewall → install a TLS cert. Mentions a load balancer for redundancy.",
        "300": "Builds the resilient version: redundant web servers behind a load balancer, automated TLS with Let's Encrypt or ACM, a CDN in front for static content, monitoring + log shipping, capacity planning for traffic spikes, runbook for cert renewal failures.",
        "400": "Reasons about the deeper questions: anycast for global presence, BGP for multi-homed reliability, DDoS protection (CloudFront + Shield Advanced or Cloudflare), origin shielding, multi-region failover with health-checked DNS, observability budget (tracing + RUM), and the deeper insight that 'available' is a SLO with a cost — pick the SLO before the architecture."
      }
    },
    {
      "domain": "Database Expertise",
      "question": "Security of data is always important within a database, discuss for me the layers of security that are available and the level of protection they represent.",
      "notes": "Defense in depth: (1) Network — private subnets, no public IPs, security groups restrict to app tier only. (2) Authentication — strong DB passwords or, better, IAM-based auth (RDS IAM auth, Aurora), rotate via Secrets Manager. (3) Authorization — least-privilege DB users (app user has only the grants needed; no SELECT * on all tables from the app), separate admin and app accounts. (4) Encryption at rest — KMS-encrypted storage/snapshots (transparent, free performance-wise on modern hardware). (5) Encryption in transit — TLS required on connections. (6) Audit logging — who queried what (pgaudit for Postgres, MySQL audit plugin, CloudTrail for control plane). (7) Application-layer — parameterized queries to prevent SQLi, input validation, least-data-exposure at the ORM. (8) Data-level — column-level encryption for PII, tokenization for PAN, masking for non-prod clones. Strong answers note that each layer stops a different attack class and that any single control is insufficient.",
      "followup_questions": [],
      "level_guidance": {
        "100": "Names 'encryption' as one mechanism. Maybe mentions strong passwords.",
        "200": "Layered model: encryption at rest (TDE), encryption in transit (TLS), authentication, authorization (least-privilege grants), audit logging. Knows about parameterized queries to prevent SQL injection.",
        "300": "Has implemented all layers in production: row-level security or column-level masking for PII, IAM authentication where the engine supports it (IAM DB auth on RDS, AzureAD on Azure SQL), encryption with customer-managed KMS keys, automated credential rotation via Secrets Manager, query-log retention for forensics.",
        "400": "Reasons about the threat model end-to-end: insider risk (DBA can read everything → field-level encryption with app-side keys), backup security (encrypted snapshots in a separate account), break-glass auditability, post-quantum readiness for long-lived encrypted data, and how each layer fails (e.g., TDE doesn't protect against a compromised app, TLS doesn't protect against query-log leaks)."
      }
    },
    {
      "domain": "Big Data / Analytics",
      "question": "Imagine you have a large log file showing visitors to your customer facing website. What methods would you use to filter out all entries coming from within your company, or to view only visits coming from outside your company?",
      "notes": "Depends on scale: (1) Small log (< few GB): `grep -v` with a regex of your known CIDR blocks, or awk/sed for more complex filtering. (2) Medium: load into Athena/BigQuery, WHERE client_ip NOT IN (internal CIDRs). (3) Production at scale: do this in the ingestion pipeline — tag each log with an 'internal/external' field based on IP range lookup against a maintained CIDR list, then dashboards filter on the tag. For reliable IP-to-company matching: IP ranges change (VPN, mobile hotspots), so also filter by known user IDs (employees sign in with corp SSO), and cookie flags set by company proxy. Strong answers mention: store logs in a format that supports query pushdown (Parquet, ORC on S3) so scans are cheap; set up Athena/Glue catalog; use CloudFront IP lists as a reference. Red flag: only suggests grep without thinking about data volume.",
      "followup_questions": [],
      "level_guidance": {
        "100": "Suggests grep or manual inspection.",
        "200": "Data pipeline approach: parse logs, aggregate by visitor/page/time, use SQL or pandas for analysis. Knows about log formats (Apache, JSON). Mentions CloudWatch Logs Insights.",
        "300": "Designs analytics pipeline: ingestion (Kinesis/Firehose), storage (S3 partitioned by date), processing (Athena for ad-hoc, Glue for ETL), visualization (QuickSight). Considers: log volume, query patterns, cost optimization (partitioning, columnar format), real-time vs batch needs.",
        "400": "Big data architecture: streaming analytics (Kinesis Analytics for real-time), data lake design (raw -> curated -> aggregated zones), schema evolution, data quality monitoring, and the organizational decision of build (custom pipeline) vs buy (managed services like OpenSearch) based on query patterns and team expertise."
      }
    },
    {
      "domain": "Security",
      "question": "In cryptography, what is the difference between symmetric algorithms and asymmetric algorithms?",
      "notes": "Symmetric: same key encrypts and decrypts (AES is the standard). Fast — can encrypt gigabits/sec — but the key must be shared secretly, which is the distribution problem. Asymmetric (public-key): a key pair where public encrypts and only private decrypts (RSA, Elliptic Curve). Solves distribution because the public half can be shared openly, but it's 100-1000x slower than symmetric. In practice nothing is encrypted with only asymmetric for bulk data — TLS uses asymmetric to securely exchange a symmetric AES session key, then encrypts all the traffic with AES. Strong answers name AES-256-GCM as the modern symmetric default, RSA-4096 or ECDSA-P256 for asymmetric, and note that ECC gives equivalent strength to RSA at much smaller key sizes.",
      "followup_questions": [],
      "level_guidance": {
        "100": "Knows symmetric uses one key, asymmetric uses two. Names AES and RSA but might mix up which is which.",
        "200": "Explains the use cases: symmetric for bulk encryption (AES-256-GCM), asymmetric for key exchange + signatures (RSA, ECDSA, Ed25519). Knows TLS uses both — asymmetric to negotiate a session key, symmetric for the data.",
        "300": "Picks deliberately: ECDSA P-256 over RSA-2048 for new code, AES-GCM over CBC, X25519 for key agreement. Discusses key rotation, HSM-backed keys, why DH/ECDH gives forward secrecy.",
        "400": "Reasons about post-quantum migration (ML-KEM / Kyber for KEX, ML-DSA / Dilithium for signatures), hybrid PQ-classical schemes, side-channel attacks on naive implementations, formal verification of crypto primitives, and the engineering cost of doing crypto correctly vs delegating to a managed service (KMS / HSM)."
      }
    },
    {
      "domain": "Content Delivery",
      "question": "Without using caching, what are some ways to speed up delivery of assets to end users from a web server?",
      "notes": "(1) Compression: gzip or Brotli on text (HTML/CSS/JS/JSON) — 60-80% size reduction; Brotli is better for modern browsers. (2) Minification: strip whitespace/comments from JS/CSS; tools: Terser, esbuild, Parcel, Vite. (3) Image optimization: serve modern formats (WebP, AVIF), correctly sized (srcset for responsive), lazy-load with `loading=\"lazy\"`. (4) HTTP/2 or HTTP/3: multiplexing lets many assets share one connection — eliminates head-of-line blocking, reduces handshake overhead. (5) TLS optimization: OCSP stapling, session resumption, TLS 1.3 (1-RTT handshake). (6) Code splitting: only ship the JS needed for the current page. (7) Prefetch/preload hints. (8) Efficient font loading: woff2, font-display: swap, self-host. Strong answers cover both bytes-on-the-wire and round-trips. Red flag: jumps to 'add a CDN' (that's caching) or stops at 'minify'.",
      "followup_questions": [],
      "level_guidance": {
        "100": "Names a CDN or says 'use a faster server'. No architectural thinking.",
        "200": "Covers: CDN/edge caching, image optimization (compression, responsive images), minification, HTTP/2 multiplexing, reducing DNS lookups. Knows CloudFront basics.",
        "300": "Designs delivery architecture: origin shield for cache efficiency, Lambda@Edge for dynamic content, connection reuse, prefetching/preloading strategies, async loading patterns, geographic routing (Route 53 latency-based). Measures with RUM.",
        "400": "Reasons about delivery as a system: content pipeline automation, edge computing for personalization without round-trips, protocol optimization (HTTP/3 QUIC), TCP optimization (congestion control tuning), cache invalidation strategies at scale, and the trade-off between freshness and performance."
      }
    },
    {
      "domain": "Performance Troubleshooting/Tuning",
      "question": "I run a web site, my users are complaining it is slow. I just hired you as my consultant, walk me through troubleshooting my site.",
      "notes": "A good candidate will dive deep asking what in particular is running slow.",
      "followup_questions": [],
      "level_guidance": {
        "100": "Says 'check the server'. Doesn't have a methodology.",
        "200": "Has a methodology: reproduce → bisect (frontend? network? backend? DB?) → narrow down with metrics or logs → form hypothesis → fix → verify. Mentions Real User Monitoring + browser DevTools.",
        "300": "Drives it with data: per-stage latency breakdown (DNS / TLS / TTFB / DOM-ready / interactive), p99 vs median, server-side flame graphs, DB slow-query log, CDN cache-hit rate, third-party blocking. Distinguishes user-perceived from server-perceived perf.",
        "400": "Reasons about a perf-tuning culture: SLOs anchor 'what is slow?', error budgets guide invest-vs-ship decisions, performance budgets at PR time stop regressions, p99 latency vs tail-amplification in fan-out architectures, and the deeper insight that 'slow' is a product question first, an engineering question second — without a target, tuning is infinite."
      }
    },
    {
      "domain": "Application Development Expertise",
      "question": "If you were to design a distributed and scalable system, what are the key issues to consider in this design and why?",
      "notes": "Not having a single point of failure, looking for decoupling, asynchronous, possibly event driven",
      "followup_questions": [],
      "level_guidance": {
        "100": "Mentions load balancers and caching. May say 'use AWS'.",
        "200": "Considers stateless services, database sharding, caching layers (CDN + app-level), async processing via queues, horizontal scaling. Mentions monitoring.",
        "300": "Designs for failure: redundancy in every tier, health checks + auto-recovery, retry-with-backoff, idempotent APIs, region-aware routing, capacity planning, blue/green or canary deploys, chaos testing.",
        "400": "Reasons about the deep design questions: CAP and which side to favour per feature, consistency boundaries via aggregates, eventual-consistency UX patterns, data-locality vs query-flexibility, multi-region active-active vs active-passive, the economic cost of each 9 of availability, and the org-design implications (Conway's Law: distributed systems mirror the team that built them)."
      }
    },
    {
      "domain": "Application Development Expertise",
      "question": "Can you talk to me about how you optimize your code for performance?",
      "notes": "They should talk about benchmarking, big O notation",
      "followup_questions": [
        {
          "question": "What tools or techniques do you use?",
          "notes": "Expect: (1) Profilers — language-specific (py-spy/cProfile for Python, Chrome DevTools + Flamegraph for JS, pprof for Go, async-profiler/JFR for Java). (2) APM tools — Datadog, New Relic, OpenTelemetry + Jaeger/Zipkin for distributed tracing. (3) Benchmarks — JMH for Java, pytest-benchmark, hyperfine for CLIs, k6/Locust for load tests. (4) Observability signals — latency histograms, p50/p95/p99, not averages. (5) Techniques — measure first (never optimize blindly), identify the hot path with profiling, look for algorithmic wins (O(n²)→O(n)) before micro-optimizing, check I/O and locking before CPU. Strong candidates say 'I'd measure before changing anything' — that's the right instinct."
        }
      ],
      "level_guidance": {
        "100": "Mentions 'make it faster' without specific techniques.",
        "200": "Covers: profiling before optimizing, algorithmic complexity (Big-O), caching, database query optimization, reducing network calls, lazy loading. Knows 'premature optimization is the root of all evil.'",
        "300": "Systematic approach: measure first (profiling tools per language), identify hotspots (usually I/O not CPU), optimize in order of impact. Techniques: connection pooling, async/non-blocking I/O, batch operations, memoization, data structure selection. Knows platform-specific tools (JProfiler, cProfile, Chrome DevTools).",
        "400": "Performance engineering: benchmarking methodology (micro vs macro, warm vs cold), understanding hardware (CPU caches, branch prediction, memory hierarchy), concurrency patterns (lock-free data structures, actor model), and the organizational practice of performance budgets, regression testing, and SLO-driven optimization prioritization."
      }
    },
    {
      "domain": "Application Development Expertise",
      "question": "Do you have any experience with web or application servers?",
      "notes": "IIS, Apache, Tomcat, Jetty, JBoss, Nginx, Unicorn, etc.",
      "followup_questions": [
        {
          "question": "Tell me some ways you'd stop/start or configure a webserver.",
          "notes": "Linux: systemctl for service management (`systemctl start nginx`, `systemctl enable nginx`, `systemctl reload nginx` for graceful config reload vs `restart` which drops connections). Config lives in /etc/nginx/ or /etc/apache2/, use `nginx -t` to test syntax before reloading, `apachectl configtest` for Apache. In containers: the container IS the process manager — restart = restart the container. In k8s: change the config via ConfigMap + rolling restart of the pods. Reload vs restart matters: `reload` re-reads config with zero dropped connections; `restart` drops active requests. Strong answers mention graceful shutdown (SIGTERM → finish in-flight → exit), zero-downtime deploys via load-balancer drain, and config-as-code (Ansible, Terraform, or container images) instead of manual edits on the server. Red flag: 'ssh in and vi the conf, then killall nginx' — not prod-safe."
        }
      ],
      "level_guidance": {
        "100": "Names Nginx or Apache. Limited understanding of purpose.",
        "200": "Explains: web server (static content, reverse proxy, TLS termination) vs application server (runs business logic, manages sessions). Examples: Nginx/Apache as web servers; Tomcat, Gunicorn, Node.js as app servers. Knows common deployment: Nginx reverse proxy -> app server.",
        "300": "Designs serving architecture: load balancing strategy (ALB vs NLB), connection management (keep-alive, HTTP/2), worker/thread model selection, graceful deployments (rolling, blue-green), health checks, auto-scaling based on request queue depth.",
        "400": "Reasons about server architecture evolution: traditional servers vs serverless (Lambda), container-based serving (ECS/Fargate), edge computing (CloudFront Functions, Lambda@Edge), and the architectural decision tree of where to terminate TLS, manage sessions, and handle routing in modern distributed systems."
      }
    },
    {
      "domain": "Application Development Expertise",
      "question": "Explain how development changes get moved into production – or – What is Continuous Integration and Continuous Deployment (CI/CD) in the software development process?",
      "notes": "CI (Continuous Integration): every commit is automatically built and tested — unit tests, linting, static analysis — on a shared branch, catching breakage within minutes of introduction. CD (Continuous Delivery): every passing build is deployable to prod at any time (someone clicks a button). CD (Continuous Deployment — same letters, different D): passing builds auto-deploy to prod with no human gate. Strong answers describe the pipeline: commit -> build -> unit test -> integration test -> security scan (SAST, SCA, container scan) -> deploy to staging -> smoke/e2e test -> deploy to prod (canary/blue-green/rolling) -> monitor -> auto-rollback on SLO breach. Mentions: feature flags to decouple deploy from release, trunk-based development vs gitflow, and that CI/CD reliability matters more than speed (flaky tests erode trust). Red flag: 'we just SSH in and pull' or no tests in the pipeline.",
      "followup_questions": [],
      "level_guidance": {
        "100": "Knows CI runs tests on every commit and CD pushes to production. Names a tool (GitHub Actions, Jenkins).",
        "200": "Has set one up: branch protection, automated tests on PR, build artifact promotion, manual approval before prod. Mentions semantic versioning + changelogs.",
        "300": "Has run CD at scale: trunk-based development with feature flags, canary deploys with automated rollback on metric regression, blue/green for stateful services, separate per-env config in SSM/Secrets Manager, fast feedback (build < 10 min, deploy < 5 min).",
        "400": "Reasons about DORA metrics (deploy frequency, lead time, change failure rate, MTTR) as the top-line health indicator, the org-culture prerequisites (trust in tests, blameless post-mortems), supply-chain hardening (signed artifacts, SBOMs, policy-as-code), and the deeper insight that CI/CD is a cultural problem first and a tooling problem second."
      }
    },
    {
      "domain": "Application Development Expertise",
      "question": "A customer wants their application to work well on both desktop and mobile. Walk me through how you'd architect that -- what decisions need to be made early, and what are the trade-offs?",
      "notes": "Key considerations: (1) Responsive design -- fluid layouts with CSS Grid/Flexbox, breakpoints for tablet (768px) and desktop (1280px); mobile-first approach starts from 375px. (2) Touch vs mouse -- minimum tap targets (44px), hover states need alternatives, gesture support. (3) Performance -- mobile networks are slower, so lazy loading, image optimization (WebP/AVIF, srcset), code splitting matter more. (4) PWA vs native vs responsive web -- trade-offs around offline capability, push notifications, app store presence. (5) State management across screen sizes -- what to show/hide vs restructure. Strong answers discuss the decision framework, not just list techniques.",
      "followup_questions": [],
      "level_guidance": {
        "100": "Mentions 'responsive design' without specifics.",
        "200": "Covers: responsive layouts (Grid/Flexbox, breakpoints), touch vs mouse interaction design, performance optimization for mobile (image compression, code splitting), PWA considerations.",
        "300": "Designs cross-platform strategy: mobile-first approach, performance budgets per device class, feature detection vs user-agent sniffing, offline capability (service workers), testing matrix (devices x browsers), accessibility across form factors.",
        "400": "Platform strategy reasoning: PWA vs native vs responsive trade-offs at business level, the economics of maintaining multiple codebases, emerging patterns (React Native, Flutter for cross-platform), and designing for the constraint (weakest device defines the baseline, progressively enhance for powerful ones)."
      }
    },
    {
      "domain": "Application Development Expertise",
      "question": "Tell me about a time you had to set up or improve version control practices for a team. What system did you choose and why? What branching strategy did you use?",
      "notes": "Strong answers: Describe a real decision (migrating from SVN to Git, choosing between GitHub/GitLab/CodeCommit, implementing GitFlow vs trunk-based development). Should discuss WHY -- team size, release cadence, CI/CD integration, code review workflows. Probe for: merge conflict resolution strategies, protecting main branch, handling hotfixes. Red flag: can only name tools but can't explain branching strategies or when to use them.",
      "followup_questions": [
        {
          "question": "What would you do differently if the team was 50 engineers vs 5?",
          "notes": "Tests scaling thinking. Large teams need trunk-based development + feature flags vs. GitFlow which works for smaller teams with longer release cycles."
        }
      ],
      "level_guidance": {
        "100": "Names Git. Limited understanding of branching or workflow.",
        "200": "Explains branching strategies (GitFlow, trunk-based), understands PRs/code review, knows basic Git operations. Can set up a reasonable workflow for a small team.",
        "300": "Designs team workflow: branching strategy matched to release cadence, CI/CD integration (tests on PR, deploy on merge), branch protection rules, mono-repo vs multi-repo trade-offs, dependency management across repos.",
        "400": "VCS at organizational scale: mono-repo strategies (Google/Meta approach), code ownership models (CODEOWNERS), automated merge queues, release engineering (release branches vs feature flags), and the cultural aspects of code review (speed vs thoroughness, review load distribution)."
      }
    },
    {
      "domain": "Application Development Expertise",
      "question": "What is a microservices design and what are its benefits?  Are there any drawbacks?",
      "notes": "I'm looking for \"interfaces\" (contracts established / apis), loose coupled, distributed, stateless, autonomous.",
      "followup_questions": [
        {
          "question": "What is the advantage or purpose of this design?",
          "notes": "(Microservices parent.) Benefits: (1) Independent deployment — each service ships on its own schedule, no coordinated monolith release. (2) Team autonomy — each team owns a service end-to-end, picks its own tech stack (within guardrails). (3) Fault isolation — one service crashing doesn't take down the whole app (if you've built circuit breakers). (4) Scale individually — only the hot service gets more capacity, not the entire app. Costs to acknowledge: distributed-systems complexity (network failures, partial failures, eventual consistency), harder debugging (distributed tracing is mandatory), operational overhead (deploys, monitoring, service discovery). Strong answers note that microservices are NOT a default — they pay off when you have enough engineers (say, 50+) and distinct domains. For small teams a well-modularized monolith is usually faster and more reliable. Red flag: treats microservices as always-better."
        }
      ],
      "level_guidance": {
        "100": "Knows microservices are 'small services that do one thing'. Says they're better than monoliths.",
        "200": "Names benefits: independent deployment, language flexibility, fault isolation. Names drawbacks: network complexity, distributed transactions, observability cost.",
        "300": "Picks microservices vs monolith based on team size and domain boundaries. Knows DDD bounded contexts as the right decomposition unit. Has run service-to-service auth, distributed tracing, contract testing, idempotent retries, circuit breakers.",
        "400": "Reasons about the deeper tradeoffs: 'distributed monolith' anti-pattern (microservices that share a DB), the cost of distributed transactions vs sagas, eventual consistency in user-facing flows, observability as a first-class capability, and the controversial-but-correct view that most teams should start with a modular monolith and only extract services when team org-design forces it."
      }
    },
    {
      "domain": "Application Development Expertise",
      "question": "What is a queue? What is an example use case?",
      "notes": "FIFO data structure (first-in-first-out) where producers push items onto one end and consumers pull from the other. Used to decouple producers and consumers so they can run at different rates and fail independently. Cloud implementations: AWS SQS (managed, at-least-once delivery, FIFO or standard), RabbitMQ, ActiveMQ, Redis lists. Use cases: (1) Order processing — web server pushes orders onto a queue, worker fleet processes at its own pace. (2) Email/notification sending — don't block the HTTP request. (3) Work distribution across many workers. (4) Buffer bursts so backing systems don't get overwhelmed. Strong answers mention: visibility timeout (message is invisible while being processed, reappears if not acknowledged), dead-letter queues for poisoned messages, message retention, and that queues should be treated as transport, not long-term storage. Bonus: contrast with pub/sub (SNS, Kafka) where multiple consumers each see every message.",
      "followup_questions": [
        {
          "question": "What is an example use case for a FIFO?",
          "notes": "Use cases where ordering matters: (1) Financial transactions — deposits must be processed before withdrawals on the same account. (2) Order fulfillment workflows — can't ship before payment clears. (3) Event sourcing — state is derived from replaying events in order. (4) User-facing notifications — messages arrive in the order sent. (5) Deduplication windows — FIFO queues (SQS FIFO) guarantee exactly-once processing within a 5-minute dedup window, not just ordering. Trade-off: Standard FIFO caps at 300 unbatched / 3,000 batched TPS per QUEUE — not per message group, which is a common misconception; with High Throughput for FIFO enabled, throughput scales per partition and reaches tens of thousands of TPS per queue in supported regions. Standard (non-FIFO) queues have effectively unlimited throughput., vs unlimited for standard. Strong answers know when you DON'T need FIFO — most workloads (email sending, metrics) are fine with at-least-once + eventual ordering, and paying for FIFO semantics is wasted cost. Red flag: picks FIFO for everything 'because ordering is good'."
        }
      ],
      "level_guidance": {
        "100": "Knows it's a FIFO data structure. May not connect to distributed systems.",
        "200": "Explains: ordered message buffer decoupling producers from consumers. Use cases: task distribution, load leveling, async processing. AWS: SQS (standard vs FIFO), dead-letter queues for failed messages.",
        "300": "Designs queue-based architecture: visibility timeout tuning, idempotent consumers, DLQ monitoring and replay, batching for throughput, long polling for cost, scaling consumers based on queue depth (ApproximateNumberOfMessages metric). Patterns: fan-out with SNS+SQS, priority queues.",
        "400": "Messaging architecture: queue semantics (at-least-once vs exactly-once and their real implications), back-pressure mechanisms, queue as the boundary between bounded contexts, event sourcing with queues, and the operational challenges (poison messages, queue depth spiraling, consumer lag monitoring) at scale."
      }
    },
    {
      "domain": "Application Development Expertise",
      "question": "What is continuous integration?",
      "notes": "CI is committing code several times per day where it is then built and \"integrated\" with other developer's code so that errors and bugs can be found quickly.",
      "followup_questions": [
        {
          "question": "Why would you want to use continuous integration?",
          "notes": "Core value: catch breakage within minutes of introduction, not hours or days. Specific benefits: (1) Merge hell reduction — short-lived branches + CI tests on every push makes merges trivial. (2) Confidence to refactor — strong test suite running on every change means you know quickly if you broke something. (3) Enables CD — you can't continuously deploy without continuous integration. (4) Forces test discipline — if CI doesn't pass, code doesn't merge, so people keep the tests passing. (5) Shared source of truth — 'it works on my machine' stops being an excuse. Secondary: metrics like lead time and change failure rate (DORA) improve measurably. Strong answers mention that flaky tests destroy CI's value — a test that fails 3% of the time is worse than no test because developers learn to re-run until green. Red flag: 'so we can deploy faster' without mentioning the quality feedback loop."
        }
      ],
      "level_guidance": {
        "100": "Knows CI means 'automated builds'. Limited depth.",
        "200": "Explains: developers merge code frequently, automated build and test on every commit, fast feedback on breakage. Tools: Jenkins, GitHub Actions, CodePipeline. Understands CI vs CD distinction.",
        "300": "Designs CI/CD pipeline: build -> unit test -> integration test -> security scan -> artifact publish -> deploy to staging -> integration/load test -> production deploy. Considers: pipeline speed (parallel stages), flaky test management, environment parity, rollback strategy, feature flags for decoupling deploy from release.",
        "400": "CI/CD as organizational capability: trunk-based development enabling continuous deployment, the investment in test infrastructure (fast, reliable, representative), progressive delivery (canary, blue-green, feature flags), deployment frequency as a DORA metric, and the cultural shift required (everyone owns the pipeline, no manual gates except for compliance)."
      }
    },
    {
      "domain": "Application Development Expertise",
      "question": "What is JSON commonly used for and what shortcomings of other languages does it address?",
      "notes": "JSON = JavaScript Object Notation. Used for: data interchange between systems (REST API payloads), config files, structured logging, NoSQL document storage (MongoDB, DynamoDB). Advantages over XML: much less verbose (no closing tags), directly maps to JavaScript/Python/most language's native data structures, faster to parse, supports arrays natively. Advantages over proprietary binary formats: human-readable, debuggable, language-agnostic. Limitations to note: no comments (strict JSON), no date type (dates are strings), no schema (use JSON Schema, OpenAPI, or move to Protobuf/Avro if you need one), no trailing commas, limited numeric precision (use strings for big integers). Strong answers mention JSON has largely replaced XML for web APIs but Protobuf/MessagePack beats both for high-throughput internal services where size and parse speed matter.",
      "followup_questions": [
        {
          "question": "For XML?",
          "notes": "(Parent: JSON shortcomings.) XML still wins for: (1) Mature tooling in enterprise (XSLT, XPath, SOAP, XML Schema/XSD for strict validation). (2) Mixed content (text + markup interleaved) — e.g., DocBook, OOXML, SVG — which JSON represents awkwardly. (3) Namespaces — multiple schemas can coexist cleanly (xmlns). (4) Digital signatures — XML-DSig is more mature than JSON Web Signatures in some regulated domains. JSON loses to XML in: strict schema validation, namespacing, and document-oriented formats. Strong answers note XML's heavy syntax (closing tags, namespace declarations) makes it verbose and slow to parse, which is why REST APIs almost universally moved to JSON. Most modern stacks use JSON + JSON Schema (when validation is needed) instead of XML + XSD. Red flag: says 'XML is bad' without acknowledging where it's still the right tool."
        }
      ],
      "level_guidance": {
        "100": "Knows JSON is for data. 'It's like XML but simpler.'",
        "200": "Explains: lightweight data interchange format, human-readable, language-agnostic, native to JavaScript. Advantages over XML: less verbose, easier to parse, maps naturally to objects/maps. Shortcomings: no schema enforcement (vs XML Schema), no comments, no date type, limited numeric precision.",
        "300": "Applied understanding: JSON Schema for validation, JSON vs Protocol Buffers/MessagePack for performance-critical paths, JSONL for streaming, JSON in databases (PostgreSQL JSONB, DynamoDB), API design with JSON (REST conventions, pagination, error formats).",
        "400": "Data format strategy: when JSON is insufficient (binary data, streaming, schema evolution) and alternatives (Avro for schema evolution in data pipelines, Protobuf for gRPC, Parquet for analytics), the versioning challenge with schemaless formats, and organizational standards for API data formats."
      }
    },
    {
      "domain": "Application Development Expertise",
      "question": "What is the purpose of a Publish and Subscribe pattern and what would be a good case to use it?",
      "notes": "Pub/sub decouples message producers (publishers) from consumers (subscribers) via an intermediary (the broker/topic). Publishers don't know who's listening; subscribers filter by topic. Key difference from queues: in queues, each message is delivered to ONE consumer; in pub/sub, each message is delivered to ALL subscribers (fan-out). AWS services: SNS (topic-based fan-out), EventBridge (content-based routing), Kafka (log-based, long retention, replay-able). Good use cases: (1) Event-driven architecture — service publishes 'OrderPlaced' event, subscribers include billing, inventory, email, analytics — each reacts independently. (2) Cache invalidation fan-out across regions. (3) Notifying multiple teams' systems of a central change. Strong answers mention: at-least-once delivery semantics, idempotency for consumers, and that pub/sub + queues compose well (SNS->SQS fan-out pattern on AWS).",
      "followup_questions": [],
      "level_guidance": {
        "100": "Knows it's about sending messages to multiple receivers.",
        "200": "Explains: publisher broadcasts events without knowing subscribers, subscribers receive events they're interested in. Decouples systems. AWS: SNS for pub/sub, with SQS subscribers for reliable delivery. Use cases: notifications, event-driven architectures.",
        "300": "Designs pub/sub architecture: topic structure strategy, filtering (SNS message attributes, EventBridge rules), fan-out patterns (SNS -> multiple SQS queues for different processing), ordering guarantees (SNS FIFO), cross-account subscriptions, DLQ for failed deliveries.",
        "400": "Event-driven architecture at scale: event schema governance (schema registry, compatibility rules), organizational event taxonomy, eventual consistency implications, event sourcing vs event notification patterns, and the operational complexity of debugging async distributed systems (distributed tracing, event replay for debugging)."
      }
    },
    {
      "domain": "Application Development Expertise",
      "question": "What is the purpose of a thread pool?",
      "notes": "A pre-created, reusable set of worker threads that accept tasks from a queue, reducing the overhead of spawning/destroying threads per request. Why it exists: thread creation is expensive (~100KB stack, kernel calls, TLB churn); reusing threads keeps latency low. Key tunables: (1) Core size (always-live threads), (2) max size (bursts), (3) queue capacity (what happens when all threads are busy — queue or reject), (4) idle timeout. Java's Executors, .NET's ThreadPool, Python's concurrent.futures, Golang's goroutines (not pooled, but cheap) all apply the concept. Pitfalls: exhausting the pool (all threads blocked on slow downstream) causes cascading latency — use bounded queues + fast-fail, not unbounded queues which just hide the problem. Strong answers separate CPU-bound work (pool size = cores) from I/O-bound (pool size much larger, or switch to async/event-loop). Red flag: 'just use a global Executors.newCachedThreadPool()' — that's unbounded and will OOM under load.",
      "followup_questions": [],
      "level_guidance": {
        "100": "May not know what a thread pool is.",
        "200": "Explains: pre-created set of threads waiting for work items, avoids overhead of creating/destroying threads per request. Controls concurrency (bounded pool prevents resource exhaustion). Examples: web server worker pools, database connection pools.",
        "300": "Design considerations: pool sizing (Little's Law: concurrent requests = arrival rate x response time), queue behavior when pool is exhausted (reject vs queue with timeout), monitoring (active threads, queue depth, rejection rate), tuning for workload type (CPU-bound vs I/O-bound -- different optimal sizes).",
        "400": "Concurrency architecture: thread pools vs event loops (Node.js model), virtual threads (Java 21 Loom), the relationship between thread pool size and downstream dependency capacity (cascading failures), and modern alternatives (async/await, reactive programming, actor model) that reduce the need for explicit thread pool management."
      }
    },
    {
      "domain": "Application Development Expertise",
      "question": "What steps would you take to troubleshoot a website error of \"Error establishing connection to database?\"",
      "notes": "Make sure username/password are correct, make sure appropriate port is accessible from the webserver to the database server (3306, 1433, etc), make sure database/database server is up, make sure user has appropriate permissions, etc.",
      "followup_questions": [],
      "level_guidance": {
        "100": "Suggests restarting the server or clearing cache.",
        "200": "Systematic debugging: check HTTP status code (500 = server error, look at logs), check connectivity (DNS, network), check application logs and error messages, check resource utilization (disk full, OOM), check recent deployments (rollback if recent change).",
        "300": "Structured incident response: reproduce the error, check monitoring dashboards (error rate trend -- sudden vs gradual), correlate with deployments/changes, examine distributed traces (X-Ray), check downstream dependencies, isolate scope (all users vs specific), communication while investigating.",
        "400": "Reliability engineering approach: error budgets and SLO-driven response, automated runbooks for common failures, chaos engineering to prevent unknown failure modes, postmortem culture, and designing systems to be debuggable (structured logging, correlation IDs, observability investment)."
      }
    },
    {
      "domain": "Application Development Expertise",
      "question": "What techniques do you use to insure that your code has a low bug rate and performs well under load?",
      "notes": "I really want to hear about the use of CI, Test Driven Development or automated testing (such as unit testing and more), and hopefully some form of performance testing.",
      "followup_questions": [],
      "level_guidance": {
        "100": "Mentions 'testing'. Limited methodology.",
        "200": "Covers: unit tests, code reviews, linting/static analysis, type systems, test-driven development. Knows test pyramid (many unit, fewer integration, few e2e).",
        "300": "Quality engineering approach: testing strategy (unit + integration + contract + e2e), code coverage as a guide not a target, mutation testing for test quality, static analysis in CI (SonarQube, CodeGuru), pair programming for complex logic, design patterns that make code testable (dependency injection, pure functions).",
        "400": "Quality at organizational scale: the economics of defect prevention vs detection (shift-left), formal methods for critical paths, property-based testing, fuzz testing for security, the culture of quality (not just tools), and measuring code health over time (technical debt metrics, defect escape rate)."
      }
    },
    {
      "domain": "Application Development Expertise",
      "question": "What's a three-tiered web application?",
      "notes": "Looking for web/application/database or presentation later, business layer, database layer, etc.",
      "followup_questions": [
        {
          "question": "How would you architect a three-tiered web application to limit downtime?",
          "notes": "(Parent: 3-tier app.) Same redundancy pattern applied at each layer: (1) Web/edge: CloudFront in front with multiple cache behaviors, origins in 2+ AZs, Route 53 health checks for failover. (2) App tier: ALB distributing to an auto-scaling group across 3 AZs, stateless so any instance can serve; container orchestrator (ECS, EKS) auto-replaces unhealthy containers. (3) DB tier: RDS Multi-AZ or Aurora (6-way storage replication, sub-minute failover), plus read replicas for read scale, cross-region replica for DR. Cross-cutting: (a) SQS between services to absorb back-pressure, (b) feature flags so a bad deploy can be toggled off instantly (LaunchDarkly, AppConfig), (c) CI/CD with canary or blue-green deploys — never big-bang to prod, (d) SLO-based alerting, runbooks for common failures, chaos engineering drills. Strong answers name a real outage pattern (single-AZ failure, thundering herd after deploy, DNS TTL causing stale routing) and how each mitigation prevents it."
        }
      ],
      "level_guidance": {
        "100": "Knows it has 'layers' -- maybe frontend and backend.",
        "200": "Explains three tiers: Presentation (web server, UI), Application/Logic (app server, business rules), Data (database). Benefits: separation of concerns, independent scaling, security isolation between tiers.",
        "300": "Designs on AWS: ALB -> ECS/Lambda (presentation + logic) -> RDS/DynamoDB (data), with ElastiCache between logic and data tiers. Discusses: scaling each tier independently, security groups between tiers, auto-scaling strategies per tier, and when to collapse tiers (serverless architectures blur the lines).",
        "400": "Architectural evolution: three-tier as a starting point that evolves (microservices decompose the logic tier, CDN + SPA eliminate the presentation tier from backend, serverless blurs all boundaries), and the trade-off analysis of traditional layered architecture vs modern patterns for different organizational contexts."
      }
    },
    {
      "domain": "Compute Expertise",
      "question": "When would you choose EC2 over Lambda over Fargate for running a workload, and what factors drive that decision?",
      "notes": "Looking for: cold start vs warm runtime, billing model (per-ms vs per-second vs per-hour), max execution time (15 min Lambda cap), persistent connections (EC2 wins), stateful workloads (EC2/Fargate), bursty short tasks (Lambda), container portability vs VM lock-in, scale-to-zero vs always-warm. A strong answer ties back to a real workload they've sized.",
      "followup_questions": [
        {
          "question": "How would you handle a 30-minute background job that exceeds Lambda's 15-minute limit?",
          "notes": "Step Functions chaining Lambdas, Fargate task, EC2 spot fleet, ECS task with EventBridge schedule, or breaking the job into shorter chunks via SQS."
        },
        {
          "question": "What's the cold start tradeoff with Lambda and how would you mitigate it for a latency-sensitive API?",
          "notes": "Provisioned Concurrency, SnapStart for Java, smaller packages, ARM (Graviton), avoid VPC unless needed, init outside handler, language choice (Node/Python warm faster than Java/.NET)."
        }
      ],
      "level_guidance": {
        "100": "Names the three services and can say at a high level that EC2 = VMs, Lambda = serverless functions, Fargate = serverless containers. May not be able to articulate when to pick which.",
        "200": "Picks between them on obvious criteria — Lambda for short bursty work, EC2 for long-running services, Fargate for containerised workloads. Knows the 15-min Lambda cap and per-ms billing.",
        "300": "Has actually run all three in production. Discusses cold-start mitigation (Provisioned Concurrency, SnapStart), persistent connections (EC2 wins for WebSockets / DB pools), VPC penalty on Lambda, Spot vs On-Demand on EC2, Fargate price premium vs operational savings.",
        "400": "Reasons about right-sizing across the three including cost-attribution at scale, references real workloads they've migrated between them, articulates failure modes (Lambda concurrency limits, EC2 instance retirement, Fargate task placement constraints) and how their architecture absorbed each."
      }
    },
    {
      "domain": "Compute Expertise",
      "question": "Walk me through how you'd choose an EC2 instance type for a given workload — what families and sizing knobs matter?",
      "notes": "Looking for: instance family literacy (T-burstable, M-general, C-compute-optimized, R/X-memory-optimized, P/G-GPU, I-storage-IO, Inf/Trn-ML accelerator), Graviton (arm64) cost/perf, EBS-optimized, ENA / SR-IOV for network, NVMe vs EBS, Spot vs On-Demand vs Savings Plans vs RIs. Bonus: how they'd benchmark before committing.",
      "followup_questions": [
        {
          "question": "Your service is CPU-bound but bursty — t3.large or c5.large?",
          "notes": "T-series accumulates CPU credits; great for IDLE-then-burst patterns. If CPU is sustained, T runs out of credits and either throttles or you pay unlimited mode (which often costs more than just sizing up to C). C5 is the right default for sustained CPU."
        },
        {
          "question": "How would you justify Graviton over x86 to a skeptical engineering team?",
          "notes": "~20% better price-performance, ARM64 is mainstream now, most languages run unmodified, easy A/B test via separate ASG, and reserved-capacity discounts compound the savings."
        }
      ],
      "level_guidance": {
        "100": "Knows EC2 has many types and that they differ in CPU/RAM. Can pick T or M for general workloads but doesn't know the family letters cold.",
        "200": "Identifies the right family for a workload (C-compute, R-memory, I-storage, P/G-GPU). Mentions Graviton/ARM cost-perf and that t-series is burstable.",
        "300": "Sizes deliberately: benchmarks before committing, knows when burstable credits run out, picks Spot vs On-Demand vs Savings Plan / RI based on workload steadiness, calls out network bandwidth tiers and EBS-optimised instances.",
        "400": "Argues from first principles — references CPU credit accounting, NUMA topology on large instances, NVMe-vs-EBS tradeoffs, ENA/SR-IOV impact, mixed-instances policies for AZ/instance-family diversity, and how to gracefully migrate between families with zero-downtime."
      }
    },
    {
      "domain": "Compute Expertise",
      "question": "Design an Auto Scaling Group for a public web tier handling a daily 10x traffic spike at 9am. Walk me through your configuration choices.",
      "notes": "Looking for: launch template vs launch config (launch templates are current), min/max/desired sizing, target tracking vs step scaling vs scheduled actions, predictive scaling for known patterns, warm pools to avoid cold-start spikes, instance refresh for rolling deploys, health checks (ELB vs EC2), termination policies, mixed instances policy with Spot, multi-AZ, lifecycle hooks for graceful shutdown.",
      "followup_questions": [
        {
          "question": "How do you ensure a scale-out event finishes BEFORE the 9am traffic hits, not during?",
          "notes": "Scheduled scaling action 10-15 min before 9am, predictive scaling based on history, warm pools so newly-launched instances skip cold init, and a tighter target-tracking metric (ALB request count per target) instead of CPU."
        },
        {
          "question": "An instance fails its ELB health check but EC2 status checks are green — what happens and what should happen?",
          "notes": "If health-check-type is EC2-only, ASG won't replace it (bug). Switch to ELB health-check-type so the ASG replaces unhealthy instances. Also: tune health-check grace period so brand-new instances aren't killed before warm-up."
        }
      ],
      "level_guidance": {
        "100": "Mentions Auto Scaling Groups and that they can add/remove instances. May suggest CPU-based scaling and a min/max.",
        "200": "Picks target tracking on CPU or request count, sets min/max/desired, knows ELB-vs-EC2 health checks, mentions multi-AZ for resilience.",
        "300": "Schedules the 9am scale-out via scheduled action OR predictive scaling, uses warm pools so new instances aren't cold, configures instance refresh for rolling deploys, mixes Spot + On-Demand for cost, sets termination policy + lifecycle hooks for graceful shutdown.",
        "400": "Tunes the entire control loop: appropriate cooldown periods to prevent thrash, careful target metric selection (request-count-per-target > raw CPU), instance-warmup to skip cold containers, capacity rebalancing for Spot, integration with CodeDeploy / blue-green, and reasoning about how pre-warming interacts with downstream connection pools."
      }
    },
    {
      "domain": "Compute Expertise",
      "question": "When should you reach for ECS, EKS, or Lambda for a containerised workload, and what are the tradeoffs?",
      "notes": "Looking for: ECS = AWS-opinionated, simpler control plane, Fargate or EC2 launch type. EKS = Kubernetes API, ecosystem compatibility (Helm, operators), more operational surface. Lambda containers = up to 10GB image, 15-min cap, scale-to-zero. Fargate vs EC2 launch: Fargate = no host management but higher $/vCPU; EC2 = cheaper at scale, but you patch the AMI. Service discovery (Cloud Map / native DNS), IAM-task-roles, secrets from SSM/Secrets Manager.",
      "followup_questions": [
        {
          "question": "Your team is multi-cloud or wants to keep portability open — does that change your answer?",
          "notes": "EKS strongly preferred — vanilla k8s manifests work elsewhere. ECS is AWS-only. Lambda is even more AWS-coupled (event source mappings, runtime)."
        },
        {
          "question": "How would you handle secrets and config for a Fargate task without baking them into the image?",
          "notes": "secrets[] in the task definition pulling from Secrets Manager / SSM Parameter Store, IAM-task-role granting the secret-read permission, environment variable injection at runtime, KMS-encrypted, never plaintext in the task def."
        }
      ],
      "level_guidance": {
        "100": "Knows ECS / EKS / Lambda all run containers but isn't precise about when to use each.",
        "200": "ECS = AWS-opinionated, EKS = Kubernetes API, Lambda containers = up to 10GB image with the 15-min cap. Picks ECS for simple cases, EKS for k8s ecosystem.",
        "300": "Runs it: chooses based on portability (EKS wins for multi-cloud), control plane cost vs operational complexity, Fargate-vs-EC2 launch type tradeoffs, IAM-task-roles, secrets injection, service discovery via Cloud Map.",
        "400": "Architects across all three: when to use Lambda containers vs zip, capacity-provider strategies on ECS, Karpenter/Cluster-Autoscaler tuning on EKS, cost modelling, multi-tenant isolation strategies, and reasons about pod-startup-latency vs cold-start vs ASG-warm-pool budgets in a single coherent story."
      }
    },
    {
      "domain": "Compute Expertise",
      "question": "Explain the difference between a process and a thread, and when you'd choose one over the other.",
      "notes": "Looking for: process = isolated address space, own pid, expensive to fork, communicates via IPC (pipes, sockets, shmem). Thread = shares parent's address space, cheap to spawn, communicates via shared memory + locks/atomics. Use processes when isolation matters (fault containment, different privilege boundaries) or for true parallelism on GIL'd runtimes (Python, Ruby). Use threads for I/O-bound fan-out and shared-state workloads. Bonus: async/await as the third option — single thread, cooperative.",
      "followup_questions": [
        {
          "question": "Python's GIL — what does it mean for CPU-bound work, and what's the workaround?",
          "notes": "GIL serialises bytecode execution per process, so threads don't give parallelism for CPU work. Workarounds: multiprocessing, C-extensions that release the GIL (NumPy, PyTorch), subinterpreters (3.12+), or just rewrite the hot path in C/Rust."
        },
        {
          "question": "What's a common race-condition bug pattern with threads sharing state, and how do you debug one?",
          "notes": "Read-modify-write on a shared counter without a lock; non-atomic compound check-then-act on a dict. Debug via thread sanitizer, deterministic re-runs with a single thread, lock instrumentation, or model-checking. Senior answer: 'I default to immutable + message-passing to make these classes of bugs unrepresentable.'"
        }
      ],
      "level_guidance": {
        "100": "Says a process is heavier than a thread; threads share memory. May confuse the two under pressure.",
        "200": "Explains address-space isolation, cheap thread spawn, IPC mechanisms (pipes, sockets, shmem) vs shared memory + locks. Picks threads for I/O fan-out, processes for isolation.",
        "300": "Discusses real cost (page-table copy on fork, thread stack ~1MB by default), scheduler interaction, when to choose async over threads, lock contention pitfalls, GIL implications in Python.",
        "400": "Reasons about process/thread/coroutine continuum, cooperative vs preemptive scheduling, NUMA-aware thread placement, why Linux threads are LWPs, and when to reach for io_uring / kqueue over threadpools entirely."
      }
    },
    {
      "domain": "Compute Expertise",
      "question": "How do you tell whether a workload is CPU-bound or I/O-bound, and how does that change how you'd scale it?",
      "notes": "Looking for: profiling first (top, htop, /proc/stat, perf, py-spy, flame graphs). CPU-bound = high user CPU% with low iowait — scales with more CPUs, faster cores, vectorisation, GPU/accelerator. I/O-bound = high iowait or sleeping in syscalls — scales with concurrency (threads, async, more connection pool), faster disks (NVMe), better caching. Bonus: 'is it actually network-bound? lock-bound?' Senior answer mentions Amdahl's law and the danger of throwing more cores at a serial bottleneck.",
      "followup_questions": [
        {
          "question": "Your service shows 90% CPU on one core but 20% on the others — what's happening?",
          "notes": "Single-threaded hot path (or GIL-locked Python). Either parallelise the hot path, switch to async/event-loop, or scale horizontally with smaller instances so the imbalance doesn't matter. Verify with thread profiles before changing architecture."
        },
        {
          "question": "A web service spends 80% of its time blocked on a downstream API call — what's the right fix?",
          "notes": "Async/await + connection pooling so the calling threads don't sit idle, OR add a timeout + bulkhead + cache layer to absorb the latency. Adding more CPU does nothing because the bottleneck is downstream."
        }
      ],
      "level_guidance": {
        "100": "Knows CPU-bound = busy on the CPU, I/O-bound = waiting on disk/network. Suggests using top.",
        "200": "Reads top/htop/iostat: high user-CPU = CPU-bound, high iowait or high idle with slow throughput = I/O-bound. Scales CPU-bound with bigger cores, I/O-bound with concurrency.",
        "300": "Reaches for profilers (perf, py-spy, flame graphs), distinguishes CPU-bound from lock-bound, picks async over threads for I/O-bound work, uses connection pooling, applies Amdahl's law before throwing more cores at a serial bottleneck.",
        "400": "Diagnoses with a structured methodology (USE method, RED, off-CPU profiles), recognises secondary symptoms (high context-switch rate = lock contention masquerading as CPU), reasons about cache vs memory bandwidth as a third bottleneck, and walks through a real production incident where the easy diagnosis was wrong."
      }
    },
    {
      "domain": "Compute Expertise",
      "question": "What does Docker actually do under the hood, and why is a container lighter than a VM?",
      "notes": "Looking for: Linux primitives — namespaces (PID, net, mount, uts, ipc, user) for isolation; cgroups for resource limits (CPU, memory, IO). Containers share the host kernel — no guest OS, no hypervisor. Image is a layered overlayfs of tarballs. Compare to VMs which boot a full kernel via KVM/Xen/Hyper-V — slower start, more memory, stronger isolation. Bonus: rootless containers, runc/containerd as the layers below docker, and gVisor/Firecracker as stronger-isolation flavours.",
      "followup_questions": [
        {
          "question": "Why is container isolation considered weaker than VM isolation, and when does that matter?",
          "notes": "Shared kernel = a kernel exploit escapes the container. Matters for multi-tenant compute (Lambda uses Firecracker microVMs partly for this), regulated workloads, untrusted code execution. Mitigations: gVisor, Kata, seccomp, AppArmor, no-privileged."
        },
        {
          "question": "A Docker image is 2GB and the container OOMs at 512MB — what's going on and how do you fix it?",
          "notes": "Image size on disk and runtime memory are unrelated — the container needs RAM for the process working set, not the image. Set --memory limit deliberately, profile actual RSS, slim the image (multi-stage build, distroless) for cold-start and registry cost rather than thinking it lowers RAM."
        }
      ],
      "level_guidance": {
        "100": "Knows Docker runs containers and that they're like lightweight VMs. May say 'it virtualises'.",
        "200": "Explains namespaces (PID, net, mount) for isolation and cgroups for limits. Knows the image is layered. Notes containers share the host kernel — no guest OS.",
        "300": "Discusses runc/containerd as the lower layers, overlayfs implementation, image build optimisation (multi-stage, distroless, layer caching), seccomp/AppArmor profiles, rootless containers, and when to reach for gVisor / Firecracker / Kata for stronger isolation.",
        "400": "Speaks fluently to specific syscalls (clone with CLONE_NEWNS etc.), kernel-version-specific feature gates (cgroup v2, idmapped mounts), the container runtime spec (OCI), how Lambda's Firecracker microVM combines microvm + container patterns, and walks through a real container escape they've defended against."
      }
    },
    {
      "domain": "Compute Expertise",
      "question": "Horizontal vs vertical scaling — when would you reach for each, and what tradeoffs do you accept with each choice?",
      "notes": "Looking for: vertical = bigger box, simpler (no distributed systems tax), but ceiling on instance size, single point of failure, downtime to resize unless live-migration. Horizontal = many smaller boxes, near-infinite ceiling, fault-tolerant, but requires statelessness or careful session/state handling, load-balancing, distributed consensus for coordination. Senior answer: 'vertical first until the workload outgrows it, because horizontal introduces a whole class of bugs (split-brain, eventual consistency, partial failure) that vertical doesn't.'",
      "followup_questions": [
        {
          "question": "What state classes block horizontal scaling and how do you address each?",
          "notes": "In-process session state → sticky sessions or move to Redis/Memcached/JWT. In-memory cache → distributed cache or accept some staleness. Local file uploads → S3/EFS. Singleton background job → leader election or a dedicated worker tier with a queue."
        },
        {
          "question": "A Postgres DB is at 90% CPU — you have headroom on the existing box but it's already an r6i.16xlarge. Go vertical or horizontal?",
          "notes": "Vertical first — read replicas (horizontal-ish for reads), then partition/shard if writes are the issue. r6i.32xlarge is a one-line config change vs months of sharding work. Spot-check that this is a CPU problem and not a query/index issue first; Aurora I/O Optimised or Aurora Serverless v2 may also be worth comparing before going to sharding."
        }
      ],
      "level_guidance": {
        "100": "Vertical = bigger box, horizontal = more boxes. Picks horizontal as default 'because cloud'.",
        "200": "Identifies state as the blocker for horizontal scaling, mentions sticky sessions / Redis / JWT, knows vertical has an instance-size ceiling, picks horizontal for fault tolerance.",
        "300": "Reaches for vertical first when state is sticky and the box has headroom, knows when to introduce read replicas vs full sharding, articulates split-brain / partial-failure / eventual-consistency tax of going horizontal, and quantifies the engineering cost vs the additional capacity.",
        "400": "Models scaling end-to-end: connection-pool exhaustion at the DB tier, network-bandwidth ceilings at the LB, cache-locality regression on vertical scale-up, partition-tolerance tradeoffs (CAP), and walks through a real migration where the 'obvious' answer (sharding) was wrong because the bottleneck moved elsewhere first."
      }
    },
    {
      "domain": "Systems Questions",
      "question": "Imagine you are responsible for ensuring that a particular blog post on your company's site could handle an extreme amount of traffic when it was published, how would you handle the preparation for that traffic?",
      "notes": "Good time to listen for a solution like an S3-hosted static page. Listen for the candidate to use a similar technique, or employ caching or other techniques that relived strain from the database/website.",
      "followup_questions": [],
      "level_guidance": {
        "100": "Suggests sharing on social media.",
        "200": "Covers: SEO optimization, social media distribution, email newsletter, paid promotion, content syndication. Knows basic SEO (keywords, meta tags, sitemap).",
        "300": "Designs content distribution strategy: SEO (technical + content), CDN for global delivery, multi-channel distribution (email, social, syndication), A/B testing headlines, analytics to measure reach/engagement, caching strategy for viral traffic spikes.",
        "400": "Content platform reasoning: organic vs paid distribution economics, content delivery architecture for viral readiness (auto-scaling, CDN pre-warming), personalization for engagement, and measuring content value (attribution beyond pageviews to business outcomes)."
      }
    },
    {
      "domain": "Systems Questions",
      "question": "In a scenario where you're building out a three-tiered web application (a front end, application layer, and database layer), what would you do to eliminate single points of failure?",
      "notes": "Each tier needs redundancy: (1) Front end: multiple web servers behind a load balancer, across 2+ AZs; CDN in front for edge caching and DDoS absorption. (2) App tier: auto-scaling group of stateless app servers in 2+ AZs, with the load balancer health-checking them; stateless means any server can handle any request. (3) Database: primary + synchronous standby in another AZ (RDS Multi-AZ, Aurora), automatic failover; read replicas for read scaling. (4) Cross-cutting: DNS health checks (Route 53 failover routing), multi-region for disaster recovery, no hard-coded IPs (use service discovery), external dependencies (payment gateways, email) must also be considered. Strong candidates name state as the hard problem — session state, file uploads, background job state — and route it to Redis/ElastiCache or a persistent store, not the app server's memory. Red flag: 'put it all on one big powerful server.'",
      "followup_questions": [
        {
          "question": "If a page in the app was slow to load, what would you do to troubleshoot or fix it?",
          "notes": "(Parent: systems 3-tier.) Systematic approach: (1) Measure FIRST — open Chrome DevTools Network tab, note TTFB (server response time), content download time, render time. Check Lighthouse for Core Web Vitals (LCP, FID/INP, CLS). (2) Identify the bottleneck — slow DNS? TCP? TLS? Server response time (backend issue)? Asset download (CDN/size issue)? JS execution (bundle size / main thread)? (3) Backend dive — APM (Datadog, New Relic, X-Ray) to find slow service calls; look at DB query logs for missing indexes or N+1 queries; check cache hit rates. (4) Frontend dive — bundle analyzer for JS size, lazy-load images, code-split routes, reduce blocking scripts/CSS. (5) Network — enable gzip/Brotli, HTTP/2, CDN, edge cache. Strong candidates say 'Don't optimize without measuring' and use the RAIL/Core Web Vitals framework. Red flag: starts optimizing at random ('let me add Redis')."
        }
      ],
      "level_guidance": {
        "100": "Names the three tiers (web, app, database). Doesn't explain the why.",
        "200": "Justifies the separation: scaling tiers independently, security boundary between web and DB, swap implementations per tier. Mentions stateless web tier as the scaling unit.",
        "300": "Designs concretely: ALB → autoscaling EC2/ECS/Lambda → RDS (with read replicas) → ElastiCache for hot reads, S3 + CloudFront for static, WAF in front, separate subnets per tier, IAM roles per tier (no DB password in the web tier), CI/CD per tier.",
        "400": "Reasons about the limits: when 3-tier hits its ceiling (e.g., the DB), CQRS / read-write splitting, cell-based architecture for blast-radius bounding, when serverless collapses the app tier into the request itself, and the deeper insight that '3-tier' is a textbook answer — real production architectures have 6-12 tiers (cache, queue, search index, OLAP store, feature store, etc.) but the mental model still applies recursively."
      }
    },
    {
      "domain": "Database Expertise",
      "question": "How do you design a database for high-availability?",
      "notes": "Pillars: (1) Synchronous replication to a standby in a different failure domain (AZ) — RDS Multi-AZ, Aurora (6-way replication across 3 AZs with quorum writes). (2) Automated failover with health checks — DNS/load-balancer repoints to the new primary within seconds, not minutes. (3) Read replicas for read scale and DR (cross-region for disaster recovery). (4) Backups: automated daily + transaction-log-continuous (PITR — point-in-time recovery to any second in the retention window). (5) Monitoring + alerting on replication lag, connection counts, long-running queries. (6) Regular failover DRILLS — untested HA is not HA. (7) For global apps: multi-region active-active (Aurora Global Database, DynamoDB Global Tables) with conflict resolution strategy. Strong answers call out that HA ≠ DR — HA is within-region failover, DR is cross-region for regional outages. Red flag: 'just enable Multi-AZ' without discussing drills, monitoring, or RPO/RTO.",
      "followup_questions": [],
      "level_guidance": {
        "100": "Mentions backups and a standby replica. May say 'use RDS multi-AZ'.",
        "200": "Knows synchronous vs async replication, RPO/RTO targets, primary failover, read replicas for scaling reads. Discusses backups + point-in-time recovery.",
        "300": "Designs the topology: multi-AZ for in-region HA, cross-region read replicas for DR, automated failover, connection-pool / proxy in front (RDS Proxy, PgBouncer), tested failover runbook, lag monitoring, snapshot lifecycle.",
        "400": "Reasons about consistency vs availability tradeoffs (CAP), quorum-based replication (Aurora multi-master / Spanner-style), failure-domain isolation, read-your-writes vs eventual semantics, application-level retry/idempotency, blast-radius bounding for runaway queries, and the operational cost of the chosen topology."
      }
    },
    {
      "domain": "Database Expertise",
      "question": "What are indexes in a database?",
      "notes": "Indexes are auxiliary data structures that let the database find rows without scanning the whole table — analogous to the index at the back of a book. Most common: B-tree (ordered, supports range queries, equality, sort) — the default for SQL. Also: hash indexes (equality only, very fast), GIN/GIST for full-text and geo, covering indexes (index includes the selected columns so no table lookup needed), clustered indexes (the table is physically sorted by the index — SQL Server default, also InnoDB primary key). Trade-offs: indexes speed up reads but slow down writes (every insert/update/delete maintains every index), consume disk space, and can be chosen badly by the optimizer when statistics are stale. Strong answers mention EXPLAIN ANALYZE to verify an index is actually being used, and that over-indexing is a real performance killer on write-heavy tables.",
      "followup_questions": [
        {
          "question": "What types of data are good candidates for indexes?",
          "notes": "(Parent: DB indexes.) Good candidates: (1) Columns frequently in WHERE, JOIN, or ORDER BY clauses. (2) High-cardinality columns (email, user_id) — indexes give the biggest win when most values are distinct. (3) Foreign keys (always index these). (4) Columns used in equality (`=`) or range (`<`, `BETWEEN`) predicates. Poor candidates: (1) Low-cardinality columns (boolean, gender) — index gives little selectivity, a scan may be faster unless combined with another column in a composite index or using bitmap index. (2) Columns that change frequently — the index write cost adds up. (3) Very small tables — DB will just scan. Strong answers mention composite indexes (column order matters — leftmost prefix rule), covering indexes (INCLUDE columns in the index so the row lookup isn't needed), and to `EXPLAIN ANALYZE` to verify the planner actually uses your index. Red flag: 'index every column' — that crushes write performance."
        }
      ],
      "level_guidance": {
        "100": "Knows indexes make queries faster. Can't articulate the tradeoff.",
        "200": "Names B-tree as the default; knows indexes speed reads but slow writes and consume storage. Mentions composite indexes and covering indexes.",
        "300": "Picks indexes based on query patterns: leftmost-prefix rule, covering indexes for hot read paths, partial indexes for sparse predicates, hash vs B-tree vs GIN/GIST. Uses EXPLAIN to validate. Discusses index bloat and rebuild strategies.",
        "400": "Reasons about index design at scale: write amplification on heavily-indexed tables, locking implications during DDL, online index builds (CREATE INDEX CONCURRENTLY), bitmap-index-scan vs heap-fetch tradeoffs, and how an index strategy on a hot table is a capacity-planning question, not a 'just add an index' question."
      }
    },
    {
      "domain": "Database Expertise",
      "question": "A customer needs a relational database for their application. How do you help them choose between the options available on AWS? What questions would you ask them?",
      "notes": "Should start with discovery: workload type (OLTP vs OLAP), scale requirements, existing expertise, licensing concerns, high availability needs, read/write ratio, latency requirements. Then map to options: Aurora PostgreSQL/MySQL (managed, HA, scalable), RDS (broader engine support, simpler migration), Aurora Serverless (variable workloads), Redshift (analytics). Strong answers discuss: engine-specific strengths (Postgres extensions vs MySQL simplicity), cost models (Aurora I/O-Optimized vs Standard), migration path from on-prem (DMS, SCT). Red flag: just names databases without discussing selection criteria.",
      "followup_questions": [
        {
          "question": "When would you recommend Aurora Serverless v2 vs a provisioned Aurora cluster?",
          "notes": "Tests nuanced understanding. Serverless v2: variable/unpredictable workloads, dev/test, spiky traffic. Provisioned: steady-state production with predictable load, where cost optimization matters and you can right-size instances."
        }
      ],
      "level_guidance": {
        "100": "Names databases but can't articulate selection criteria.",
        "200": "Knows RDS engine options and basic trade-offs (Postgres vs MySQL). Can ask about workload size and type. Understands read replicas for read-heavy patterns.",
        "300": "Structured discovery: workload type (OLTP vs OLAP), scale/growth, latency needs, HA requirements, migration complexity. Maps to Aurora (scalable, HA), RDS (broader engines, simpler), Redshift (analytics). Discusses cost models and migration path (DMS/SCT).",
        "400": "Database strategy: total cost of ownership (licensing, operational overhead, team expertise), multi-database architectures (polyglot persistence), Aurora Limitless for horizontal scale, DSQL for distributed SQL, and the organizational decision of standardizing on one engine vs best-fit-per-service."
      }
    },
    {
      "domain": "Performance Troubleshooting/Tuning",
      "question": "How can you identify network latency?",
      "notes": "Delay.\nNetwork latency = how much time it takes for a packet of data to get from one designated point to another one",
      "followup_questions": [
        {
          "question": "How would you go about reducing it in a local network or on the internet?",
          "notes": "(Parent: identify network latency.) Local network: (1) Switch from 100Mb/1Gb to 10Gb/25Gb NICs and switches. (2) Reduce hops — collapse network layers, use jumbo frames (MTU 9000) for storage traffic. (3) Eliminate duplex mismatches and check cables/optics for CRC errors. (4) Shorten physical distance or move services to same rack. Internet: (1) CDN for static assets — brings content to the user's edge. (2) Anycast DNS and API endpoints (e.g., Cloudflare, Route 53 latency-based routing) — routes to the nearest region. (3) TCP optimizations: TCP Fast Open, BBR congestion control, enable HTTP/3 (QUIC) which eliminates HOL blocking and reduces handshake latency. (4) Keep-alive connections to avoid reconnection cost. (5) Multi-region deploys + edge compute (CloudFront Functions, Lambda@Edge). Strong answers mention measuring with `mtr`, `traceroute`, browser waterfall, and that the biggest gain usually comes from reducing RTTs (geo + connection reuse), not bandwidth. Red flag: 'buy a bigger pipe' — that's throughput, not latency."
        }
      ],
      "level_guidance": {
        "100": "Knows ping shows latency. Limited approach.",
        "200": "Tools: ping (ICMP RTT), traceroute (hop-by-hop latency), CloudWatch metrics (NetworkIn/Out), VPC Flow Logs, application-level timing. Knows to distinguish network latency from application latency.",
        "300": "Systematic diagnosis: measure at each layer (client -> CDN -> LB -> server -> DB), use distributed tracing (X-Ray) to identify slow segments, VPC Flow Logs for network issues, test from multiple locations (CloudWatch Synthetics), baseline comparison (is this new or normal?).",
        "400": "Network performance engineering: understanding latency components (propagation, transmission, processing, queuing), bandwidth-delay product, TCP window optimization, the impact of TLS handshake overhead, and designing for latency-sensitive applications (placement groups, enhanced networking, Global Accelerator for optimized internet paths)."
      }
    },
    {
      "domain": "Performance Troubleshooting/Tuning",
      "question": "How would you describe cache?",
      "notes": "Faster read/write access\nGenerally hosts most often accessed data\nMakes up for the cost of i/o\nCan be at various levels (from CPU cache to HDD cache to Database-friendly distributed in-memory cache, memcached, redis)",
      "followup_questions": [
        {
          "question": "What are some pros and cons of using cache?",
          "notes": "Pros: (1) Dramatic latency reduction (memory vs disk vs network is orders of magnitude). (2) Reduces load on expensive backends (DB, downstream APIs). (3) Handles read spikes better — cache shields the origin. (4) Lower cost per request. Cons: (1) Stale data — the fundamental trade-off; invalidation is 'one of the two hard problems in CS'. (2) Cache stampede / thundering herd — when a hot key expires, N concurrent clients all hit the origin at once. Mitigate with request collapsing, stale-while-revalidate, or probabilistic early expiration. (3) Memory cost. (4) Cold-start problem — after a cache restart/flush, origin sees full traffic. (5) Consistency complexity — multi-region caches have their own propagation delay. (6) Bug class: 'works with warm cache, broken without it' — tests should include cold-cache scenarios. Strong answers mention: cache-aside vs write-through vs write-back patterns and that choosing a TTL is a product decision (freshness vs hit rate)."
        }
      ],
      "level_guidance": {
        "100": "Knows caching makes things faster by storing previous results.",
        "200": "Names cache levels (CPU L1/L2/L3, OS page cache, app-level, CDN). Discusses cache hit rate, eviction policies (LRU), TTL, invalidation. Cites Phil Karlton's 'two hard things' quote.",
        "300": "Has run a real cache: cache-aside vs read-through vs write-through, stampede prevention (request coalescing, jittered TTLs), distributed-cache invalidation (pub-sub, versioned keys), serialization cost, when not to cache (low hit rate, frequent writes, freshness-critical data).",
        "400": "Reasons about cache as a coherence problem: weak vs strong consistency between cache and source of truth, cache poisoning, cache as a security boundary (per-user cache keys, vary headers), edge-side caching with CDN purge propagation, and the deeper insight that caching is a complexity tradeoff — every cache is a distributed-system problem in disguise."
      }
    },
    {
      "domain": "Performance Troubleshooting/Tuning",
      "question": "What is a Denial of Service attack?",
      "notes": "Distributed Denial of service: Attempt at making a machine or network unavailable by having many nodes attacking",
      "followup_questions": [
        {
          "question": "What are some steps you might take to protect yourself from such attacks?",
          "notes": "(Parent: DoS.) Layered defense: (1) DDoS protection at the edge — AWS Shield (Standard is free + automatic, Advanced for high-value targets), Cloudflare, Akamai. These absorb volumetric floods. (2) WAF to block application-layer attacks (Slowloris, form floods, scrapers, credential stuffing). (3) Rate limiting at the API gateway, per-IP and per-user (token bucket, sliding window). (4) CDN absorbs traffic spikes and caches responses, shielding origin. (5) Auto-scaling for legitimate growth, but cap it (don't let attacks scale your AWS bill to infinity). (6) Origin protection — block direct-to-origin traffic, only allow traffic via CDN (Origin Access Controls, VPC endpoints). (7) CAPTCHA for suspicious traffic patterns. (8) Incident-response playbook — who to page, how to tighten rate limits in real time, BGP blackhole if it's that bad. Strong answers mention monitoring (spikes in 5xx, requests/sec, unique IPs) and that the attacker always has the asymmetry advantage — layered defense is required."
        }
      ],
      "level_guidance": {
        "100": "Knows it floods a server with requests.",
        "200": "Explains: overwhelms target with traffic to exhaust resources (bandwidth, connections, CPU). Types: volumetric (UDP flood), protocol (SYN flood), application-layer (HTTP flood). DDoS = distributed. Mitigation: rate limiting, WAF, Shield.",
        "300": "Designs DDoS protection: AWS Shield Standard (free, L3/L4), Shield Advanced (managed response team, cost protection), CloudFront for absorption (massive global capacity), WAF rate-based rules for L7, auto-scaling to absorb while filtering, architecture patterns (no single points of failure, IP allowlisting for admin).",
        "400": "DDoS as an operational concern: the economics of attack vs defense (attacker cost vs defender cost), multi-layered mitigation (edge -> origin), application-layer attacks that look like legitimate traffic (credential stuffing, API abuse), incident response playbooks, and the business decision of Shield Advanced (insurance model, SLA, WAR credits)."
      }
    },
    {
      "domain": "Performance Troubleshooting/Tuning",
      "question": "A customer asks 'do I need GPUs for my workload?' How do you assess this, and what are the trade-offs between GPU instance types on AWS?",
      "notes": "Tests understanding of GPU compute applicability and AWS options. Strong answers: first ask what the workload is (ML training/inference, video processing, scientific simulation, graphics rendering) -- not all 'AI' needs GPUs. Then match to instance type: P5/P4d for training (large models), G5/G6 for graphics/inference, Inf2 for cost-optimized inference, Trn1 for training at scale. Trade-offs: cost (GPUs are expensive), availability (not always available in all AZs), GPU memory as constraint (model size), alternatives (CPU for small models can be more cost-effective, Graviton for inference of smaller models). Red flag: assumes all AI needs GPUs, or can't distinguish training vs inference requirements.",
      "followup_questions": [
        {
          "question": "What purposes do GPUs have?",
          "notes": "Originally: graphics rendering (transforming 3D geometry, shading pixels) — hence the name. Today: any workload with massive parallelism across simple operations. (1) ML training and inference — matrix multiplications map perfectly to thousands of GPU cores; a single H100 can do ~2 petaFLOPs of FP8. (2) Video encoding/transcoding (NVENC). (3) Scientific computing — molecular dynamics, weather simulation, genomics. (4) Cryptocurrency mining (pre-ASIC). (5) Data processing (RAPIDS for pandas-on-GPU, Dask+GPU). (6) Rendering and ray tracing for visual effects. Strong candidates know the CPU vs GPU trade-off: CPUs have ~8-100 complex cores optimized for low-latency branching code; GPUs have thousands of simple cores optimized for throughput on data-parallel workloads. Mentions TPUs (Google), Trainium/Inferentia (AWS) as domain-specific alternatives. Red flag: 'GPUs are just for games'."
        }
      ],
      "level_guidance": {
        "100": "Knows GPU is for graphics/gaming.",
        "200": "Explains: massively parallel processor (thousands of cores vs CPU's few), optimized for matrix math, used for ML training/inference, video encoding, scientific computing. Knows GPU vs CPU trade-offs (parallel vs sequential workloads).",
        "300": "Applied in cloud: instance selection (P5/P4d for training, G5/G6 for graphics/inference, Inf2 for inference), GPU memory as constraint (model size), multi-GPU communication (NVLink, EFA), spot instances for cost optimization in training, right-sizing GPU instances.",
        "400": "GPU architecture reasoning: memory bandwidth as the real bottleneck (not compute), GPU programming models (CUDA, Triton), custom silicon trajectory (Trainium, Inferentia), the economics of GPU compute (on-demand vs reserved vs spot, time-sharing via MIG), and when NOT to use GPUs (inference of small models, CPU may be more cost-effective)."
      }
    },
    {
      "domain": "Performance Troubleshooting/Tuning",
      "question": "What is the difference between swapping and paging?",
      "notes": "Note: Details depend on the OS discussed.\nConventional wisdom = Extra memory (known as virtual) allocated on disk rather than in RAM",
      "followup_questions": [
        {
          "question": "What are the performance impacts of either?",
          "notes": "(Parent: swapping vs paging.) Swapping out an entire process to disk causes massive latency when the process is re-scheduled — all its memory must be read back. Paging (swapping just the pages being referenced) is far more efficient but still orders of magnitude slower than RAM (1μs RAM vs 10-100μs SSD vs 10ms HDD per page). Production impact: once your system starts swapping heavily, latency becomes unpredictable, CPU looks idle while processes wait on I/O, and apps feel 'stuck' — this is the 'swap death spiral'. Best practice: size RAM so swap is effectively never used under normal load, and monitor vmstat/page faults to catch memory pressure before it hurts. On cloud instances with fast NVMe, the cost of swapping is reduced but the rule still holds: if you're swapping steadily, you've under-sized the instance. Red flag: 'swap is a performance boost' — it's a safety valve, not a feature to rely on."
        }
      ],
      "level_guidance": {
        "100": "Mixes the two up or only knows one term.",
        "200": "Defines them: paging moves individual pages between RAM and disk; swapping (historically) moves whole processes. Knows that on Linux 'swap' colloquially means paging. Mentions the disk-cost penalty.",
        "300": "Has tuned it: vm.swappiness, ZRAM / zswap, knows that swap thrashing kills latency, monitors si/so in vmstat, sizes RAM to fit the working set, prefers eviction (oom-killer + alarms) over silent paging in latency-critical services.",
        "400": "Reasons about virtual-memory mechanics: page tables, TLB pressure, huge pages tradeoffs (THP latency spikes), NUMA-aware allocation, why managed-runtime GC interacts poorly with paging, mmap-vs-read tradeoffs, and the deeper insight that paging is a hint that capacity planning is wrong, not a feature to rely on."
      }
    },
    {
      "domain": "Security",
      "question": "How are certificates used to validate the authenticity of a server?",
      "notes": "TLS handshake: (1) Server presents its X.509 certificate which contains its public key + domain name + issuer's signature. (2) Client checks the certificate chain up to a trusted Certificate Authority (CA) in its root store. (3) Each certificate in the chain is signed by the next-higher CA using that CA's private key — the client verifies each signature with the CA's public key. (4) Client also checks: not expired, domain matches (SAN or CN), not revoked (OCSP/CRL). (5) If all pass, client trusts the server's public key and uses it to establish a session key. Strong answers mention: CA compromise risk, certificate transparency logs, why self-signed certs only work if both sides pre-share trust, and SNI for virtual hosting. Red flag: thinks the cert itself encrypts traffic — it only proves identity.",
      "followup_questions": [],
      "level_guidance": {
        "100": "Knows certificates are used for HTTPS. May mention 'the padlock icon'.",
        "200": "Explains certificate chain: CA signs server cert, browser trusts CA root, validates cert matches domain, checks expiration/revocation (CRL/OCSP). Knows TLS handshake basics.",
        "300": "Deep understanding: certificate pinning, CT logs for transparency, mutual TLS (mTLS) for service-to-service, ACM for automated issuance/rotation, private CA for internal services, certificate-based auth vs token-based.",
        "400": "Reasons about PKI at scale: organizational CA hierarchy, short-lived certificates (SPIFFE/SPIRE), post-quantum cryptography readiness, certificate lifecycle automation across thousands of services, and the trust model implications of different CA approaches."
      }
    },
    {
      "domain": "Security",
      "question": "How would you store encrypted keys in the cloud?",
      "notes": "Never in source code, never in environment variables for high-security use cases. Use a dedicated KMS: AWS KMS (managed CMKs, backed by HSMs), Azure Key Vault, Google Cloud KMS, or HashiCorp Vault. Pattern: (1) Data encryption keys (DEKs) encrypt the data itself, (2) the KMS holds the master key (KEK) and encrypts/decrypts the DEKs, (3) the encrypted DEK is stored alongside the data ('envelope encryption'). Use IAM to restrict who can call the KMS decrypt API, and log every decrypt in CloudTrail. For ultra-sensitive keys use CloudHSM (single-tenant FIPS 140-2 Level 3 device). Strong answers mention: key rotation policies, separation of duty (cryptographic admin vs data user), and never exporting the master key from the HSM.",
      "followup_questions": [],
      "level_guidance": {
        "100": "Suggests storing keys in a file or environment variable.",
        "200": "Knows AWS KMS for key management, Secrets Manager for credentials, envelope encryption concept. Understands key rotation.",
        "300": "Designs key management strategy: KMS CMKs with key policies, CloudHSM for regulatory requirements (FIPS 140-2 Level 3), automated rotation, cross-account key sharing, envelope encryption for large data, audit trail via CloudTrail.",
        "400": "Reasons about cryptographic key management as an organizational capability: key hierarchy design, separation of duties (admin vs user), BYOK vs AWS-managed trade-offs, multi-region key replication, crypto-agility for algorithm transitions, and the operational risk of key loss vs compromise."
      }
    },
    {
      "domain": "Security",
      "question": "Tell me a few ways passwords or credentials can be compromised, and how you would prevent it.",
      "notes": "Attack vectors: (1) Phishing — the #1 cause. Prevent with MFA, FIDO2/WebAuthn hardware keys, email filtering, user training. (2) Credential stuffing from breached password dumps — prevent with unique passwords per site (password managers), MFA, and breach-check APIs like HaveIBeenPwned. (3) Keyloggers/malware on endpoints — prevent with EDR, patched OS, and phishing-resistant MFA (hardware keys). (4) Weak passwords brute-forced — prevent with complexity rules + length >= 12, rate limiting, and account lockout. (5) Credentials checked into source code — prevent with pre-commit secret scanners (git-secrets, truffleHog), Secrets Manager, and rotating exposed keys immediately. (6) Man-in-the-middle on plaintext channels — always use TLS. Strong answers lead with 'passwords alone are insufficient — MFA is table stakes'. Red flag: only mentions password complexity rules.",
      "followup_questions": [],
      "level_guidance": {
        "100": "Names phishing and weak passwords. May suggest 'use strong passwords'.",
        "200": "Covers: brute force, credential stuffing, phishing, keyloggers, exposed secrets in code/logs, shoulder surfing. Prevention: MFA, password policies, secrets scanning, hashing+salting, rate limiting.",
        "300": "Systematic approach: attack surface analysis (endpoint, network, application, human), defense in depth (MFA + anomaly detection + secrets rotation + WAF), incident response for compromised credentials, automated secrets rotation (Secrets Manager), GuardDuty for anomaly detection.",
        "400": "Reasons about identity security as a system: passwordless architectures (FIDO2/WebAuthn), zero-trust principles (never trust, always verify), behavioral analytics for compromise detection, credential exposure monitoring at scale, and the economic trade-off between security friction and user experience."
      }
    },
    {
      "domain": "Security",
      "question": "What are other ways you can secure login access besides a password?",
      "notes": "SSH / certificate based authentication, MFA via SMS, biometrics",
      "followup_questions": [],
      "level_guidance": {
        "100": "Names MFA or biometrics. Limited depth.",
        "200": "Covers: MFA (TOTP, SMS, hardware keys), biometrics (fingerprint, face), SSO/federation (SAML, OIDC), certificate-based auth, IP allowlisting.",
        "300": "Designs authentication architecture: FIDO2/WebAuthn for phishing-resistant auth, risk-based adaptive authentication, device trust (managed device certificates), step-up authentication for sensitive operations, federated identity with IdP (Okta/Azure AD + AWS SSO).",
        "400": "Reasons about authentication strategy: passwordless roadmap, continuous authentication (behavioral biometrics), zero-trust device posture assessment, identity-as-a-service architecture, and the organizational change management required to move beyond passwords."
      }
    },
    {
      "domain": "Security",
      "question": "What does it mean to federate an identity?",
      "notes": "Federation = trusting identities issued by another system, so users can sign in once and access multiple applications without separate accounts. Common protocols: SAML 2.0 (enterprise SSO — Okta, Azure AD to SaaS apps), OIDC/OAuth 2.0 (modern web/mobile — 'sign in with Google'), WS-Federation (older Microsoft). Mechanics: the 'identity provider' (IdP) authenticates the user and issues a signed token; the 'service provider' (SP) trusts the IdP's signature and reads the user's identity + claims (groups, email) from the token. Strong answers describe a real scenario — e.g., corporate employee logs into Okta, clicks a Salesforce tile, SAML assertion is posted to Salesforce, user is logged in without a Salesforce password. Bonus: mentions SCIM for user provisioning alongside federation, and that federation eliminates one of the biggest attack surfaces — per-app password sprawl.",
      "followup_questions": [],
      "level_guidance": {
        "100": "Has heard the term. Might describe it as 'sharing logins between systems'.",
        "200": "Explains: an identity provider (IdP) issues tokens (SAML / OIDC) that other systems trust, so users sign in once. Names a few IdPs (Okta, Cognito, Entra ID).",
        "300": "Has implemented one: SAML vs OIDC tradeoffs, JIT user provisioning, attribute mapping, group-to-role mapping, session lifetime, federation for B2B (third-party orgs) vs B2C (social).",
        "400": "Reasons about the trust model: how to bound the blast radius if the IdP is compromised, multi-IdP architectures with audience-restricted tokens, SCIM for lifecycle, deprovisioning latency, the 'nobody federates with the SAML server' problem (workforce sign-in for the IdP itself), and the deeper insight that federation is a delegation of trust — pick wisely."
      }
    },
    {
      "domain": "Security",
      "question": "What is the design intent of antivirus software?",
      "notes": "Detect and remove malicious software on endpoints. Traditional design: signature database of known-bad file hashes and byte patterns; scans files at rest and on execute. Modern design (EDR — Endpoint Detection and Response): adds behavior analytics — flags suspicious process behavior, unusual network connections, memory injection — because signatures are trivially defeated by polymorphic malware. Strong answers note the limitations: AV is a detect-and-respond control, not a prevention — zero-days and fileless attacks bypass it. Defense-in-depth matters more: patching, least privilege, application allowlisting (Windows Defender Application Control), network segmentation. Bonus: mentions that enterprise EDR (CrowdStrike, SentinelOne, Microsoft Defender) now includes threat hunting and automated response playbooks.",
      "followup_questions": [
        {
          "question": "What is the most common implementation of antivirus software?",
          "notes": "Endpoint AV installed on each device, signature-based with heuristic extensions. Signatures: vendor maintains a database of known-malicious file hashes and byte patterns; scanner hashes files on-access and on-schedule. Modern implementations add: (1) Heuristic/behavioral detection — unusual process behavior (spawning cmd.exe from Word, injecting into lsass.exe). (2) Cloud lookup — unknown hash? Query the vendor's cloud reputation DB. (3) ML models for polymorphic malware that defeats static signatures. (4) EDR capabilities — continuous monitoring, not just scan-on-access. Major products: Microsoft Defender (built into Windows, genuinely competitive now), CrowdStrike Falcon (enterprise EDR leader), SentinelOne, Sophos, Malwarebytes. Strong answers mention that traditional signature AV is the floor, not the ceiling — modern threats require behavioral EDR, allow-listing (Windows Defender Application Control), and DLP alongside. Red flag: 'just install McAfee' — 2005 answer."
        }
      ],
      "level_guidance": {
        "100": "Knows antivirus 'scans for viruses'. May describe signature matching.",
        "200": "Explains: signature-based detection (known malware hashes/patterns), heuristic analysis (suspicious behavior), real-time file scanning, quarantine, definition updates. Knows limitations (zero-day, fileless malware).",
        "300": "Modern endpoint security view: AV is one layer -- add EDR (behavioral detection, lateral movement detection), application allowlisting, sandboxing, machine learning-based anomaly detection. In cloud: GuardDuty for threat detection, Inspector for vulnerability scanning, Macie for data security.",
        "400": "Reasons about endpoint security evolution: traditional AV is necessary but insufficient, defense requires assume-breach mentality, detection + response speed matters more than prevention alone, cloud-native threat detection patterns, and the architectural shift from perimeter defense to identity-centric security."
      }
    },
    {
      "domain": "Security",
      "question": "Why would  you use file hashing?",
      "notes": "Three main uses: (1) Integrity verification — compute the hash of a downloaded file and compare to the publisher's published hash to detect tampering or corruption. (2) Deduplication — content-addressed storage (git, S3 object lock, Dropbox) stores by hash, so identical files are only stored once. (3) Password storage — never store plaintext; store a salted hash (bcrypt, argon2, scrypt) so a breached DB doesn't reveal passwords. Hash properties: deterministic (same input → same output), fast, collision-resistant. MD5 and SHA-1 are BROKEN for security — use SHA-256 or SHA-3 for general hashing, and bcrypt/argon2/scrypt specifically for passwords (these are slow by design). Red flag: suggests SHA-256 for password storage (too fast, brute-forceable with GPUs).",
      "followup_questions": [
        {
          "question": "Can you name any hash functions?",
          "notes": "(Parent: file hashing.) General-purpose cryptographic: SHA-256 (SHA-2 family, current default), SHA-3 (newer, sponge-based, different construction from SHA-2 so resistant to different attacks), SHA-512 (bigger output, faster on 64-bit). BROKEN — don't use for security: MD5 (collisions found in 2004 by Wang et al.), SHA-1 (collisions demonstrated 2017 via SHAttered, deprecated). Password-specific (deliberately slow — that's the point): bcrypt, scrypt, argon2 (Argon2id is the current OWASP recommendation). Non-cryptographic (fast but NOT collision-safe — only for hash tables, checksums): xxHash, CityHash, MurmurHash, FNV. Strong answers distinguish cryptographic (resist preimage and collision attacks, slow on purpose for passwords) from non-cryptographic (fast, not secure), and mention SHA-256 as the general default but argon2id for passwords. Red flag: suggests MD5 or SHA-1 for anything security-relevant in 2025."
        }
      ],
      "level_guidance": {
        "100": "Knows hashing creates a 'fingerprint'. May confuse with encryption.",
        "200": "Explains: integrity verification (detect tampering), file deduplication, malware signature matching, download verification (checksums). Knows SHA-256 vs MD5 (deprecated). Distinguishes hashing from encryption (one-way vs reversible).",
        "300": "Applied uses: code signing (verify binary integrity), configuration drift detection (compare hashes of deployed files), forensic investigation (evidence integrity chain of custody), S3 object integrity (Content-MD5, SHA-256 checksums), container image digests.",
        "400": "Reasons about integrity verification at scale: Merkle trees for efficient large-dataset verification (blockchain, git), hash-based message authentication (HMAC), content-addressable storage systems, supply chain security (SLSA framework, sigstore), and cryptographic agility when hash algorithms are deprecated."
      }
    },
    {
      "domain": "Security",
      "question": "Why should customers use MFA - multi-factor authentication?",
      "notes": "MFA makes stolen-password attacks ineffective — even if phished or breached, the attacker still needs the second factor. Microsoft and Google both report MFA blocks >99% of automated account-takeover attacks. The three factors: something you KNOW (password), something you HAVE (phone, token, hardware key), something you ARE (biometrics). Any two from different categories count as MFA. Best to worst: FIDO2/WebAuthn hardware keys (phishing-resistant) > push-based app (Duo, Authy) > TOTP codes (Google Authenticator) > SMS (SIM-swap vulnerable — avoid for high-value accounts). Strong answers highlight SMS MFA has real flaws (SIM swap, SS7) and should be replaced for admin accounts with FIDO2 or hardware tokens. Red flag: says SMS is 'good enough'.",
      "followup_questions": [
        {
          "question": "Have you ever worked with any MFA devices?",
          "notes": "Listen for hands-on experience. Hardware tokens: YubiKey (FIDO2/WebAuthn + smart card + OATH TOTP — the gold standard), Titan Keys (Google), RSA SecurID (legacy, time-based display). Software: Authy, Google Authenticator, Microsoft Authenticator, Duo Mobile (push-based is faster than typing TOTP codes). Phishing-resistant (FIDO2/WebAuthn + passkeys) is the modern direction — hardware key or synced passkey in Apple/Google/Microsoft keychain. SMS should be deprecated for high-value accounts (SIM-swap attacks are routine). Strong candidates describe: deployment at scale (YubiKey provisioning programs, lost-key recovery), user support issues, and the trade-off between phishing-resistance (hardware keys win) and recoverability (synced passkeys win). Red flag: 'we use SMS because it's easier' — that's a regression."
        }
      ],
      "level_guidance": {
        "100": "Knows MFA = multiple authentication factors (something you know + something you have). Says it stops phishing.",
        "200": "Explains the factor categories (knowledge / possession / inherence) and that SMS-based MFA is weak (SIM swap). Suggests TOTP or push-based MFA.",
        "300": "Picks WebAuthn / passkeys as the strong default; understands why TOTP is phishable but passkeys aren't (origin-bound, hardware-backed). Discusses backup factors, recovery flows, and the friction-vs-security tradeoff.",
        "400": "Reasons about phishing-resistant MFA at scale: passkey rollout strategy, device-bound vs synced passkeys, attestation for high-assurance contexts, FIDO2 + step-up auth, account-recovery as the actual weakest link, and the threat-model gap that MFA does NOT close (session token theft, malicious browser extensions)."
      }
    },
    {
      "domain": "Security",
      "question": "Why do organization use a bastion host or jump box?",
      "notes": "A bastion host is a single hardened server sitting at the network boundary that admins must pass through to reach internal resources. Purpose: (1) Reduce attack surface — only the bastion is internet-exposed, not every internal server. (2) Centralize access control — all SSH/RDP sessions are authenticated and authorized at one choke point. (3) Audit trail — bastions log every session for compliance. Modern alternatives: AWS Systems Manager Session Manager (no open ports, all traffic via AWS API, logged to CloudWatch), Teleport, HashiCorp Boundary, or zero-trust approaches (Tailscale, Cloudflare Access) that eliminate the bastion entirely. Strong answers mention the bastion is itself a single point of compromise — if it's popped, the attacker has a pivot into the whole network — so it needs heavy hardening (no standing credentials, MFA, short-lived session tokens, monitoring).",
      "followup_questions": [
        {
          "question": "Why would you use one?",
          "notes": "(Parent: bastion host.) Centralized, auditable access to internal infrastructure. Pros over direct SSH-everywhere: (1) Single hardened surface to defend and patch. (2) Single choke point to log — know exactly who accessed what, when. (3) Can enforce MFA, step-up auth, session recording at the bastion. (4) Reduces exposure of internal servers (they don't need public IPs or open SSH). Modern alternatives that obsolete the bastion pattern: AWS Systems Manager Session Manager (API-based, no open ports, CloudTrail logged, IAM-authed), HashiCorp Boundary, Teleport (records sessions + per-role access), Cloudflare Zero Trust. Strong candidates prefer these over traditional bastions: they eliminate the 'public IP on the bastion' attack surface entirely. Red flag: proposes a bastion without discussing the alternatives, or leaves standing credentials on the bastion instead of short-lived issued tokens."
        }
      ],
      "level_guidance": {
        "100": "Knows it's a server you connect through. May not explain why.",
        "200": "Explains: single hardened entry point to private network, reduces attack surface (only one host exposed), enables audit logging of all admin sessions, session recording. Knows SSH tunneling through bastion.",
        "300": "Modern alternatives and design: AWS Systems Manager Session Manager (no bastion needed, no inbound ports, full audit via CloudTrail), EC2 Instance Connect, bastion in public subnet with restrictive SGs, auto-scaling bastion for HA, time-limited access via temporary credentials.",
        "400": "Reasons about access patterns: bastion is a legacy pattern being replaced by identity-aware proxies (BeyondCorp model), zero-trust network access, just-in-time access provisioning, and the architectural shift from network-perimeter security to identity-centric access control."
      }
    },
    {
      "domain": "Security",
      "question": "What's the difference between IDS and IPS?",
      "notes": "(Intrusion detection vs intrusion prevention)",
      "followup_questions": [],
      "level_guidance": {
        "100": "Knows both detect threats. May not clearly distinguish inline vs passive.",
        "200": "IDS = passive monitoring, alerts on suspicious traffic (out-of-band copy). IPS = inline, actively blocks malicious traffic. Trade-off: IPS can cause latency and false-positive blocking. Knows signature-based vs anomaly-based detection.",
        "300": "AWS mapping: Network Firewall (IPS with Suricata rules), GuardDuty (IDS -- monitors VPC Flow Logs, DNS, CloudTrail), third-party IDS/IPS in gateway architecture. Designs detection architecture: what to detect where (network vs host vs application layer).",
        "400": "Reasons about detection strategy: defense in depth across layers, tuning false positive rates vs detection coverage, threat intelligence integration, automated response (SOAR), and the operational reality that most breaches aren't caught by IDS/IPS but by anomaly in identity/data access patterns."
      }
    },
    {
      "domain": "Security",
      "question": "What's a firewall and how would you use it to secure your network?",
      "notes": "A firewall enforces traffic policy at a network boundary. Types: (1) Packet-filter — allow/deny by IP+port (fast, stateless, very limited context). (2) Stateful — tracks connections so return traffic is automatically allowed (this is what AWS Security Groups do). (3) Application-layer / NGFW — inspects HTTP, TLS SNI, DNS — can block by URL, user identity, app signature. (4) WAF — specifically filters web-app attacks (SQLi, XSS). Good practices: default deny, only open what you need, segment networks (DMZ, internal tiers), egress filtering (many orgs forget this — block outbound to prevent C2 beacons), allow-lists for admin access, regular rule review (firewall rules rot). Strong answers mention that cloud security groups are a better fit than perimeter firewalls for east-west traffic (microservices talking to each other).",
      "followup_questions": [],
      "level_guidance": {
        "100": "Knows firewalls block traffic. May describe it as 'keeping hackers out'.",
        "200": "Explains: packet filtering on IP/port/protocol, Security Groups (stateful, instance-level) vs NACLs (stateless, subnet-level), allow/deny rules, default deny principle. Can design basic SG rules for web app (80/443 from internet, app port from LB only, DB port from app only).",
        "300": "Designs network security architecture: layered SGs + NACLs, AWS Network Firewall for advanced inspection (domain filtering, IPS, stateful rules), centralized egress firewall, micro-segmentation between services, firewall rule management at scale (Firewall Manager).",
        "400": "Reasons about network security evolution: traditional firewalls assume trusted internal network (outdated), zero-trust requires identity-aware policies regardless of network position, service mesh for application-layer security, and the operational challenges of firewall rule sprawl in large organizations."
      }
    },
    {
      "domain": "Security",
      "question": "What'a a WAF?",
      "notes": "WAF = Web Application Firewall. Operates at Layer 7 (HTTP/HTTPS) and inspects requests for application-layer attacks: SQL injection, XSS, path traversal, OGNL injection (Log4Shell), DoS at the app layer, bot traffic. Sits between clients and the web app — AWS WAF attaches to CloudFront/ALB/API Gateway; Cloudflare WAF is edge-based; Akamai, Imperva, ModSecurity are other names. Key rule sources: OWASP Core Rule Set (free, baseline coverage) + managed rules from the vendor (updated for new CVEs like Log4Shell within hours). Strong answers note WAFs aren't a substitute for secure coding — they're one layer. They also have false-positive risk (blocking legit traffic), so deploy in 'count' mode first, then gradually enable blocking. Red flag: confuses WAF (L7 app attacks) with a network firewall (L3/L4 IP/port).",
      "followup_questions": [
        {
          "question": "What's its relationship to a firewall?",
          "notes": "(Parent: WAF.) Traditional firewalls operate at Layer 3/4 (IP, port, protocol) — allow/deny based on who's talking on what port. WAFs operate at Layer 7 (HTTP) — they understand request methods, headers, URL paths, body content, query parameters, and can make decisions based on application semantics: block SQL injection in a POST body, block path traversal in a URL, block scrapers by user-agent. A network firewall can't see inside HTTPS traffic; a WAF terminates TLS (or sits after a load balancer that does) and inspects decrypted HTTP. They're COMPLEMENTARY not replacements: network firewalls restrict who can reach the server at all; WAFs filter what the legitimate HTTP traffic can contain. Strong answers mention layered defense — firewall at the perimeter + WAF in front of web apps + security groups for workload-to-workload traffic + application-level input validation in code."
        }
      ],
      "level_guidance": {
        "100": "Knows WAF protects web applications. May confuse with network firewall.",
        "200": "Explains: application-layer (L7) firewall, inspects HTTP requests, protects against OWASP Top 10 (SQLi, XSS, CSRF), rate limiting, geo-blocking. Knows AWS WAF with managed rule groups.",
        "300": "Designs WAF strategy: custom rules for application-specific threats, rate-based rules for DDoS, Bot Control for credential stuffing, CloudFront + WAF for edge protection, logging to S3/CloudWatch for analysis, Security Automations for auto-blocking, integration with Shield Advanced for DDoS.",
        "400": "Reasons about application security architecture: WAF is one layer (not sufficient alone), defense requires secure coding + WAF + runtime protection, ML-based anomaly detection for zero-day, the false positive challenge at scale, and the organizational model (security team manages rules vs developers own their WAF configs)."
      }
    },
    {
      "domain": "Security",
      "question": "What is TLS mutual authentication?",
      "notes": "Standard TLS authenticates only the SERVER (client verifies server's cert, server never checks client identity). Mutual TLS (mTLS) adds the reverse: both sides present a certificate and both verify each other's chain. Used when you need strong identity on both ends — service-to-service auth in a service mesh (Istio, AWS App Mesh), B2B API integrations (financial institutions, government), IoT device auth (each device gets a provisioned cert). Advantages over shared-secret tokens: stronger identity (private key never leaves device), automatic revocation via CRL/OCSP, and cryptographic auth at connection time rather than per-request bearer token. Downside: cert lifecycle management is non-trivial — need issuance, rotation, revocation. Strong answers mention zero-trust networking as the modern use case, with tools like SPIFFE/SPIRE for automated mTLS at scale.",
      "followup_questions": [],
      "level_guidance": {
        "100": "Knows TLS encrypts traffic. May not understand mutual aspect.",
        "200": "Explains: both client and server present certificates (standard TLS only server authenticates). Use cases: service-to-service auth in microservices, API authentication without API keys, IoT device authentication. Understands trust store concept.",
        "300": "Designs mTLS architecture: private CA (ACM PCA) for issuing client certs, certificate rotation strategy, ALB/API Gateway mTLS termination, service mesh (App Mesh/Istio) for automatic mTLS between services, fallback strategies during cert rotation.",
        "400": "Reasons about service identity: mTLS vs alternative identity mechanisms (JWT, SPIFFE), operational complexity of certificate management at scale, zero-trust service-to-service authentication, and the emerging patterns (workload identity, short-lived certificates) that simplify mTLS operations."
      }
    },
    {
      "domain": "Security",
      "question": "What is the difference between Authentication and Authorization?",
      "notes": "Authentication (AuthN) = proving WHO you are (password, token, biometric, certificate). Authorization (AuthZ) = what you're ALLOWED TO DO once authenticated (read file X, call API Y, access resource Z). You must authenticate before you can authorize, but they're separate concerns. Common mix-ups: a user can be authenticated but not authorized (403 Forbidden), or unauthenticated (401 Unauthorized). Typical implementations: OAuth/OIDC handles authN; RBAC (role-based access), ABAC (attribute-based, e.g., AWS IAM policies), or ReBAC (relationship-based, e.g., Google Zanzibar) handle authZ. Strong answers give a concrete example — 'I logged into the banking app (authN) but I can't see my coworker's account (authZ fails)' — and mention least-privilege as the authZ design principle.",
      "followup_questions": [],
      "level_guidance": {
        "100": "Has a rough idea -- 'who you are' vs 'what you can do'. May conflate them.",
        "200": "Clear distinction: Authentication = verifying identity (proving who you are), Authorization = determining permissions (what you're allowed to do). Gives examples: login is authn, IAM policy evaluation is authz. Knows they're separate steps.",
        "300": "Designs auth architecture: federated authentication (SAML/OIDC with corporate IdP), fine-grained authorization (ABAC vs RBAC, policy engines), token-based flows (OAuth2 + JWT), session management, authorization caching for performance.",
        "400": "Reasons about identity architecture: externalized authorization (OPA, Cedar/Amazon Verified Permissions), attribute-based access control at scale, continuous authorization (re-evaluate on context change), and the organizational challenge of consistent authorization across hundreds of services."
      }
    },
    {
      "domain": "Security",
      "question": "What is the difference between a stateful and a stateless firewall?",
      "notes": "Stateless: evaluates each packet independently against rules (source IP, dest IP, ports, protocol). Fast, simple, but has to allow BOTH directions of a connection explicitly — if you allow outbound 80, you must also allow inbound ephemeral-port responses. AWS NACLs are stateless, which is why they need separate inbound and outbound rules. Stateful: tracks active connections in a state table; once an outbound connection is allowed, the return traffic is automatically permitted. Most modern firewalls (Security Groups, iptables with conntrack, next-gen firewalls) are stateful. Strong answers note: stateful is easier to configure but has a state-table-size limit (can be DoS'd by flooding with half-open connections), while stateless scales better but is a pain to author correctly. Red flag: can't explain why AWS SGs are easier than NACLs — that's the stateful advantage.",
      "followup_questions": [],
      "level_guidance": {
        "100": "May know firewalls filter traffic but can't distinguish stateful/stateless.",
        "200": "Stateful: tracks connection state, automatically allows return traffic for established connections (AWS Security Groups). Stateless: evaluates each packet independently against rules, must explicitly allow both inbound and outbound (AWS NACLs). Trade-off: stateful is simpler to manage, stateless is faster but more rules needed.",
        "300": "Applied design: use Security Groups (stateful) as primary defense at instance level, NACLs (stateless) as additional subnet-level guardrail for explicit denies, understand performance implications at scale, knows when stateless is preferred (high-throughput, simple allow/deny decisions).",
        "400": "Reasons about filtering at scale: connection tracking overhead in stateful firewalls (memory, CPU), conntrack table exhaustion under DDoS, hardware offload for stateless processing, and the architectural pattern of using stateless at the edge (high volume, simple rules) with stateful deeper in the stack (complex policies, lower volume)."
      }
    },
    {
      "domain": "Big Data / Analytics",
      "question": "Give me an example of how you might use analytics to enhance a product or bring benefit to your company.",
      "notes": "Listen for concrete examples tied to business outcomes, not buzzwords. Good examples: (1) Funnel analytics — identifying where users drop off in signup/checkout and A/B testing interventions (e.g., moved 'create account' to after first action, conversion +15%). (2) Cohort retention — comparing users from different acquisition channels or product variants to prioritize channel spend. (3) Churn prediction — training a model on engagement signals, targeting at-risk users with retention offers. (4) Content recommendations — collaborative filtering on user behavior (Netflix, Amazon). (5) Operational analytics — p99 latency by customer segment drove capacity planning. Strong candidates say 'we went from X to Y and it saved/earned $Z' with specifics. Red flag: can only talk about vanity metrics (page views, total signups) without tying to action or outcome.",
      "followup_questions": [],
      "level_guidance": {
        "100": "Mentions basic reporting or dashboards.",
        "200": "Gives concrete examples: A/B testing to optimize features, recommendation engines based on user behavior, funnel analysis for conversion optimization, cohort analysis for retention. Connects analytics to business decisions.",
        "300": "Designs analytics-enhanced product: event tracking architecture, experimentation platform, real-time personalization pipeline, segmentation for targeted features, predictive analytics (churn prediction, LTV modeling), data-driven feature prioritization.",
        "400": "Analytics as product strategy: the feedback loop (measure -> learn -> build), causal inference vs correlation, long-term metric definition (engagement vs value), privacy-preserving analytics (differential privacy, aggregation), and building an experimentation culture (organizational readiness, statistical literacy, accepting negative results)."
      }
    },
    {
      "domain": "Big Data / Analytics",
      "question": "Imagine a big stream of data. Can you talk about the lifecycle of working with a big data stream?",
      "notes": "Stages: (1) Ingest — collect from many sources into the pipeline; tools: Kinesis Data Streams, Kafka, Kinesis Firehose, AWS IoT. Consider partitioning strategy (partition key) for throughput. (2) Process — transform, enrich, aggregate in real-time; tools: Kinesis Data Analytics, Flink, Spark Streaming, Lambda consumers. Key concepts: windowing (tumbling, sliding, session), event time vs processing time, watermarks for late data. (3) Store — raw to S3/data lake for replay and batch; processed/aggregated to a serving store (DynamoDB, OpenSearch, Redshift, Aurora). (4) Serve — dashboards (QuickSight, Grafana), APIs, alerts. (5) Monitor — lag, error rate, backlog size. (6) Govern — schema evolution, PII handling, retention. Strong answers mention exactly-once vs at-least-once semantics, reprocessing from the raw data lake, and the Lambda architecture (batch + speed layer) vs Kappa (streaming only). Red flag: only describes batch Hadoop/Spark — missed that it's a stream.",
      "followup_questions": [],
      "level_guidance": {
        "100": "Mentions ingest → process → store. May not know specific tools.",
        "200": "Names the stages: ingest (Kafka/Kinesis), buffer/decouple, transform (stream processor), persist (data lake / warehouse), query/analyse. Mentions schema enforcement.",
        "300": "Has built a streaming pipeline: exactly-once vs at-least-once semantics, checkpointing, watermarks for out-of-order events, late-arrival handling, schema registry, dead-letter handling, replay strategy, monitoring (lag + throughput + error rate).",
        "400": "Reasons about end-to-end correctness: idempotent sinks, transactional outboxes for source-side guarantees, the duality of streams and tables, when to use Kappa vs Lambda architecture, cost modelling for hot-path streaming vs micro-batch, and the operational cost of streaming vs batch (streaming pipelines are 10x more operational complexity than batch — be sure you need it)."
      }
    },
    {
      "domain": "Big Data / Analytics",
      "question": "Let us say that you need to provide real-time analysis of a microblogging/social network platform (e.g. people unhappy vs. people happy about a product announcement). What would be the technology components you would use?",
      "notes": "Pipeline: (1) Ingest — Kinesis/Kafka consuming from the platform's firehose/API (Twitter/X streaming API, Bluesky firehose). (2) Enrich — extract text, clean (remove retweets/bots), language detection. (3) Sentiment classification — either Amazon Comprehend (managed, multilingual) for simplicity, or a fine-tuned BERT/DistilBERT for domain-specific language, or Bedrock Claude for nuanced sentiment + aspect extraction. (4) Aggregate — windowed counts of positive/neutral/negative per minute, by keyword/hashtag/geo. (5) Store — DynamoDB for real-time counters, S3 for raw archive, OpenSearch for searchable timeline. (6) Serve — real-time dashboard (QuickSight, Grafana, or custom with WebSockets). Strong answers mention: bot filtering, multilingual handling, spikes detection for anomaly alerting, and that simple rules + keyword lists often outperform sentiment ML for specific product-mention detection. Bonus: topic modeling (LDA or embedding clusters) to surface emerging themes beyond sentiment.",
      "followup_questions": [
        {
          "question": "How would you analyze the results?",
          "notes": "(Parent: real-time social sentiment.) Multi-stage: (1) Real-time dashboard showing sentiment distribution (positive/negative/neutral) and volume over time, ideally sliced by geo, hashtag, demographic if available. (2) Anomaly detection — spike in negative sentiment triggers an alert to the comms team. (3) Aspect-based analysis — not just 'is this positive or negative' but 'positive about WHAT' — pricing, UX, features, support, shipping. Bedrock/Comprehend can extract entities and aspects. (4) Drill-down — click a spike → see the actual posts driving it, not just the aggregate number. (5) Comparison — vs prior launch, vs competitor mentions. (6) Feedback loop — route genuinely unhappy customers to support; surface feature requests to product. Strong answers mention: don't over-trust sentiment models (sarcasm, irony, domain jargon throw them off — human review on critical decisions), and connect the analysis to a concrete decision or action. Red flag: just reports numbers without proposing what's done with them."
        }
      ],
      "level_guidance": {
        "100": "Suggests storing tweets in a database and querying.",
        "200": "Stream processing approach: ingest with Kinesis, process with Lambda or KDA, aggregate/window for trends, store results. Knows about windowing concepts (tumbling, sliding).",
        "300": "Designs real-time analytics: Kinesis Data Streams for ingestion (partition by hashtag/user), Kinesis Data Analytics (Flink) for windowed aggregation and pattern detection, DynamoDB for real-time serving (trending topics), S3 for historical archive, OpenSearch for search/dashboards. Considers: late-arriving data, out-of-order events, exactly-once semantics.",
        "400": "Streaming architecture at scale: the CAP theorem applied to streaming (consistency vs availability of analytics), lambda vs kappa architecture trade-offs, streaming joins (enrichment from reference data), watermarks and completeness guarantees, and the operational challenges of streaming systems (backpressure, reprocessing after bugs, state management)."
      }
    },
    {
      "domain": "Big Data / Analytics",
      "question": "What is HDFS?",
      "notes": "Hadoop Distributed File System — the storage layer of the classic Hadoop ecosystem. Design: splits large files into 128MB blocks, replicates each block 3x across a cluster of commodity servers (DataNodes), with metadata on a central NameNode (the SPOF in classic Hadoop; high-availability HDFS uses paired NameNodes). Optimized for large sequential reads (analytics scans) not random I/O. Today it's mostly legacy — S3 + a query engine (Presto, Trino, Athena, Spark) has largely replaced HDFS because S3 is 11-nines durable, elastic, and decouples compute from storage. Strong answers mention that HDFS is still seen in on-prem enterprise data lakes and some Cloudera/Hortonworks deployments, but new builds on the cloud use object storage + a metastore (Glue, Hive Metastore, Iceberg). Red flag: thinks HDFS is still the default for new big-data projects.",
      "followup_questions": [
        {
          "question": "Why does it exist?",
          "notes": "(Parent: HDFS.) HDFS was built at Yahoo/Hadoop team circa 2006 to solve: how do you store and process petabyte-scale datasets on commodity hardware when no single server can hold it? Design principles: (1) Commodity hardware — assume disks fail regularly, so replicate blocks 3x. (2) Write-once-read-many workloads — simple consistency model (no random writes, just appends). (3) Bring compute to data, not data to compute — MapReduce jobs run on the same nodes that store the data blocks, avoiding huge network transfers. That was revolutionary in 2006 when SANs were the alternative. Today, S3 + compute-storage separation (Spark on S3, Athena, Trino) won because: (a) network is fast enough that locality doesn't matter as much, (b) S3 is 11-nines durable with no ops burden, (c) compute scales independently. Strong answers note HDFS is now legacy for new builds but still runs in enterprise on-prem data platforms."
        }
      ],
      "level_guidance": {
        "100": "Knows it's related to Hadoop/big data.",
        "200": "Explains: Hadoop Distributed File System -- stores large files across commodity hardware clusters with replication for reliability. Concepts: NameNode (metadata), DataNodes (storage), block replication (default 3x), write-once-read-many pattern.",
        "300": "Contextualizes: HDFS as the foundation of Hadoop ecosystem (MapReduce reads from HDFS), AWS equivalent is S3 (decoupled storage + compute), EMR uses HDFS for temporary storage during processing. Trade-offs vs S3: HDFS for locality-optimized processing, S3 for durability and decoupled compute.",
        "400": "Storage architecture evolution: HDFS represents co-located storage+compute era, modern data architecture separates them (S3 + Spark/Presto/Athena), the performance implications (S3 Select, Iceberg for ACID on object storage), and when HDFS-style locality still matters (extremely latency-sensitive iterative algorithms, though rare with modern network speeds)."
      }
    },
    {
      "domain": "Big Data / Analytics",
      "question": "What is MapReduce?",
      "notes": "A programming model for processing large datasets by splitting work into two phases: Map (transform each record independently, producing key-value pairs) and Reduce (aggregate values per key). Classic example: word count — map emits (word, 1) for each token; reduce sums the 1s per word. Strengths: embarrassingly parallel, fault-tolerant (re-run failed tasks), works on commodity hardware. Weaknesses: disk-heavy (intermediate data hits HDFS between phases), iterative algorithms are painful (each pass is a new job), high latency. Has been largely replaced by Spark (in-memory, DAG-based, 10-100x faster for iterative workloads) and cloud-native serverless equivalents (Glue, Dataflow, Athena). Strong candidates know MapReduce conceptually but say they'd reach for Spark, Flink, or SQL-on-S3 for real work today.",
      "followup_questions": [
        {
          "question": "Can you describe what a mapper function or reducer function might do?",
          "notes": "(Parent: MapReduce.) Classic word-count example: map function takes a line of text, tokenizes, and emits (word, 1) for each token. Shuffle stage (framework-managed) groups all (word, 1) pairs by word across the cluster. Reduce function receives (word, [1, 1, 1, ...]) and sums to emit (word, count). More realistic: log-analysis map reads each log line, extracts user_id and action, emits (user_id, action); reduce groups by user_id and computes distinct actions or sequences. Strong candidates describe: (1) Mappers are embarrassingly parallel — N mappers run independently across the cluster, no coordination. (2) Shuffle is where the magic AND the cost live — all records with the same key must end up on the same reducer, so data moves across the network. (3) Reducers can be chained for complex workflows. (4) Spark's `flatMap + reduceByKey` is the modern equivalent — same concept, faster because it avoids writing intermediate results to disk."
        }
      ],
      "level_guidance": {
        "100": "Knows it's a big-data programming model. May say 'Hadoop'.",
        "200": "Explains the two phases: map produces (key, value) pairs; reduce aggregates by key. Mentions shuffle and partitioning. Knows it's batch-oriented.",
        "300": "Picks MapReduce vs Spark vs Flink based on the workload: MR for throughput-bound batch on cheap storage, Spark for iterative / interactive, Flink for streaming. Understands data skew, combiner functions, and why MR fell out of favour for most workloads.",
        "400": "Reasons about the deeper computation model: MR as a special case of a DAG executor, why Spark's in-memory model wins for iterative jobs, how exactly-once semantics differ across the three engines, and when modern equivalents (BigQuery, Athena/Trino, Snowflake) replace the engine entirely with a managed query layer."
      }
    },
    {
      "domain": "Big Data / Analytics",
      "question": "When someone says \"Big Data,\" what does that mean to you? Can you provide examples?",
      "notes": "Classical definition — the 3 Vs (now 5): Volume (too big for a single machine — TB to PB), Velocity (ingested too fast for traditional DBs — millions of events/sec), Variety (text, video, logs, clickstreams — not just tabular). Plus Veracity (data quality) and Value (actionable insight). Pragmatically: data you can't reasonably fit or process on a single beefy server. Examples: clickstream from a large e-commerce site (10B events/day), IoT telemetry (100K devices @ 1Hz), social media firehose, CDN logs, genomics data. Strong answers note that 'big data' as a buzzword has faded — today we just say 'data engineering' and the tools (Spark, dbt, Snowflake, BigQuery, Databricks, Iceberg) are commodity. Red flag: says 'big data = Hadoop' or frames it around 2012-era tooling instead of modern data platforms.",
      "followup_questions": [],
      "level_guidance": {
        "100": "Says 'a lot of data'. Maybe names the 3 V's (volume/velocity/variety).",
        "200": "Articulates the 3 V's with examples: petabytes of clickstream (volume), millions of events/sec (velocity), heterogeneous semi-structured logs (variety). Names a real big-data tool (Hadoop, Spark, Kafka).",
        "300": "Frames 'big data' operationally: data that's too big or too fast for a single machine to process within the time budget. Discusses the cost-quality-latency triangle, data partitioning strategies, the 'small files problem,' and why most teams' 'big data' is actually a single Postgres instance away.",
        "400": "Reasons about architecture: lakehouse (Iceberg/Delta) vs warehouse vs lake, schema evolution at scale, governance (Lake Formation / Unity Catalog), the data-team org-design implications, and the deeper insight that 'big data' is a moving target — what was big in 2010 is a single laptop's RAM today."
      }
    },
    {
      "domain": "Big Data / Analytics",
      "question": "Why are NoSQL databases usually thought of as a better alternative for big data applications compared to traditional relational databases?",
      "notes": "Mainly horizontal scalability. Relational DBs are hard to shard — joins, transactions, and foreign keys make splitting across nodes complex. NoSQL designs trade some of that flexibility for scale: (1) Key-value / document (DynamoDB, MongoDB) — partition by key, scale to trillions of items with single-digit-ms reads. (2) Wide-column (Cassandra, HBase, BigTable) — massive write throughput, time-series friendly. (3) Graph (Neo4j, Neptune) — relationship-heavy queries. Trade-offs: weaker consistency (eventual by default), denormalized data (same value in multiple places), queries must be planned up front — no ad-hoc JOINs across arbitrary fields. Strong answers note that modern distributed SQL (Aurora, Spanner, CockroachDB) has narrowed the gap: you can now get SQL + horizontal scale. Pick NoSQL for predictable access patterns at massive scale; pick SQL when you need ad-hoc querying and transactions. Red flag: says 'NoSQL is better than SQL' without nuance — they're different tools.",
      "followup_questions": [],
      "level_guidance": {
        "100": "Knows NoSQL is 'not SQL' or 'for big data'.",
        "200": "Explains: schema flexibility for unstructured/semi-structured data, horizontal scaling (partition across nodes), optimized for specific access patterns (key-value, document, graph, column-family). Trade-offs: no complex joins, eventual consistency (often). Better for big data because: handles write-heavy workloads, scales horizontally, tolerates schema evolution.",
        "300": "Nuanced comparison: NoSQL isn't universally better -- it's better for specific patterns (high write throughput, known access patterns, denormalized data models). SQL is better for: complex queries, ad-hoc analysis, transactions, data integrity. Discusses specific AWS options: DynamoDB (key-value), DocumentDB (document), Neptune (graph), Keyspaces (wide-column).",
        "400": "Data architecture strategy: the NoSQL vs SQL debate is outdated -- modern systems are converging (DynamoDB adds transactions, Aurora adds JSON, DSQL adds serverless distribution). The real question is access pattern: if you know your queries upfront, NoSQL can be optimized; if queries are exploratory, SQL flexibility wins. Discusses multi-model approaches and the total cost of ownership (development complexity vs operational simplicity)."
      }
    },
    {
      "domain": "Big Data / Analytics",
      "question": "Why is JSON an attractive protocol for data encapsulation or moving data between loosely coupled systems?",
      "notes": "(1) Universal language support — every mainstream language parses/emits JSON natively or with one import; no IDL code-gen step like Protobuf. (2) Human-readable — trivial to debug with curl and a text editor, unlike binary formats. (3) Self-describing — the structure is in the payload, so consumers can handle partial schema knowledge gracefully. (4) Flexible — optional fields, nested structures, arrays, no strict type enforcement, schema can evolve without breaking old consumers (as long as you only add fields). (5) Native to the web — JavaScript `JSON.parse`, REST conventions. Trade-offs (note as mature balance): verbose vs Protobuf, slower to parse than binary, no enforced schema (use JSON Schema / OpenAPI), no native dates. Strong answers mention that for internal high-throughput service-to-service traffic, Protobuf/Avro/MessagePack beat JSON on size + speed; JSON remains king at the public-API boundary.",
      "followup_questions": [],
      "level_guidance": {
        "100": "Knows JSON is lightweight and readable.",
        "200": "Explains: self-describing (keys in the data), language-agnostic, easy to parse in any language, maps naturally to objects/documents, widely supported by APIs and databases. Advantages over XML (less verbose) and CSV (nested structures, mixed types).",
        "300": "Applied context: JSON in event-driven architectures (message payloads), API design (REST standard), document databases (native storage format), logging (structured logs), configuration. Discusses limitations and when alternatives are better (Avro/Parquet for big data, Protobuf for performance).",
        "400": "Data format strategy at scale: JSON's flexibility becomes a liability without governance (schema drift, type inconsistencies), the role of schema registries, binary encoding for performance-critical paths, and how data format choices in year 1 compound into integration challenges in year 5."
      }
    },
    {
      "domain": "Storage Expertise",
      "question": "I have a database on a physical server that needs more disk space.  Walk me through the steps you'd take to add storage capacity to the server to make it useable for the database.  Assume you have all necessary assets on hand.  Use whatever OS you'd like….",
      "notes": "Linux example: (1) Install the drive(s) physically or via iSCSI/SAN. (2) Run `lsblk` or `fdisk -l` to identify the new device (/dev/sdc). (3) Create a partition (`parted` or `fdisk`), format it with a filesystem (XFS for databases — no fragmentation issues, or ext4), (4) Add to LVM if using: `pvcreate /dev/sdc1`, `vgextend` the volume group, `lvextend -L +500G /dev/vg0/data`, then `xfs_growfs` or `resize2fs` to expand the filesystem online. (5) Or mount as a new mount point and migrate data. (6) Update /etc/fstab with UUID for persistence. (7) Verify with `df -h` and validate DB sees the space. Strong candidates mention: do this during low-traffic window, snapshot first, test with a non-prod copy, and consider whether adding capacity masks a growth problem that needs partitioning/archiving instead.",
      "followup_questions": [],
      "level_guidance": {
        "100": "Suggests adding a bigger disk.",
        "200": "Options: add additional disks (expand storage), RAID for performance/redundancy, LVM for logical volume management (extend without downtime), migrate to larger storage. Cloud: resize EBS volume, add EBS volumes, use EFS for elastic storage.",
        "300": "Considers the fuller picture: why is it running out (data growth rate? should old data be archived?), options by urgency (extend LVM now, plan migration to appropriate storage), database-specific considerations (tablespace management, partition pruning, archival strategy), monitoring to prevent recurrence (CloudWatch disk utilization alarms).",
        "400": "Storage architecture decision: this is a symptom, not the problem. Questions: is the data model causing bloat (no archival strategy, no TTL)? Should the database be split (sharding, read replicas)? Is the right storage engine being used? Designs sustainable data lifecycle management rather than solving the immediate capacity issue repeatedly."
      }
    },
    {
      "domain": "Storage Expertise",
      "question": "What is the difference between durability and availability?",
      "notes": "Durability = probability your data is NOT lost — measured as nines (S3 Standard advertises 11 nines, or 99.999999999%, meaning on average if you store 10 million objects you would expect to lose one on average every 10,000 years). Achieved via replication, erasure coding, and checksums. Availability = probability the system is READABLE when you try to access it (S3 Standard = 99.99% = ~52 min downtime/year). Strong candidates note that you can have high durability but low availability (data is safe but temporarily unreachable during an outage) — and vice versa is bad (high availability but losing data = corruption). Design implication: pick storage tiers by the ratio you need (S3 Glacier = same durability as Standard but lower availability for cold data).",
      "followup_questions": [],
      "level_guidance": {
        "100": "Mixes them up or treats them as the same.",
        "200": "Defines them clearly: durability = data not lost (S3's 11 9s); availability = service reachable (S3 Standard's 99.99%). Knows you can have one without the other (e.g., archived data: durable but slow to retrieve).",
        "300": "Picks storage classes deliberately: S3 Standard for hot, IA / Glacier Instant for warm, Glacier Deep Archive for cold compliance. Discusses RPO/RTO targets, cross-region replication for both DR and lower latency, MFA-delete for ransomware protection.",
        "400": "Reasons about the engineering behind 11-nines durability: erasure coding, geo-distributed replicas, scrubbing for silent corruption, write-quorum semantics, and the deeper insight that 'availability' is what users feel and 'durability' is what the company can prove in court — both need explicit SLOs and verifiable backups."
      }
    },
    {
      "domain": "Storage Expertise",
      "question": "What are SAN, NAS, and DAS?",
      "notes": "DAS (Direct-Attached Storage): disks physically attached to one server (SATA, SAS, NVMe) — fastest, simplest, but only that server can use it. NAS (Network-Attached Storage): file-level access over a network via NFS or SMB/CIFS — multiple clients share the same filesystem; great for user home directories and media libraries. SAN (Storage Area Network): block-level access over a dedicated network (Fibre Channel, iSCSI) — each server sees a 'LUN' as if it were a local disk; used for databases and virtualization. Strong answers map each to a use case: DAS for single-host, NAS for shared files, SAN for shared block storage. Bonus: on AWS this maps to instance store (DAS), EFS/FSx (NAS), EBS (SAN-like block).",
      "followup_questions": [],
      "level_guidance": {
        "100": "Has heard the acronyms. May get one right.",
        "200": "Defines all three: DAS = directly attached (single-server), NAS = file-level network share (NFS/SMB), SAN = block-level network storage (iSCSI/FC). Names use cases for each.",
        "300": "Picks deliberately: DAS / NVMe local for lowest-latency single-host workloads, NAS for shared file workloads (EFS / FSx for OpenZFS), SAN-like block (EBS / FSx for Lustre) for shared block. Discusses backup, snapshot, replication strategy per choice.",
        "400": "Reasons about modern equivalents: object storage (S3) replacing many SAN/NAS use cases, the cost / consistency / latency tradeoff of each, multi-attach EBS for HA cluster storage, FSx for Lustre for HPC, and the deeper insight that the SAN/NAS/DAS taxonomy is increasingly less relevant — workloads now choose access pattern (object / block / file / streaming) and the cloud picks the substrate."
      }
    },
    {
      "domain": "Content Delivery",
      "question": "Can you talk about the importance of internationalization or localization when delivering content to your end users?",
      "notes": "i18n (internationalization) = designing the app so content CAN be translated/adapted — extracting strings, supporting Unicode end-to-end, right-to-left layout (Arabic, Hebrew), date/number/currency formatters that respect locale, plural rules that differ by language (Russian has 3, Arabic has 6). l10n (localization) = actually translating and culturally adapting for a specific locale. CDN relevance: serve region-appropriate content from edge locations — CloudFront origin-request policy or Lambda@Edge can pick the variant based on CloudFront-Viewer-Country header. Considerations: (1) Don't hard-code strings, use message catalogs (ICU MessageFormat, gettext). (2) SEO — hreflang tags, per-locale URLs. (3) Legal/regulatory — GDPR in EU, accessibility laws vary. (4) Testing — pseudo-localization (wrap strings in Xs) to catch hard-coded text. Strong answers note i18n is MUCH cheaper to bake in early than retrofit later.",
      "followup_questions": [],
      "level_guidance": {
        "100": "Knows translations are needed. May mention language files.",
        "200": "Distinguishes i18n (architecture for locale support) from l10n (actual translation). Knows about Unicode/UTF-8, locale-aware formatting (dates, numbers, currency), RTL layout support.",
        "300": "Designs i18n infrastructure: content management for translations, CDN-based locale routing, URL structure strategy (/en/ vs subdomains), dynamic content localization, testing across locales, fallback strategies.",
        "400": "Considers i18n as a product strategy: cultural adaptation beyond translation, legal/compliance per region (GDPR, data residency), performance implications of serving locale-specific assets, A/B testing across markets, and accessibility as a dimension of localization."
      }
    },
    {
      "domain": "Content Delivery",
      "question": "Can you think of an example of when you'd need to pass query-string parameters to a CDN?",
      "notes": "CDN caches by URL, and by default query strings are either ignored or treated as part of the cache key. Good reasons to pass query strings through: (1) A/B testing — `?variant=a` vs `?variant=b` must return different content (cache key on variant). (2) Image transformation — `?w=400&h=300&fit=cover` generates resized images at the edge. (3) Cache-busting for versioned assets — `?v=1234abc` forces a new fetch when asset changes. (4) Signed URLs for private content — `?Signature=...&Expires=...` for authorized access. (5) Locale or currency — `?lang=fr&cur=EUR`. Reasons NOT to pass: tracking parameters like `?utm_source=...` pollute the cache without changing content — strip them at the edge. CloudFront configuration: Cache Policy controls which query strings are cached, Origin Request Policy controls which are forwarded upstream. Strong answers distinguish 'cache on' vs 'forward to origin' — they're different levers.",
      "followup_questions": [],
      "level_guidance": {
        "100": "Knows query strings exist in URLs but unclear on CDN implications.",
        "200": "Understands query strings create unique cache keys in CDN. Knows when to forward vs ignore (signed URLs, API parameters vs tracking params).",
        "300": "Designs caching strategy: whitelist specific query parameters to forward, normalize ordering for cache hit rate, use Lambda@Edge to strip/rewrite, signed URLs/cookies for access control without cache-busting.",
        "400": "Reasons about cache key design as a system optimization: cache hit ratio analysis, A/B testing through query params vs cookies vs headers, personalization at edge without destroying cache rates, and cost implications of cache miss rates at scale."
      }
    },
    {
      "domain": "Content Delivery",
      "question": "How would you help a customer with short life span content that needs to be massively distributed globally?",
      "notes": "Short-TTL content (minutes to hours) at global scale — sports scores, breaking news, stock tickers, live event metadata. Architecture: (1) Use a global CDN (CloudFront, Cloudflare, Fastly) — don't reinvent geo-distribution. (2) Edge compute (CloudFront Functions, Lambda@Edge, Cloudflare Workers) for per-request personalization without hitting origin. (3) Short TTL + stale-while-revalidate semantics so clients get cached-but-fresh content, and a background refresh happens transparently. (4) Cache invalidation on publish — invalidate specific paths (not `/*`) via CloudFront's invalidate API, or use versioned URLs to invalidate by URL change. (5) Origin shield to reduce origin load during popular events. (6) For true real-time (sub-second): shift to WebSocket or HTTP/2 server-push or API Gateway WebSockets; CDNs don't help there. Strong answers mention capacity planning for flash events and how to use request collapsing / cache-key normalization to protect the origin during spikes.",
      "followup_questions": [],
      "level_guidance": {
        "100": "Suggests email or 'put it on a website'. No architecture for mass distribution.",
        "200": "Uses CDN for distribution, knows about pre-warming/pre-loading CDN caches, origin scaling for burst reads. Mentions S3 + CloudFront.",
        "300": "Designs for viral/burst distribution: origin shield to protect origin, auto-scaling origin, TTL strategy for short-lived content, invalidation workflow, signed URLs with time-limited access, multi-region replication for global audience.",
        "400": "Reasons about live/ephemeral content at scale: streaming vs download trade-offs, peer-to-peer augmentation, cost of serving TB in hours vs days, capacity planning for predictable bursts (events, launches), and the infrastructure economics of short-lived CDN content."
      }
    },
    {
      "domain": "Content Delivery",
      "question": "Imagine your employees are spread worldwide. What methods might you employ to make sure only authorized employees could get content from your CDN?",
      "notes": "Options ranked by strength: (1) Signed URLs / signed cookies — CloudFront and Cloudflare issue time-limited, tamper-evident URLs; auth service generates them after verifying the user's session. Good for private video, downloads. (2) JWT verification at the edge — Lambda@Edge or CloudFront Functions verify a Cognito/Auth0 JWT before allowing the cache hit, reject otherwise. (3) mTLS on the client — strong identity (certificate per device/user) but operationally complex. (4) IP allow-listing — fragile with remote workers (IPs change), use only as one layer. (5) Zero-trust proxies — Cloudflare Access, AWS Verified Access, Cloudflare Tunnel — puts an auth gate in front of the CDN. Defense in depth: combine JWT auth + signed URLs for download links. Strong answers note that CDN content is cached — so if you don't auth, anyone who gets the URL can download. Also: block common bypasses (direct origin access) via origin access controls.",
      "followup_questions": [],
      "level_guidance": {
        "100": "Suggests VPN. May not consider content delivery optimization.",
        "200": "Knows: CDN for static content, Global Accelerator for dynamic, regional application deployment, caching strategies. Mentions latency testing.",
        "300": "Designs global delivery: multi-region active-active or active-passive, Global Accelerator for TCP optimization, CloudFront for content, database replication strategy, DNS-based failover, employee-specific considerations (VPN split tunneling, workspace solutions).",
        "400": "Reasons about global architecture decisions: data sovereignty constraints per region, consistency vs latency trade-offs (CAP theorem in practice), cost of multi-region vs acceptable latency, network path optimization, and organizational model (central platform team vs regional autonomy)."
      }
    },
    {
      "domain": "Content Delivery",
      "question": "What is a CDN?",
      "notes": "Looking for examples of common CDNs: CloudFront, Akami, etc",
      "followup_questions": [
        {
          "question": "When would you use a CDN?",
          "notes": "(Parent: What is a CDN.) Use a CDN when: (1) Globally-distributed users — latency benefits are proportional to user-to-server distance. (2) Static or cacheable assets — images, JS, CSS, video, downloadable binaries. (3) Traffic spikes — CDN absorbs the flash crowd before it hits your origin (launch days, news events, viral content). (4) DDoS resilience — CDN IPs absorb volumetric attacks; origin is hidden. (5) Reduce origin load — offload 90%+ of requests, shrink origin capacity. (6) Edge functions for personalization at low latency (CloudFront Functions, Cloudflare Workers, Lambda@Edge). Less useful for: (a) real-time personalized content that can't be cached (though edge compute + origin shield helps), (b) internal-only apps with no public traffic, (c) tiny apps where the CDN's min-cost exceeds origin cost. Strong answers mention that a CDN pays back quickly at any meaningful scale — even modest apps see p95 latency improvements from edge TLS termination alone."
        }
      ],
      "level_guidance": {
        "100": "Knows CDN speeds up websites. May say 'it caches things closer to users'.",
        "200": "Explains: network of edge locations caching content close to users, reduces latency and origin load, handles static and dynamic content differently. Names CloudFront, Akamai, Cloudflare.",
        "300": "Explains CDN architecture: PoP/edge location topology, origin groups for failover, cache behaviors and TTLs, invalidation strategies, signed URLs for access control, Lambda@Edge/CloudFront Functions for edge compute. Discusses when NOT to use a CDN.",
        "400": "Reasons about CDN as infrastructure: anycast networking, cache hierarchy design, origin shielding economics, edge compute for personalization without origin round-trips, CDN selection criteria (CloudFront vs multi-CDN), and the strategic value of controlling the edge layer."
      }
    },
    {
      "domain": "Infrastructure Expertise",
      "question": "Can you explain to me the concept of containerization?",
      "notes": "I would want to hear docker, I would want to hear someone draw a correlation between virtualization of hardware to almost a \"software\" virtualization so that you can do some multi-tenant stuff on one guest OS.",
      "followup_questions": [
        {
          "question": "Why would you want to take advantage of it?",
          "notes": "(Parent: containerization.) Key benefits: (1) Consistency — 'works on my machine' is eliminated; the container is the machine. Same image runs locally, in CI, in staging, in prod. (2) Density — containers share the host kernel, so you can run 10-100 containers on a VM where you'd only run 1-2 VMs. Much better hardware utilization. (3) Fast startup — containers start in ~1s vs 30-60s for VMs. Enables auto-scaling and burst capacity. (4) Portable — Docker images run on Linux, Windows, macOS; same image runs on laptop and cloud. (5) Ecosystem — k8s, ECS, Fargate, App Runner all speak the OCI container standard. (6) Immutable artifacts — you deploy a fixed image, not 'run this install script on a new server'. Mentions: containers aren't full isolation (kernel is shared; rootless containers and gVisor improve this), storage is ephemeral by default, and orchestration (k8s) is where most complexity lives. Red flag: 'because Docker is cool' — not a reason."
        }
      ],
      "level_guidance": {
        "100": "Knows containers package an app with its dependencies. Names Docker.",
        "200": "Explains containers vs VMs (shared kernel vs hypervisor), image layers, registry. Has run docker build / docker run. Mentions Kubernetes.",
        "300": "Has shipped containerized workloads: multi-stage builds for small images, distroless / scratch base, healthchecks, resource limits, secrets handling (no baked-in secrets), per-image SBOM, rootless containers, immutable tags + digest pinning.",
        "400": "Reasons about the deeper containment story: cgroups and namespaces as the actual mechanism, gVisor / Kata for stronger isolation when multi-tenant, OCI image-spec evolution, supply-chain attacks (typosquatted base images, malicious layers), and the cost-of-ownership choice between EKS / ECS Fargate / App Runner / Lambda containers based on operational appetite."
      }
    },
    {
      "domain": "Infrastructure Expertise",
      "question": "Have you ever used Infrastructure/System Monitoring tools?",
      "notes": "Listen for specific tool names and what they're good at: CloudWatch (AWS metrics/logs/alarms — deep AWS integration but expensive at scale), Datadog (APM + infra + logs in one, easy UX, pricey), Prometheus + Grafana (open-source, pull-based metrics, the cloud-native default), Nagios/Zabbix (legacy host monitoring, still common in on-prem), New Relic (APM-first), Splunk (log analytics). Strong answers cover the four signals from the SRE book — latency, traffic, errors, saturation — and explain the difference between metrics (time-series numbers), logs (structured events), and traces (distributed request flow). Also listen for: has the candidate actually set up an alert that paged them at 3 AM? That's operator experience, not just tool familiarity.",
      "followup_questions": [
        {
          "question": "If yes, which ones?",
          "notes": "(Parent: monitoring tools.) Listen for specific, named tools + what they're for. Metrics: CloudWatch (AWS-native, expensive at scale), Datadog (full-stack, premium), Prometheus + Grafana (open-source, self-hosted). Logs: CloudWatch Logs, Splunk (enterprise, expensive), ELK/OpenSearch, Loki (cheaper open-source, pairs with Grafana). APM/Tracing: Datadog APM, New Relic, Honeycomb (query-based, for debugging complex prod issues), AWS X-Ray. Synthetic: Pingdom, StatusCake, Uptime Robot. Infrastructure: Nagios (legacy), Zabbix, LibreNMS. Distributed tracing: OpenTelemetry is the modern vendor-neutral standard. Strong answers describe what they actually did with the tool: set up an alert, debugged an incident, identified a capacity issue. Red flag: lists 10 tools but can't describe how any of them helped solve a real problem."
        }
      ],
      "level_guidance": {
        "100": "Names CloudWatch or Datadog. May not articulate why monitoring matters.",
        "200": "Distinguishes metrics / logs / traces. Has set up dashboards and basic threshold alerts. Mentions uptime monitoring + on-call.",
        "300": "Has built observability for a real system: USE/RED method for service health, structured JSON logs with correlation IDs, distributed tracing (X-Ray, OpenTelemetry), synthetic canaries, SLO + error budget burn alerts, alert fatigue management.",
        "400": "Reasons about observability as a first-class deliverable: what's actionable vs noise, the 3 pillars + profiles + events as a unified telemetry plane, OpenTelemetry as the interop layer, cost discipline (high-cardinality metrics are expensive), and the deeper truth that you can only operate what you can observe — and you can only observe what you instrumented at design time."
      }
    },
    {
      "domain": "Infrastructure Expertise",
      "question": "You've been tasked with identifying the level of efficiency of hardware use in your datacenters, how would you approach this?",
      "notes": "Start with measurement: (1) PUE (Power Usage Effectiveness) = total facility power / IT equipment power — industry target is <1.5, hyperscalers hit <1.1. (2) Per-server utilization — CPU, memory, network, disk. Typical on-prem servers run 10-20% average CPU, meaning massive waste. (3) Rack density — kW per rack vs capacity. Mitigations: virtualization/containers to consolidate workloads, right-sizing (many VMs are over-provisioned), power down or decommission idle servers, use cooling zoning and hot/cold aisle containment, switch to ARM/AMD for more perf/watt. Strong answers mention carbon intensity (gCO2/kWh) as a modern extension. Red flag: can only talk about 'buying more energy-efficient servers' without mentioning utilization.",
      "followup_questions": [
        {
          "question": "What tools would you use?",
          "notes": "(Parent: datacenter efficiency.) Datacenter-level: DCIM tools (Schneider EcoStruxure, Nlyte, Sunbird) for power/cooling/capacity. Per-server utilization: Prometheus + node_exporter for Linux, perfmon/telegraf for Windows — track CPU, memory, disk, network. Cloud: CloudWatch + Trusted Advisor + Cost Explorer for AWS; GCP's Recommender; Azure Advisor. Right-sizing tools: AWS Compute Optimizer, Densify, CloudHealth, Spot.io. Carbon analytics: AWS Customer Carbon Footprint Tool, Microsoft Sustainability Calculator. For power: intelligent PDUs report per-outlet consumption; Modbus/SNMP polling. Strong candidates mention the process: baseline current state -> identify top under/over-utilized assets -> consolidate or decommission -> measure impact -> repeat quarterly. Red flag: names tools but has no process."
        }
      ],
      "level_guidance": {
        "100": "Suggests checking if servers are busy.",
        "200": "Metrics: CPU utilization, memory usage, disk I/O, network throughput. Tools: CloudWatch, Trusted Advisor, Compute Optimizer. Identifies underutilized instances for rightsizing.",
        "300": "Designs efficiency assessment: baseline utilization across fleet, identify waste (idle instances, oversized instances, unattached EBS), rightsizing recommendations with performance validation, scheduling (stop dev/test off-hours), Reserved/Savings Plan coverage for steady-state, Spot for fault-tolerant workloads.",
        "400": "Infrastructure efficiency as an organizational practice: unit economics (cost per transaction, not just utilization percentage), capacity planning with traffic modeling, Graviton migration ROI analysis, and the organizational operating model for continuous optimization (FinOps team, engineering accountability, automated recommendations pipeline)."
      }
    },
    {
      "domain": "Network Expertise",
      "question": "Can you explain the difference between Recovery Point Objective  and Recovery Time Objective?",
      "notes": "RPO = maximum acceptable data loss, measured in time (e.g., 'we can afford to lose 5 minutes of data') — drives backup frequency and replication strategy. RTO = maximum acceptable downtime before service is restored (e.g., 'we must be back up within 1 hour') — drives failover and recovery automation. RPO/RTO are business decisions, not technical ones, and they directly dictate cost: near-zero RPO needs synchronous replication (expensive); a 24h RPO can use nightly snapshots. Strong candidates pair each number with a real technique: RPO<1min = multi-AZ sync replication; RTO<5min = hot-standby or active-active; RTO<1h = warm standby with automated failover; RTO>4h = restore from backup.",
      "followup_questions": [],
      "level_guidance": {
        "100": "May know one but confuse them.",
        "200": "RPO: maximum acceptable data loss (time between last backup and failure). RTO: maximum acceptable downtime (time to restore service). Lower values = more expensive architecture. Examples: RPO=0 needs synchronous replication; RTO=minutes needs automated failover.",
        "300": "Designs DR architecture mapped to RPO/RTO: Backup/Restore (high RPO/RTO, cheap), Pilot Light (lower RTO, minimal always-on), Warm Standby (lower both), Multi-site Active-Active (near-zero both, expensive). Maps to customer requirements and budget. Tests regularly.",
        "400": "DR as business risk management: RPO/RTO are derived from business impact analysis (cost of downtime per hour, regulatory requirements for data durability), the relationship between stated RTO and actual recovery (untested DR plans fail), testing strategies (game days, chaos engineering), and the architectural patterns that make DR transparent (multi-region active-active eliminates the concept of 'recovery')."
      }
    },
    {
      "domain": "Network Expertise",
      "question": "Can you talk about any layers of the OSI model?",
      "notes": "The 7 layers (bottom-up): Physical (cables, signals), Data Link (MAC, switching, Ethernet frames), Network (IP, routing), Transport (TCP/UDP, ports, reliability), Session (connection state), Presentation (encoding, TLS), Application (HTTP, DNS, SMTP). Interviewer listen for: candidate can name at least 4-5 layers, knows TCP is layer 4 not 7, understands that TLS is presentation/session and HTTP is application. Strong answers tie each layer to a troubleshooting scenario (e.g., 'layer 2 issues = flapping switch ports; layer 3 = routing table problems; layer 4 = port blocked by firewall'). Red flag: confuses TCP and IP or can't name anything above layer 4.",
      "followup_questions": [
        {
          "question": "Can you give examples of what operates at each layer?",
          "notes": "(Parent: OSI layers.) L1 Physical — Ethernet cables (Cat6), fiber, RJ45 connectors, radio waves (Wi-Fi). L2 Data Link — Ethernet frames, MAC addresses, switches, ARP, VLANs (802.1Q), PPP. L3 Network — IP (v4, v6), ICMP (ping), routers, BGP, OSPF. L4 Transport — TCP (reliable, ordered), UDP (fast, unordered), port numbers. L5 Session — NetBIOS, SMB, RPC. L6 Presentation — TLS/SSL (often shoved in here or L5), compression (gzip), encoding (UTF-8). L7 Application — HTTP, HTTPS, DNS, SMTP, FTP, SSH, WebSocket, gRPC. Strong candidates note that layers 5 and 6 are mostly academic today — TCP/IP's original 4-layer model (link, internet, transport, application) maps more cleanly to real systems. Useful mnemonic: 'Please Do Not Throw Sausage Pizza Away' (L1→L7). Red flag: can't give any example beyond HTTP and TCP."
        }
      ],
      "level_guidance": {
        "100": "Names some layers (physical, network, transport, application). Mixes up the order.",
        "200": "Names all 7 in order, gives an example protocol per layer (Ethernet at L2, IP at L3, TCP at L4, HTTP at L7). Knows TCP/IP doesn't map cleanly to OSI.",
        "300": "Uses the OSI lens to debug: 'is the issue at L4 (TCP handshake failing) or L7 (TLS cert expired)?'. Knows L7 load-balancer vs L4 NLB tradeoffs, where ALB / NLB / Gateway Load Balancer fit on AWS, and why TLS termination at L7 enables host-based routing.",
        "400": "Reasons about the model's limits: QUIC/HTTP/3 collapsing several layers, TLS now sitting between L4 and L7 (SNI is L7 inside L4 payload), modern overlay networks (Istio/Linkerd) adding L7 controls, and the deeper insight that OSI is a mental model for diagnosis — not a literal blueprint of any real stack."
      }
    },
    {
      "domain": "Network Expertise",
      "question": "Describe the difference between unicast and multicast. Can you explain when you would use unicast vs  multicast transmission?",
      "notes": "Unicast = one-to-one (each sender-receiver pair has its own stream); multicast = one-to-many (sender emits one stream, the network replicates it to all subscribers). Multicast saves bandwidth for live distribution (IPTV, stock-market tickers, video conferencing to many receivers) because the source only sends once. Unicast is used for virtually all general internet traffic because multicast isn't supported across the public internet — only within managed networks. Bonus: mention broadcast (one-to-all on a subnet) and anycast (one-to-nearest, used by CDNs and DNS roots). Red flag: confuses multicast with broadcast, or thinks multicast works over the public internet.",
      "followup_questions": [],
      "level_guidance": {
        "100": "May not know the terms.",
        "200": "Unicast: one-to-one communication (standard IP traffic). Multicast: one-to-many (single transmission reaches multiple receivers simultaneously). Broadcast: one-to-all. Use case for multicast: video streaming, software updates to many hosts, market data distribution.",
        "300": "Applied in cloud: AWS doesn't support traditional IP multicast in VPC (design around it), alternatives (application-layer fanout with SNS/SQS, Transit Gateway multicast for specific use cases), on-prem multicast for media/trading floor. Understands IGMP for group management.",
        "400": "Network design implications: multicast at scale (IGMP snooping, PIM routing), why cloud providers avoided it (complexity of multi-tenant multicast), modern alternatives (application-layer pub/sub more flexible), and specific workloads that genuinely need network-layer multicast (live video encoding, financial trading, cluster communication) vs those that can use application-layer alternatives."
      }
    },
    {
      "domain": "Network Expertise",
      "question": "How does the process of translating a hostname to an IP address work?",
      "notes": "DNS resolution, roughly: (1) browser/OS checks local DNS cache and hosts file, (2) queries the configured recursive resolver (ISP/8.8.8.8/Route 53), (3) if not cached, resolver queries root servers (.), which return the TLD server (.com), (4) TLD returns the authoritative nameserver for the domain, (5) authoritative server returns the A/AAAA record, (6) resolver caches with TTL and returns to client. Strong candidates mention CNAME chains, TTL's role in caching, that EDNS0 enables larger UDP responses, and that DoH/DoT encrypt the queries. Red flag: jumps straight to 'DNS gives you an IP' without naming any of the resolution steps.",
      "followup_questions": [],
      "level_guidance": {
        "100": "Says 'DNS does it'. Knows /etc/hosts exists.",
        "200": "Walks the chain: stub resolver → recursive resolver → root → TLD → authoritative. Knows about A vs AAAA vs CNAME records and TTL caching.",
        "300": "Has debugged DNS in production: split-horizon DNS, DNSSEC, glue records, geo-DNS / latency-routing (Route 53), failover health-checks, 0.0.0.0 vs NXDOMAIN responses. Knows why low TTLs hurt cache hit rates and why high TTLs delay failover.",
        "400": "Reasons about DNS as critical infra: DDoS resilience (anycast, multiple authoritative providers), DoH/DoT privacy implications, propagation truthfulness (TTL is a recommendation, not a guarantee), service-discovery alternatives at scale (Consul, Cloud Map, mesh-native), and the operational rule that 'it's always DNS' is a meme because DNS failures cascade everywhere."
      }
    },
    {
      "domain": "Network Expertise",
      "question": "What are ephemeral ports?",
      "notes": "Short-lived source-side TCP/UDP ports that the OS allocates dynamically when a client opens an outbound connection. IANA-suggested range is 49152-65535; Linux uses 32768-60999 by default; Windows uses 49152-65535. They pair with the well-known destination port (e.g., client:54321 -> server:443) to uniquely identify the connection in the 4-tuple. Strong answers mention port exhaustion as a real production issue (common with outbound-heavy services like proxies or NAT gateways) and how tuning net.ipv4.ip_local_port_range or using connection pooling mitigates it. Also: AWS NAT Gateway allocates 55000 ephemeral ports per unique destination — a common cold-start scaling pitfall.",
      "followup_questions": [],
      "level_guidance": {
        "100": "May not know the term.",
        "200": "Explains: short-lived ports (typically 1024-65535) assigned by the OS for client-side connections. When you connect to a server on port 443, your OS picks a random ephemeral port for the return traffic. Important for firewall/NACL rules (must allow ephemeral port range for return traffic).",
        "300": "Applied knowledge: NACL rules must allow ephemeral port range (1024-65535) for outbound responses, Security Groups handle this automatically (stateful), port exhaustion under heavy load (too many connections from one client), NAT Gateway/instance port allocation limits, and how this affects containerized applications (many containers sharing host ports).",
        "400": "Networking depth: port exhaustion as a real scalability concern (NAT Gateway throughput limits, connection tracking table overflow), SO_REUSEPORT for high-performance servers, TIME_WAIT state consuming ports, and the architectural implications for high-connection-count applications (connection pooling, HTTP/2 multiplexing to reduce port consumption)."
      }
    },
    {
      "domain": "Network Expertise",
      "question": "What are three different network routing protocols?",
      "notes": "Classic examples: OSPF (link-state, within an AS — builds a full topology map and runs Dijkstra), BGP (path-vector, between autonomous systems — the protocol that runs the public internet), RIP (distance-vector, legacy, 15-hop max — still seen in tiny networks). Bonus: EIGRP (Cisco hybrid), IS-IS (used by large ISPs). Strong answers distinguish IGP (OSPF, EIGRP, RIP, IS-IS — inside your own network) from EGP (BGP — between organizations), and mention that BGP decisions are policy-driven not just shortest-path. Red flag: can only name one, or mistakes a routed protocol (IP) for a routing protocol.",
      "followup_questions": [
        {
          "question": "How do these protocols differ?",
          "notes": "(Parent: routing protocols.) OSPF: link-state — each router floods its link info to every other router in the area, all routers compute the full topology and run Dijkstra's shortest-path. Fast convergence (seconds), scales to medium networks, supports hierarchy via areas. Within an autonomous system only (IGP). BGP: path-vector — routers exchange full AS-path info to destinations; decisions are policy-driven (longest prefix match, local preference, AS-path length). Slow convergence (minutes — intentional for stability), scales to the entire internet. Between autonomous systems (EGP). RIP: distance-vector — each router shares only hop count to its neighbors; neighbors update their tables. Simple but slow convergence (several rounds), 15-hop max (16 = infinity), legacy. Strong candidates tie each to a scenario: 'OSPF inside a data center for rapid failover; BGP for peering with my ISP; RIP for a lab toy'. Mentions ECMP (equal-cost multi-path) for load balancing across links."
        }
      ],
      "level_guidance": {
        "100": "May name one or confuse with other networking concepts.",
        "200": "Names and explains: BGP (internet backbone, path vector, autonomous system routing), OSPF (interior gateway, link-state, fast convergence), RIP (distance vector, simple but limited hop count). Knows when each is appropriate.",
        "300": "Applied context: BGP for Direct Connect and Transit Gateway peering (customers need to understand BGP for hybrid connectivity), OSPF/EIGRP inside corporate networks, route propagation in AWS (VPC route tables, TGW route tables), and how to troubleshoot routing (route priority, longest prefix match, asymmetric routing).",
        "400": "Routing at scale: BGP as the 'duct tape of the internet' (fragile but flexible), route filtering and security (RPKI, route origin validation), traffic engineering with BGP communities, multi-homed Direct Connect failover design, and the emerging patterns (segment routing, SD-WAN) that abstract away traditional routing protocol complexity."
      }
    },
    {
      "domain": "Network Expertise",
      "question": "What is a virtual interface?",
      "notes": "A software-defined network interface not tied to a single physical NIC — the OS presents it to applications the same as a hardware interface. Common types: VLAN sub-interface (tag traffic with an 802.1Q VLAN ID on one physical port), loopback (for management or BGP peer IDs), tun/tap (used by VPNs to inject packets into user space), and bridge (connects multiple interfaces at layer 2). In cloud/AWS: Virtual Interfaces (VIFs) on Direct Connect are the logical channels over a physical DX connection — private VIF for VPC access, public VIF for AWS public services, transit VIF for Transit Gateway. Strong answers connect the concept to a use case like multi-tenant VLANs or VPN tunnel interfaces.",
      "followup_questions": [],
      "level_guidance": {
        "100": "May confuse with virtual machine or VLAN.",
        "200": "Explains: logical network interface (not tied to physical hardware). AWS context: ENI (Elastic Network Interface) -- virtual NIC attached to instances, can have multiple IPs, security groups, moved between instances. Use cases: dual-homed instances, management networks, failover (move ENI to standby).",
        "300": "Applied design: multi-ENI patterns (separate management/data traffic), ENI for high-availability (move to standby instance), Lambda in VPC (uses ENIs, affects cold start), EFA (Elastic Fabric Adapter) for HPC, trunk interfaces for container networking (awsvpc mode).",
        "400": "Virtual networking architecture: ENI as the fundamental building block of VPC networking (every service creates ENIs), PrivateLink ENIs for service access, hyperplane and the evolution of AWS networking (from ENI-per-function to shared infrastructure), and the performance implications of virtual interfaces (overhead vs bare-metal, hardware offload with Nitro)."
      }
    },
    {
      "domain": "Leadership Principles",
      "question": "Tell me about a time when you had to make a decision with incomplete information. How did you approach it?",
      "notes": "Looking for bias for action, ownership, and decision-making skills.",
      "followup_questions": [
        {
          "question": "What was the outcome and what would you do differently?",
          "notes": "(Parent: LP - decision with incomplete info.) Looking for reflective self-assessment, not just success stories. Strong answers: (1) State the actual outcome clearly — good, bad, mixed. (2) Identify what they'd do differently — probe for specifics (ask more questions of X stakeholder, spend a day instrumenting before deciding, set an explicit checkpoint to revisit the decision, write down the assumptions at the time). (3) Show they've applied the lesson since. Weak signals: 'it worked out great, wouldn't change a thing' (no reflection), or blames external factors for the outcome. Strong for LP rubric: demonstrates Learn & Be Curious + Earn Trust by owning the imperfect parts. Red flag: can't articulate a real lesson or pretends everything was perfect."
        }
      ]
    },
    {
      "domain": "Leadership Principles",
      "question": "Describe a situation where you had to learn a new technology or skill quickly to solve a problem.",
      "notes": "Assessing learn and be curious, ownership, and problem-solving abilities.",
      "followup_questions": [
        {
          "question": "How did you ensure you learned it effectively and what resources did you use?",
          "notes": "(Parent: LP - learn new tech fast.) Listen for learning process, not just 'I read docs'. Good patterns: (1) Goal-directed — 'I needed to ship X, so I learned the minimum to unblock that, then expanded'. (2) Mix of resources — docs for reference, tutorials for structure, a real project to build muscle memory, community (Slack, StackOverflow, AWS re:Post) for unsticking. (3) Deliberate practice — wrote a spike project, broke it intentionally to understand failure modes, paired with someone more experienced. (4) Teaching — wrote up notes, did a lunch-and-learn, mentored a teammate. Teaching cements understanding. (5) Acknowledges knowing when to stop learning and start doing. Red flag: 'I watched a video course' — passive, no validation of understanding."
        }
      ]
    },
    {
      "domain": "Leadership Principles",
      "question": "Tell me about a time when you disagreed with a technical decision made by your team or manager.",
      "notes": "Looking for have backbone; disagree and commit, customer obsession, and communication skills.",
      "followup_questions": [
        {
          "question": "How did you handle the disagreement and what was the final outcome?",
          "notes": "(Parent: LP - disagreed with decision.) Great LP signal. Strong answers: (1) Raised the disagreement early, in private, with data — not in a meeting to embarrass the decision-maker. (2) Articulated the concern specifically: 'I'm worried about X because of Y; the impact would be Z.' (3) Listened to the counterargument genuinely — maybe they were missing context. (4) If the decision went against them: committed fully (Disagree AND Commit), executed the decision, and helped make it succeed. Didn't undermine. (5) Followed up later with data on whether the prediction held. (6) Owned the outcome either way. Red flag: told teammates the decision was wrong behind the manager's back, or resigned in protest, or 'I was right all along' with no empathy for the decision-maker's context."
        }
      ]
    },
    {
      "domain": "Leadership Principles",
      "question": "Describe a project where you had to work with limited resources or tight constraints.",
      "notes": "Assessing frugality, ownership, and deliver results.",
      "followup_questions": [
        {
          "question": "What trade-offs did you make and how did you prioritize?",
          "notes": "(Parent: LP - frugality.) Probes for decision-making under constraints. Good answers walk through a real prioritization framework: (1) Explicitly listed what was cut (not just what was kept). (2) Connected trade-offs to business outcomes — 'we cut feature X because Y customers weren't asking for it' not 'we cut X because we ran out of time'. (3) Used MoSCoW, RICE, ICE, or similar prioritization method, or at least an explicit north-star metric. (4) Got stakeholder sign-off on the cuts, so nobody was surprised. (5) Identified the MINIMUM viable version early and built that first, then iterated. (6) Bonus: found creative solutions (reused existing components, bought vs built, partner vs custom) that reduced effort without reducing value. Red flag: just worked harder / longer hours as the answer — not a trade-off, and doesn't scale."
        }
      ]
    },
    {
      "domain": "Leadership Principles",
      "question": "Tell me about a time when you made a mistake that impacted your team or customers.",
      "notes": "Looking for ownership, earn trust, and learn and be curious.",
      "followup_questions": [
        {
          "question": "How did you handle it and what did you learn from the experience?",
          "notes": "(Parent: LP - mistake with impact.) Listen for Ownership + Earn Trust + Learn & Be Curious. Strong answers: (1) Owned it unambiguously — 'I made this mistake' not 'the team made a mistake' or 'the process failed us'. (2) Communicated broadly and promptly — told affected teams/customers/leadership, didn't try to hide. (3) Fixed immediately — concrete actions, not just apologies. (4) Root-caused properly — a real postmortem (blameless but rigorous), found the class of failure, not just this instance. (5) Preventive changes — added a test, a process, a guardrail that would have caught it. (6) Reflected honestly — what did this teach about their judgment, blind spots, or decision-making pattern? Red flag: 'the tools failed me' / 'the team didn't follow the process' — shifts blame. Or only talks about fixing the symptom, not the class of problem."
        }
      ]
    },
    {
      "domain": "Leadership Principles",
      "question": "Give me an example of a complex problem you solved with a simple solution. What made the problem complex?",
      "notes": "Assessing invent and simplify, problem-solving abilities, and thinking big.",
      "followup_questions": [
        {
          "question": "How did you identify that a simple solution would work?",
          "notes": "(Parent: LP - invent & simplify.) Probes analytical discipline. Good answers: (1) Questioned assumptions — 'people said we needed X, I asked why, and the real requirement was actually Y'. (2) Broke the problem down — stripped away incidentals, isolated the one thing that actually mattered. (3) Looked at analogies — 'we used a similar pattern in another system'. (4) Prototyped cheap — tried the simple thing first, discovered it worked before building the complex solution. (5) Challenged complexity — why 3 services when one would do? Why a new DB when an existing one had capacity? (6) Validated with data, not intuition. Red flag: 'I just thought about it harder' or makes it sound like genius inspiration. Simplicity comes from understanding, not magic."
        }
      ]
    },
    {
      "domain": "Leadership Principles",
      "question": "Describe a time when you took on work outside of your comfort area.",
      "notes": "Looking for learn and be curious, ownership, and growth mindset.",
      "followup_questions": [
        {
          "question": "What did you learn from this experience and how did you approach the learning process?",
          "notes": "(Parent: LP - work outside comfort zone.) Looking for Learn & Be Curious + Ownership. Strong answers: (1) Accepted the challenge without hedging — didn't insist on waiting for perfect prep. (2) Named specific gaps they had going in, not generic 'I didn't know X'. (3) Built a concrete learning plan — specific resources, specific milestones, specific support (mentor, community). (4) Made intentional mistakes safely — prototypes, test envs — instead of learning in prod. (5) Asked for help early rather than struggling in silence. (6) Reflected on what transferred — meta-learning. 'Next time I enter a new domain, I'll X' shows growth. Red flag: 'I figured it out on my own' without acknowledging anyone who helped, or says they mastered it in a week (suggests they didn't actually go deep)."
        }
      ]
    },
    {
      "domain": "AI/ML",
      "question": "Give me a 60-second overview of your AI/ML journey — how did you get into this space and what's shaped your technical perspective?",
      "notes": "Warmup. Listen for breadth (research vs. applied), scale of systems worked on, and customer-outcome thinking.",
      "followup_questions": []
    },
    {
      "domain": "AI/ML",
      "question": "What excites you most about the current state of AI/ML, and what do you think is overhyped?",
      "notes": "Tests whether they have genuine opinions and can think critically. Listen for nuanced view (not just 'GenAI is amazing'), awareness of limitations, practical grounding.",
      "level_guidance": {
        "100": "Repeats headlines ('AI will change everything') without specifics; no clear view on what's overhyped.",
        "200": "Names a concrete trend they're excited about (e.g., RAG, multimodal) and one over-hyped area (e.g., 'AGI is near'), with a basic reason.",
        "300": "Gives a grounded take backed by hands-on experience: cites where a technique works vs where it breaks (cost, latency, hallucination, eval gaps), and distinguishes hype from durable value.",
        "400": "Articulates an industry-shaping perspective with explicit trade-offs — connects technical limits to business/organizational impact, predicts where the field is over-investing, and explains how they'd bet resources differently."
      },
      "followup_questions": []
    },
    {
      "domain": "AI/ML",
      "question": "Walk me through how you'd design an end-to-end ML system for real-time fraud detection. Start from data ingestion through to production monitoring.",
      "notes": "Good answer covers: streaming ingestion (Kinesis/Kafka), feature store (offline + online), model selection (XGBoost/LightGBM for tabular fraud — fast, interpretable), class imbalance handling (SMOTE, class weights, focal loss), <100ms inference, drift monitoring (data drift AND concept drift), feedback loops with confirmed labels. Red flags: only accuracy, ignores latency/scale, can't explain drift.",
      "followup_questions": [
        {
          "question": "How would you handle class imbalance when fraud is <1% of transactions?",
          "notes": "Techniques: class weights in the loss function (cheapest, works for most models), SMOTE/ADASYN (synthetic minority oversampling), random undersampling of the majority class, focal loss (down-weights easy examples), or threshold tuning at inference time. Good candidates mention the tradeoffs: SMOTE can create unrealistic samples, undersampling loses information, class weights are simplest but may not be enough at extreme imbalance. Bonus: mention optimizing for recall/PR-AUC rather than accuracy."
        },
        {
          "question": "What's the difference between data drift and concept drift, and how would you detect each?",
          "notes": "Data drift (covariate shift): input feature distributions change (e.g., a new merchant category appears) but the relationship between features and label is unchanged. Detect with statistical tests (KS test, PSI) comparing live feature distributions to training distributions. Concept drift: the relationship itself changes (e.g., fraudsters invent new patterns), so the same inputs now produce different outputs. Detect via degradation in live model metrics (precision, recall) against labeled feedback. Mitigation: scheduled retraining, online learning, or champion/challenger setups."
        }
      ],
      "level_guidance": {
        "100": "Names some ML concepts (training, prediction) but can't design an end-to-end system.",
        "200": "Outlines basic pipeline: data collection, feature engineering, model training, deployment. Knows real-time vs batch. Mentions fraud-specific considerations (class imbalance, false positive cost).",
        "300": "Designs production system: streaming ingestion (Kinesis), real-time feature computation, model serving with low latency (SageMaker endpoint), feedback loop for retraining, concept drift detection, A/B testing, human-in-the-loop for edge cases. Considers business context (block vs flag vs review).",
        "400": "Designs fraud detection as a product: multi-model ensemble (rules + ML + graph analysis), feature store for consistency, real-time + offline model combination, explains decision latency vs accuracy trade-off, regulatory requirements (explainability), adversarial robustness (fraudsters adapt), and organizational operating model (data science + ops + business)."
      }
    },
    {
      "domain": "AI/ML",
      "question": "Explain the bias-variance tradeoff. How do you diagnose which one is the problem?",
      "notes": "Bias = error from overly simple assumptions (underfitting). Variance = error from sensitivity to training data (overfitting). High bias: train+val errors both high. High variance: train low, val high (big gap). Diagnose with learning curves. Fix bias: more complex model, better features, less regularization. Fix variance: more data, regularization, simpler model, ensembles.",
      "followup_questions": [
        {
          "question": "Walk me through a learning curve — what does it look like for each case?",
          "notes": "A learning curve plots model error (y-axis) against training set size (x-axis), with separate lines for training error and validation error. High bias: both lines converge at high error and plateau — adding more data doesn't help. High variance: training error stays low, validation error stays high, with a large persistent gap — adding more data would help close the gap. Ideal: both lines converge at low error. Strong candidates will mention that learning curves are how you decide whether to collect more data vs. use a more complex model."
        }
      ],
      "level_guidance": {
        "100": "Vague understanding -- 'something about overfitting and underfitting'.",
        "200": "Clearly explains: high bias = underfitting (model too simple), high variance = overfitting (model too complex). Diagnoses via train/test performance gap. Knows regularization helps variance.",
        "300": "Systematic diagnosis: learning curves (train vs validation error as data grows), cross-validation, feature importance analysis. Fixes: for high bias -- more features, more complex model, less regularization; for high variance -- more data, regularization (L1/L2/dropout), feature selection, ensemble methods.",
        "400": "Nuanced understanding: bias-variance decomposition mathematically, how modern deep learning somewhat breaks the traditional tradeoff (double descent), practical implications for model selection, and when to accept higher bias for deployment simplicity (smaller model, faster inference, easier to explain)."
      }
    },
    {
      "domain": "AI/ML",
      "question": "What's the difference between supervised, unsupervised, and self-supervised learning? Give a real-world use case for each.",
      "notes": "Supervised: labeled data (spam classification, medical imaging). Unsupervised: no labels, finds structure (customer segmentation, anomaly detection). Self-supervised: creates labels from data structure (LLM pre-training, BERT masked LM, SimCLR). Key insight: self-supervised is what made foundation models possible.",
      "followup_questions": [],
      "level_guidance": {
        "100": "Names the three categories. Says supervised has labels, unsupervised doesn't, self-supervised generates labels from the data itself. May not give concrete examples.",
        "200": "Pairs each with a real algorithm — supervised: linear regression / random forest; unsupervised: k-means, PCA; self-supervised: masked language modelling, contrastive learning. Gives use-cases for each.",
        "300": "Picks between them on actual problems: 'this is a labelled classification problem so supervised is the default; if labels are scarce I'd consider self-supervised pretraining first then fine-tune.' Discusses cost of labelling, data leakage, train/val/test discipline.",
        "400": "Reasons about the tradeoffs at scale — when you'd combine paradigms (e.g. self-supervised pretraining + supervised fine-tuning + RLHF), failure modes in production (distribution shift, label noise), and the engineering cost of each beyond the algorithm choice."
      }
    },
    {
      "domain": "AI/ML",
      "question": "Explain gradient descent. What are the variants and when do you use each?",
      "notes": "Core: iteratively adjust params opposite to gradient to reduce loss. Batch GD: entire dataset per update (stable, slow). SGD: one sample (noisy, fast, escapes local minima). Mini-batch: 32-512 samples (standard). Optimizers: Adam (most common default), AdaGrad (sparse data), RMSprop. Learning rate critical — too high diverges, too low slow. Schedulers: cosine annealing, warmup.",
      "followup_questions": [
        {
          "question": "Why is Adam typically the default optimizer? What makes it adaptive?",
          "notes": "Adam = Adaptive Moment Estimation. It combines two ideas: (1) momentum — uses a moving average of past gradients to accelerate in consistent directions, (2) per-parameter adaptive learning rates — scales each parameter's update by a moving average of squared gradients, so parameters with frequently large gradients get smaller updates. Result: works well out-of-the-box across tasks, less sensitive to learning rate choice, handles sparse gradients. Trade-offs: can converge to sharper minima that generalize slightly worse than SGD+momentum, which is why some large-scale training uses SGD with warmup."
        }
      ],
      "level_guidance": {
        "100": "Knows it's about 'finding the minimum' of a function.",
        "200": "Explains: iteratively update weights in direction of negative gradient, learning rate controls step size. Variants: batch GD (all data), SGD (one sample), mini-batch (compromise). Knows learning rate matters.",
        "300": "Deep understanding: momentum, Adam/AdamW optimizers, learning rate schedules (warmup, cosine decay), gradient clipping, challenges (saddle points, local minima in non-convex landscapes). Practical choices for different problems.",
        "400": "Nuanced: second-order methods vs first-order trade-offs, distributed gradient computation (all-reduce, gradient compression), mixed precision training implications for gradient stability, and the empirical observations about loss landscapes in large models (mode connectivity, linear interpolation)."
      }
    },
    {
      "domain": "AI/ML",
      "question": "What metrics would you use to evaluate a classification model? When is accuracy misleading?",
      "notes": "Accuracy misleading with imbalanced classes (99% non-fraud baseline). Precision: of predicted positives, how many correct (important when FPs costly). Recall: of actual positives, how many caught (important when FNs costly). F1: harmonic mean. AUC-ROC: discrimination across thresholds. AUC-PR: better for imbalanced. Always connect to business metrics.",
      "followup_questions": [
        {
          "question": "In a fraud detection system, would you optimize for precision or recall? Why?",
          "notes": "It depends on the cost ratio. Optimize for RECALL when missing fraud is very expensive (losses from undetected fraud, reputation damage) — you'd rather investigate more false alarms than let fraud through. Optimize for PRECISION when false positives are costly (customer friction, blocked legitimate transactions, support burden). In practice, most fraud systems target a PRECISION-RECALL trade-off operating point — e.g., 'we want 95% recall at the highest precision possible' — and tune the classification threshold accordingly. Strong answers mention: the business economics drive the choice, not a default technical preference."
        }
      ],
      "level_guidance": {
        "100": "Knows accuracy as a metric. May not understand its limitations.",
        "200": "Names: precision, recall, F1, AUC-ROC, confusion matrix. Knows accuracy fails with class imbalance (99% accurate by predicting majority class). Chooses metrics based on business cost (precision for spam, recall for cancer detection).",
        "300": "Designs evaluation framework: business-aligned metrics (cost matrix mapping FP/FN to dollars), calibration (predicted probabilities match actual frequencies), threshold selection based on business trade-offs, stratified evaluation across segments, statistical significance testing for model comparisons.",
        "400": "Evaluation as a system: online vs offline metric divergence, counterfactual evaluation for recommendation systems, long-term impact metrics (user lifetime value, not just click-through), fairness metrics across subgroups, and the organizational challenge of aligning data science metrics with business KPIs."
      }
    },
    {
      "domain": "AI/ML",
      "question": "Explain overfitting. What are 5 techniques to prevent it?",
      "notes": "Overfitting: model memorizes training noise. Prevention: (1) more training data, (2) regularization (L1=Lasso/feature selection, L2=Ridge/shrinks weights), (3) dropout (randomly zero neurons), (4) early stopping (stop when val loss rises), (5) cross-validation (k-fold), (6) data augmentation, (7) ensembles (bagging reduces variance).",
      "followup_questions": [],
      "level_guidance": {
        "100": "Knows overfitting = memorising training data. Suggests adding more data.",
        "200": "Explains training vs validation accuracy gap; uses cross-validation or hold-out set; mentions regularisation (L1/L2, dropout) and early stopping.",
        "300": "Names 5+ techniques deliberately: data augmentation, dropout, weight decay, early stopping, ensembling, simpler model, label smoothing — and discusses when each helps. Talks about learning curves, bias-variance tradeoff.",
        "400": "Reasons about overfitting at scale: distribution shift detection in production, calibration drift, label noise vs genuine overfit, when more data does NOT help (irreducible error / Bayes optimal), and how to think about it for foundation-model fine-tuning where the base model is already overfit to its pretraining set."
      }
    },
    {
      "domain": "AI/ML",
      "question": "A customer is migrating their ML platform from on-premises GPUs to AWS. How would you architect it?",
      "notes": "Discovery first: workloads, team size, governance, current tools. Compute: SageMaker Training Jobs or EC2 P5/P4d; inference on SageMaker Endpoints, Batch Transform, or Inf2 (40% cheaper). Cost: Managed Spot Training (up to 90% savings), autoscaling to zero, Savings Plans, right-sizing. Data: S3 lake, FSx Lustre, Feature Store. MLOps: SageMaker Pipelines, Model Registry, Model Monitor. Governance: IAM, VPC, CloudTrail, encryption. Migration: lift-and-shift on containers first, then adopt managed services incrementally.",
      "followup_questions": [
        {
          "question": "How would you handle cost optimization for a training-heavy workload that runs nightly?",
          "notes": "Key levers: (1) Managed Spot Training on SageMaker (up to 90% savings; add checkpointing so interruptions don't lose progress), (2) right-size instances — benchmark on a small instance before committing to p5.48xlarge, (3) use Trainium (Trn1/Trn2) for supported frameworks (40-50% cheaper than equivalent GPU), (4) distributed training if a single-GPU job takes too long (more GPUs = shorter wall time = cheaper at hourly rates), (5) data loading optimization (FSx for Lustre or pre-cached S3 reads) so GPUs aren't idle waiting on I/O, (6) Savings Plans or Reserved Instances for baseline steady-state training. Also: incremental training where feasible instead of full retrains."
        }
      ],
      "level_guidance": {
        "100": "Suggests EC2 with GPUs. Limited architecture.",
        "200": "Knows: P4d/P5 instances for training, inf2 for inference, EBS/FSx for data storage, SageMaker for managed training. Basic cost comparison with on-prem.",
        "300": "Designs migration plan: assess workload requirements (GPU memory, interconnect needs), select appropriate instances (Trn1 for cost-effective training, P5 for largest models, Inf2 for inference), data migration strategy (parallel upload to S3, FSx Lustre for high-throughput training), MLOps pipeline migration (SageMaker Pipelines), cost optimization (Spot for training, right-sizing inference).",
        "400": "Reasons about ML platform architecture: multi-tenancy and cluster scheduling, distributed training at scale (EFA networking, custom collectives), hybrid training strategies (burst to cloud for peak), organizational ML platform team operating model, and total cost of ownership analysis (not just compute but engineering time, reliability, iteration speed)."
      }
    },
    {
      "domain": "AI/ML",
      "question": "Explain the difference between real-time inference, batch inference, and asynchronous inference on AWS. When would you use each?",
      "notes": "Real-time (<100ms-seconds): SageMaker Real-time Endpoints, user-facing apps. Batch (minutes-hours): SageMaker Batch Transform, nightly scoring, no persistent infra. Async (seconds-minutes): SageMaker Async Inference, large payloads (video/audio/docs), scales to zero between requests. Match latency/cost/payload-size to pattern.",
      "followup_questions": [],
      "level_guidance": {
        "100": "Knows models can make predictions. May not distinguish deployment patterns.",
        "200": "Real-time: synchronous endpoint, low latency, always-on. Batch: process large datasets on schedule, higher throughput, cost-effective. Async: queue-based, handles variable latency, good for large inputs. Knows SageMaker supports all three.",
        "300": "Designs inference architecture: auto-scaling policies for real-time (based on invocations, latency, or custom metrics), multi-model endpoints for cost sharing, batch transform for offline scoring, async inference with SNS notification for large payloads. Considers model compilation (Neo) and hardware selection (Inf2 for cost, GPU for flexibility).",
        "400": "Reasons about inference at scale: model serving optimization (batching, dynamic batching, model sharding), cost per prediction economics, edge inference for latency-critical paths, inference graph optimization (ONNX, TensorRT), and the system design trade-offs between freshness (real-time) and cost (batch) at organizational scale."
      }
    },
    {
      "domain": "AI/ML",
      "question": "A customer wants sub-50ms P99 latency at 10,000 req/sec for their ML model. Design the inference architecture.",
      "notes": "Model optimization: quantization (FP32→FP16/INT8), compilation (SageMaker Neo/TensorRT), distillation. Infra: Inf2 (best price/perf) or G5 with TensorRT, multi-AZ. Scaling: pre-warm endpoints (min instance count), target-tracking autoscaling on InvocationsPerInstance, provision headroom. Architecture: caching for repeated inputs, ALB/API Gateway. Monitoring: track P50/P95/P99 separately, alarms on P99, load test before launch.",
      "followup_questions": [
        {
          "question": "What are the tradeoffs of INT8 quantization vs keeping FP16?",
          "notes": "FP16 (half-precision float): ~2x smaller than FP32 with minimal accuracy loss — almost always safe. INT8: another ~2x smaller, 2-4x faster inference, but can lose 1-3% accuracy on sensitive tasks without calibration. Tradeoffs: (1) accuracy — INT8 needs a calibration dataset to set per-layer scale/zero-point correctly, (2) hardware support — Inf2 and H100 have dedicated INT8 units, older hardware may actually be slower, (3) outliers — some transformer layers (attention scores, activations after softmax) have outlier values that break naive INT8, needing smoothquant or per-channel quantization, (4) dev effort — FP16 is nearly free, INT8 needs validation. Rule of thumb: start with FP16, move to INT8 only if latency/cost demand it, validate accuracy on a held-out test set."
        }
      ],
      "level_guidance": {
        "100": "Suggests 'use a big GPU'. No system design.",
        "200": "Knows: model optimization (quantization, pruning), right-sized instances, auto-scaling, load balancing. Can estimate rough capacity needs.",
        "300": "Designs for P99 latency: model compilation (SageMaker Neo, TensorRT), appropriate hardware (Inf2 for transformer models), connection pooling, multi-AZ deployment, pre-warming, horizontal scaling with target tracking on latency metric, model caching, input validation to reject early.",
        "400": "Systems-level design: tail latency analysis (hedged requests, backup inference paths), model partitioning across accelerators, custom serving framework vs managed (trade-offs), capacity planning with traffic modeling, graceful degradation (simpler fallback model under load), and the cost engineering of achieving P99 targets (diminishing returns of optimizing past a point)."
      }
    },
    {
      "domain": "AI/ML",
      "question": "What is Amazon SageMaker Feature Store and why does it matter?",
      "notes": "Centralized repo for ML features. Online store: low-latency reads (single-digit ms) for real-time inference. Offline store: S3-based, for training and batch scoring. Why it matters: (1) consistency — same feature computation for training AND inference (avoids train-serve skew), (2) reusability across teams, (3) point-in-time correctness (no data leakage), (4) governance and lineage tracking.",
      "followup_questions": [],
      "level_guidance": {
        "100": "May not know what a Feature Store is.",
        "200": "Explains: centralized repository for ML features, ensures consistency between training and inference (train-serve skew prevention), supports online (low-latency lookup) and offline (batch training) access patterns.",
        "300": "Designs feature management: feature engineering pipelines feeding the store, point-in-time correctness for training (prevents data leakage), feature sharing across teams, feature monitoring for drift, integration with SageMaker Pipelines for automated retraining.",
        "400": "Reasons about feature platforms: organizational feature discovery and reuse, feature freshness requirements driving architecture, cost of maintaining real-time features vs accuracy gain, governance (who owns features, versioning, deprecation), and how feature stores enable faster ML iteration cycles at org scale."
      }
    },
    {
      "domain": "AI/ML",
      "question": "Compare Trainium (Trn1/Trn2) vs. NVIDIA GPUs (P5/P4d) on AWS. When would you recommend each?",
      "notes": "Trainium: AWS custom silicon, up to 50% better price-performance for supported workloads, best for transformers/LLM pre-training. Trade-off: narrower framework support via Neuron SDK, may require porting. Trn2 (2025+): 4x Trn1. NVIDIA P5/P4d: broadest ecosystem (CUDA, all frameworks), best for custom architectures/research. Decision: standard transformer training → Trainium; custom CUDA kernels/research → NVIDIA; inference → Inferentia2 (Inf2) best value. Team capability matters.",
      "followup_questions": [],
      "level_guidance": {
        "100": "Knows GPUs are used for ML. May not know Trainium.",
        "200": "Basic comparison: Trainium = AWS custom silicon, better price/performance for training, requires framework adaptation (Neuron SDK). NVIDIA = universal compatibility, larger ecosystem, any framework works. Recommends NVIDIA for flexibility, Trainium for cost at scale.",
        "300": "Nuanced recommendation: Trainium for large-scale training workloads with supported architectures (transformers, LLMs), where cost savings of 40-50% justify Neuron SDK adaptation. NVIDIA P5 for research/experimentation, custom architectures, or when time-to-market > cost. Considers: EFA networking, cluster topology, Neuron compiler optimization.",
        "400": "Strategic reasoning: custom silicon trajectory (Trainium2 roadmap), vendor lock-in vs cost analysis, workload portability strategy, the organizational investment to adopt new hardware (team training, pipeline adaptation, debugging tools), and how to make the build-vs-buy decision for ML infrastructure."
      }
    },
    {
      "domain": "AI/ML",
      "question": "I'm a VP of Engineering at a large retailer. My CEO wants us to 'use AI everywhere.' We have a data warehouse, some dashboards, and 3 data scientists. Where do we start?",
      "notes": "Roleplay. Good answer: asks clarifying questions first (pain points, data, budget). Doesn't jump to GenAI. Crawl-walk-run: (Crawl) managed services — Personalize, Forecast, Bedrock+Knowledge Bases chatbot. (Walk) build internal ML capability, establish MLOps, tackle one high-value problem. (Run) custom models, GenAI products. Addresses data quality, governance, skills gap. Connects to business outcomes — 'AI everywhere' isn't a strategy, find 2-3 use cases with ROI. Realistic timeline. Red flags: jumps to foundation models, ignores data readiness, doesn't ask questions.",
      "followup_questions": [
        {
          "question": "What would you do if they push back and want a GenAI chatbot in 30 days?",
          "notes": "Don't say no — scope it. Propose a tightly-scoped MVP: pick ONE internal document corpus (e.g., HR policies or product FAQ), build a RAG chatbot on Bedrock + Knowledge Bases, deploy to a single internal team for feedback. That's realistically doable in 30 days with Bedrock's managed RAG. What NOT to promise in 30 days: company-wide rollout, custom fine-tuning, multi-agent workflows, replacing existing systems. Strong candidates set clear success criteria upfront (e.g., '70% of questions answered without escalation') and explicitly call out what's out-of-scope. They also mention governance requirements that can't be compressed (data classification review, security approvals)."
        }
      ],
      "level_guidance": {
        "100": "Suggests specific ML use cases without strategic framing.",
        "200": "Structured approach: assess data maturity, identify high-impact use cases, start with quick wins (forecasting, personalization), build data platform foundation. Mentions change management.",
        "300": "Strategic advisory: help prioritize use cases by business impact x feasibility matrix, identify data gaps, recommend organizational structure (central ML team vs embedded), define success metrics before building, propose phased roadmap (3-6-12 months), quick wins to build confidence while investing in platform.",
        "400": "Executive-level advisory: AI strategy as a business strategy (not a technology initiative), data as strategic asset (investment in data quality/governance), organizational design for AI (CoE vs federated), talent strategy, ethical AI framework, measurement framework (ROI of AI portfolio), and the common failure modes of 'AI everywhere' mandates (spreading too thin, no clear ownership, missing data foundations)."
      }
    },
    {
      "domain": "AI/ML",
      "question": "We tried ML last year — hired a team, they built some models, but nothing made it to production. What went wrong and how would you fix it?",
      "notes": "Roleplay. Common failures: no MLOps/deployment infra (stuck in notebooks), misalignment with business, data quality discovered late, no success metrics defined upfront, 'last mile' integration problem. Fix: start with business problem backward (metrics first), MLOps from day 1 (CI/CD, registry, testing), small end-to-end wins first, embedded ML engineers (not isolated lab), regular stakeholder demos. AWS: SageMaker Pipelines, Model Monitor.",
      "followup_questions": [],
      "level_guidance": {
        "100": "Suggests trying again with better data.",
        "200": "Identifies common failure reasons: no clear business problem, research-to-production gap, model without MLOps, no feedback loop. Suggests starting with the business problem.",
        "300": "Diagnoses the pattern: probably built models in notebooks without production engineering (monitoring, data pipelines, CI/CD for models), no clear success metrics, no product owner driving requirements. Recommends: audit what was built, identify closest-to-production model, add MLOps layer (SageMaker Pipelines), assign product ownership, measure business metrics not just model metrics.",
        "400": "Organizational diagnosis: ML teams fail when they optimize for model accuracy instead of business value, lack integration with engineering teams (handoff gap), have no feedback mechanism from production to research, and face organizational misalignment (DS team incentivized on papers/models, business needs outcomes). Prescribes: ML platform investment, product-minded ML engineers, clear OKRs tied to revenue/cost, and the 'boring ML' principle (deploy simple models that work before complex ones)."
      }
    },
    {
      "domain": "AI/ML",
      "question": "We're worried about data security with AI. Our data is highly regulated (healthcare/finance). Convince me it's safe on AWS.",
      "notes": "Data isolation: Bedrock doesn't train base models on your data, data stays in region. SageMaker: VPC isolation, private subnets. Customer-managed KMS keys. Compliance: HIPAA, SOC 2, FedRAMP, PCI-DSS; Bedrock HIPAA-eligible; BAA available. Access: IAM least privilege, CloudTrail audit, PrivateLink (no public internet). Governance: Bedrock Guardrails (content filter, PII redaction), invocation logging, SageMaker Model Cards. Key: 'AWS gives MORE control than on-prem — encryption, logging, access are built in, not bolted on.'",
      "followup_questions": [],
      "level_guidance": {
        "100": "Suggests encryption. Limited depth on AI-specific security.",
        "200": "Covers: data encryption at rest/transit, VPC isolation for training, IAM controls, data residency. Knows SageMaker runs in customer VPC. Mentions HIPAA/SOC2.",
        "300": "Designs secure ML architecture: VPC-isolated training with no internet access, KMS for model artifact encryption, data access auditing, model output filtering (PII detection), differential privacy for training, secure inference endpoints (PrivateLink), compliance mapping (HIPAA eligible services, BAA).",
        "400": "Comprehensive security strategy for regulated AI: data governance framework (lineage, classification, minimization), model security (adversarial robustness, model extraction prevention, prompt injection for GenAI), responsible AI governance (bias testing, explainability for regulated decisions), and the emerging regulatory landscape (EU AI Act, FDA guidance for health AI) that shapes architectural decisions."
      }
    },
    {
      "domain": "AI/ML",
      "question": "Explain distributed training. How does data parallelism vs. model parallelism work?",
      "notes": "Data parallelism: same model on multiple GPUs, each processes different batches, gradients averaged. Standard when model fits on one GPU. Model parallelism: model split across GPUs (too large for one). Types: pipeline parallelism (layers split), tensor parallelism (individual layers split). Use model parallelism for LLMs with billions of params. AWS: SageMaker distributed training libraries handle this.",
      "followup_questions": [],
      "level_guidance": {
        "100": "Knows training can use multiple GPUs. Can't explain how.",
        "200": "Data parallelism: split data across GPUs, each has full model copy, sync gradients. Model parallelism: split model layers across GPUs (for models too large for one GPU). Knows communication overhead is the bottleneck.",
        "300": "Detailed understanding: data parallel with all-reduce (ring, tree), pipeline parallelism (micro-batching across layers), tensor parallelism (split individual layers), ZeRO optimizer stages (memory optimization). AWS implementation: SageMaker distributed training, EFA for low-latency communication, instance topology awareness.",
        "400": "Designs distributed training at scale: 3D parallelism (data + pipeline + tensor), expert parallelism for MoE models, communication-computation overlap, gradient accumulation trade-offs, fault tolerance (checkpointing strategy, preemption handling with Spot), and the systems engineering of training clusters (network topology, failure domains, scheduling)."
      }
    },
    {
      "domain": "AI/ML",
      "question": "How would you implement A/B testing for ML models in production?",
      "notes": "Traffic splitting via SageMaker Production Variants — route X% to Model A, Y% to Model B. Define primary metric before starting (CTR, conversion). Statistical rigor: power analysis for sample size, run until significant. Shadow mode: new model gets traffic but predictions not served — compare offline. Canary deployment: start 5%, monitor errors/latency, gradually increase. Automated rollback if thresholds breached.",
      "followup_questions": [],
      "level_guidance": {
        "100": "Knows A/B testing means comparing two versions.",
        "200": "Basic approach: split traffic between models, measure business metric, ensure statistical significance. Knows shadow deployment (run both, compare offline) as a safer alternative.",
        "300": "Designs ML experimentation platform: traffic splitting at inference layer (SageMaker production variants), guardrail metrics (latency, error rate) separate from decision metrics (conversion, revenue), multi-armed bandit for faster convergence, staged rollout (1% -> 10% -> 50% -> 100%), automated rollback triggers.",
        "400": "Experimentation as a system: counterfactual evaluation methods, network effects that invalidate naive A/B tests, long-term metric measurement (user LTV vs short-term engagement), experiment interaction effects (multiple simultaneous experiments), and the organizational culture of experimentation (hypothesis-driven, accepting negative results)."
      }
    },
    {
      "domain": "AI/ML",
      "question": "What is training-serving skew and how do you prevent it?",
      "notes": "Differences between how features are computed at training vs. inference — model trained on one reality, serves on another. Causes: different code paths (batch vs real-time), data processing bugs, feature drift. Prevention: Feature Store as single source of truth, SAME code for train and serve (shared pipeline), data validation (Great Expectations, Deequ, SageMaker Data Quality), monitoring (compare prod vs train feature distributions).",
      "followup_questions": [],
      "level_guidance": {
        "100": "Has heard the phrase. May say 'when training and serving don't match'.",
        "200": "Defines it: features computed differently in training vs inference (different libs, data freshness, missing-value handling). Suggests using the same code path for both.",
        "300": "Has shipped a feature store (SageMaker Feature Store, Feast, Tecton) so offline + online features are computed by the same code. Knows about point-in-time correct joins, time-travel for backfills, monitoring feature distributions.",
        "400": "Architects the whole platform: streaming feature computation with the same code as the offline pipeline (Spark Structured Streaming, Flink), schema enforcement across both sides, automated feature drift alarms, contract tests in CI between training and serving paths, and the deeper class of skew (label leakage, feedback loops)."
      }
    },
    {
      "domain": "GenAI",
      "question": "A customer wants to build a domain-specific AI assistant using their internal documents. Walk me through the architecture options.",
      "notes": "Option 1 RAG (start here): ingest docs, chunk (500-1000 tokens with overlap), embed (Titan/Cohere), vector store (OpenSearch Serverless, Aurora pgvector, Kendra), retrieve top-k with hybrid search + reranking, pass to LLM (Claude, Nova). Option 2 Fine-tuning: when RAG alone isn't enough — specific format/tone/reasoning. Bedrock custom models or SageMaker JumpStart. Hundreds-thousands of examples. Option 3 Continued pre-training: deep domain knowledge (proprietary terms, specialized reasoning). Option 4 Agentic (Bedrock Agents): multi-step reasoning, tool use, DB queries. Best practice: start with RAG, add fine-tuning if needed, go agentic for complex workflows.",
      "followup_questions": [
        {
          "question": "How would you choose between RAG and fine-tuning for a customer support use case?",
          "notes": "Start with RAG almost always: customer support content changes frequently (product updates, policy changes), and RAG lets you swap the knowledge base without retraining. Fine-tuning is better when you need (a) specific tone/style (e.g., brand voice), (b) a specific output format (structured JSON every time), or (c) domain-specific reasoning patterns not in the base model. Decision framework: if the problem is 'the model doesn't know X', use RAG. If the problem is 'the model doesn't say things the right way', fine-tune. Practical answer: do RAG first, evaluate, then add fine-tuning only for the gaps RAG can't close."
        },
        {
          "question": "When would you combine RAG with fine-tuning?",
          "notes": "When you need BOTH current factual knowledge (RAG) AND domain-specific output style or reasoning (fine-tuning). Examples: (1) medical assistant that needs to cite current clinical guidelines (RAG) but also phrase responses in clinically-appropriate language (fine-tune), (2) legal research tool that pulls current case law (RAG) but formats answers as IRAC briefs (fine-tune), (3) customer-support agent that retrieves current policies (RAG) but follows a specific de-escalation script (fine-tune). Order: fine-tune first on stable stylistic patterns, then deploy with RAG for the volatile knowledge layer."
        }
      ],
      "level_guidance": {
        "100": "Suggests 'fine-tune a model on their documents'. Limited architecture.",
        "200": "Knows RAG pattern: embed documents, store in vector DB, retrieve relevant chunks at query time, feed to LLM for answer generation. Can name components (Bedrock, OpenSearch, Kendra).",
        "300": "Designs RAG architecture: chunking strategy (size, overlap, semantic boundaries), embedding model selection, vector store choice (OpenSearch Serverless vs Pinecone vs pgvector), retrieval strategy (hybrid search: keyword + semantic), prompt engineering for grounded answers, citation/source attribution, evaluation pipeline (relevance, faithfulness, answer quality).",
        "400": "Production RAG system design: advanced retrieval (re-ranking, query decomposition, multi-hop reasoning), document processing pipeline (parsing complex formats, table extraction, metadata enrichment), guardrails (Bedrock Guardrails for content filtering), observability (trace retrieval quality, detect hallucination drift), cost optimization (caching frequent queries, tiered retrieval), and organizational considerations (data freshness SLA, access control on documents, multi-tenant isolation)."
      }
    },
    {
      "domain": "GenAI",
      "question": "How do you handle hallucination in a RAG system?",
      "notes": "(1) Better retrieval: semantic chunking, hybrid search (keyword + semantic), cross-encoder reranker. (2) Prompt engineering: 'Only answer from context. If not in context, say I don't know.' Require citations. (3) Guardrails: Bedrock Guardrails (filter responses, block topics). (4) Grounding checks: automated fact-checking vs sources, confidence scoring, human review for low-confidence. (5) Architecture: chain-of-thought prompting, multi-step verification (generate then verify), smaller focused KBs per domain.",
      "followup_questions": [
        {
          "question": "What's the difference between keyword search, semantic search, and hybrid search for retrieval?",
          "notes": "Keyword search (BM25, TF-IDF): matches exact words — fast, precise when queries use the same terminology as documents, but misses synonyms and paraphrases. Semantic search (embeddings + cosine similarity): finds conceptually-similar content even with different words — great for natural language queries, but can retrieve off-topic results when the embedding space doesn't separate fine-grained meanings well. Hybrid search: combines both (e.g., BM25 score + semantic score, often via reciprocal rank fusion) — best of both worlds and usually the production default. Strong answers mention that hybrid catches BOTH exact term matches (names, SKUs, error codes) AND paraphrased questions, and that reranking (with a cross-encoder) on top of hybrid retrieval is the gold standard."
        }
      ],
      "level_guidance": {
        "100": "Knows hallucination = LLM making things up. Says 'use RAG' or 'use a better model'.",
        "200": "Names mitigation tactics: temperature=0 for deterministic tasks, RAG with citations, structured output (JSON schema), self-consistency / multiple samples + vote.",
        "300": "Engineering-grade: separate retrieval-vs-generation evals, grounding score, automated factuality checks (FActScore / FActC), cite-then-quote prompts, abstain prompts ('say I don't know'), human-in-the-loop on low-confidence outputs, model selection by hallucination rate on the specific domain.",
        "400": "Treats hallucination as a calibration problem: confidence elicitation, ensemble disagreement as an uncertainty signal, fine-tuning for refusal behaviour, formal verification on critical outputs, monitoring for hallucination drift in production, and the deeper insight that some 'hallucinations' are spec ambiguity not model failure."
      }
    },
    {
      "domain": "GenAI",
      "question": "Compare Amazon Bedrock vs. SageMaker AI for GenAI workloads. When do you recommend each?",
      "notes": "Bedrock: serverless, fully managed, multi-provider marketplace (Claude, Llama, Nova, Cohere, Mistral), pay-per-token, built-in Guardrails/Knowledge Bases/Agents. Best for: fast-to-production, app devs, standard GenAI patterns. SageMaker AI: infrastructure control, any model (BYO, HuggingFace, custom), full training control, dedicated endpoints, MLOps features (Pipelines, Feature Store, Model Monitor). Best for: ML engineers, custom models, max control. Chatbot fast → Bedrock. Fine-tune Llama 3 with custom loops → SageMaker. Need guardrails/RAG out of box → Bedrock. 20 custom models with A/B → SageMaker. Many use both.",
      "followup_questions": [],
      "level_guidance": {
        "100": "Knows both can run models. Can't clearly distinguish when to use which.",
        "200": "Bedrock: fully managed, API access to multiple FMs (Claude, Nova, Llama), no infrastructure management, pay-per-token. SageMaker: bring your own model, full control over training/hosting, custom containers, optimize performance. Bedrock for quick start, SageMaker for customization.",
        "300": "Nuanced guidance: Bedrock when -- using supported models, RAG with Knowledge Bases, need Guardrails, want fastest time-to-value, consumption pricing works. SageMaker when -- custom model training, specific hardware needs (Trainium/Inf2), need model optimization (quantization, compilation), multi-model endpoints for cost, complex inference pipelines. Hybrid: train on SageMaker, deploy custom model to Bedrock (custom model import).",
        "400": "Strategic reasoning: total cost modeling (Bedrock token pricing vs SageMaker self-hosted at scale crossover point), vendor flexibility (Bedrock locks to supported models, SageMaker is model-agnostic), organizational capability requirements (SageMaker needs ML engineering team, Bedrock needs prompt engineers), and the evolution path (start Bedrock, graduate to SageMaker as expertise grows and scale demands it)."
      }
    },
    {
      "domain": "GenAI",
      "question": "Explain how transformers work at a high level. What makes them better than RNNs for language?",
      "notes": "Core: self-attention — each token attends to every other token simultaneously, computes relevance scores across all pairs. No sequential processing → massive parallelization. Architecture: encoder+decoder, or decoder-only (GPT-style). Key components: multi-head attention, positional encoding, feed-forward layers, layer norm, residual connections. Better than RNNs: (1) parallelizable (GPU-friendly), (2) long-range dependencies (RNNs forget due to vanishing gradients; attention sees everything equally), (3) scales better with data/compute. Trade-off: attention is O(n²) in sequence length — expensive for very long sequences (hence context window limits).",
      "followup_questions": [
        {
          "question": "Why is attention O(n²) and what are some approaches to reduce this?",
          "notes": "Attention computes a score between every pair of tokens — that's n × n comparisons for a sequence of length n, so O(n²) in both compute and memory. Approaches to reduce: (1) Sparse attention (Longformer, BigBird): each token attends to a fixed local window + a few global tokens, reducing to O(n), (2) FlashAttention: doesn't change the asymptotic cost but dramatically cuts memory by computing attention in tiles that fit in SRAM — the current production default, (3) Linear attention (Performer, Linformer): approximates attention via low-rank or kernel methods, O(n), with some accuracy loss, (4) State-space models (Mamba): replace attention entirely with a recurrent-like mechanism, O(n), gaining attention over long sequences. Strong answers mention that FlashAttention is what made 100K+ context windows practical."
        }
      ],
      "level_guidance": {
        "100": "Knows transformers are used in LLMs. Can't explain mechanism.",
        "200": "High-level: self-attention allows model to weigh relationships between all tokens in parallel (unlike RNN sequential processing). Encoder-decoder architecture. Attention = 'which parts of input matter for this output.' Parallelizable training (unlike RNNs).",
        "300": "Detailed understanding: multi-head attention, positional encoding (since no sequential processing), key/query/value mechanism, layer normalization, feed-forward layers. Knows architectural variants: encoder-only (BERT, embeddings), decoder-only (GPT, generation), encoder-decoder (T5, translation). Understands why transformers scale better (parallel training, scaling laws).",
        "400": "Deep understanding: quadratic attention complexity (O(n^2)) and mitigations (sparse attention, linear attention, sliding window), KV-cache for inference efficiency, scaling laws (Chinchilla), architectural innovations (MoE, GQA, RoPE embeddings), hardware implications (memory bandwidth bound at inference, compute bound at training), and the research frontier (state-space models as alternatives, long-context approaches)."
      }
    },
    {
      "domain": "GenAI",
      "question": "What are embeddings? How do they work and why are they important for AI systems?",
      "notes": "Dense vector representations of data (text, images) in continuous high-dim space (768-1536 dims typical). Key property: similar things are close in vector space. Text: trained on large corpora to predict context (Word2Vec, BERT, sentence transformers). Images: CNN/ViT features. Why important: semantic search (meaning not keywords), RAG retrieval, clustering, downstream model inputs. AWS: Amazon Titan Embeddings (text + multimodal), Cohere Embed on Bedrock.",
      "followup_questions": [
        {
          "question": "How would you evaluate the quality of an embedding model for a specific domain?",
          "notes": "Build a domain-specific evaluation set: (1) collect real queries from the domain (support tickets, search logs), (2) for each query, have SMEs label 5-10 known-relevant documents from the corpus. Then measure retrieval quality with standard IR metrics: Recall@K (did we retrieve the relevant docs in the top K?), Precision@K, MRR (mean reciprocal rank), and NDCG. Compare your candidate embedding models (Titan, Cohere, all-MiniLM, domain-tuned BERT) on the same eval set. Also consider: latency (batch encoding speed), cost (per-token pricing or self-hosted infra), dimensionality (lower dims = cheaper vector storage), and multilingual support if needed. Bonus: test embedding quality on edge cases — jargon, abbreviations, numerical/code content."
        }
      ],
      "level_guidance": {
        "100": "Knows embeddings represent things as numbers. Vague on why.",
        "200": "Explains: dense vector representations that capture semantic meaning in N-dimensional space. Similar concepts have similar vectors (cosine similarity). Used for: search, recommendation, clustering, RAG retrieval. Knows Word2Vec/sentence transformers at a high level.",
        "300": "Applied understanding: embedding model selection (Titan Embeddings, Cohere, OpenAI -- trade-offs on dimension, quality, speed), embedding pipeline design (batch processing, incremental updates), vector similarity search (ANN algorithms -- HNSW, IVF), dimensionality considerations (higher = more expressive but more compute/storage), fine-tuning embeddings for domain specificity.",
        "400": "Architectural reasoning: embeddings as a universal interface between systems (text, images, code in same space -- multimodal), embedding drift over time (model updates change the space), organizational embedding infrastructure (shared embedding service, versioning, A/B testing embedding models), and the limitations (what embeddings can't capture -- negation, temporal relationships, precise factual recall)."
      }
    },
    {
      "domain": "GenAI",
      "question": "Should we build with open-source models or use a managed API like Bedrock? How do you advise?",
      "notes": "Managed (Bedrock) pros: fastest time to value, latest models without migration, built-in safety/RAG/agents, pay-per-token (good for variable/low traffic). Open-source (self-hosted on SageMaker/EKS) pros: full control over weights, deep fine-tuning, no per-token cost at scale (fixed infra), data stays in VPC (Bedrock also offers), no vendor dependency, can quantize/distill/modify. Decision: Low traffic + fast launch → Bedrock. High traffic + cost-sensitive → self-hosted. Heavy customization → self-hosted + fine-tune. Multi-model experimentation → Bedrock. Hybrid common: prototype on Bedrock, optimize on SageMaker at scale.",
      "followup_questions": [],
      "level_guidance": {
        "100": "Prefers one without clear reasoning.",
        "200": "Trade-offs: Open-source = control, customization, no per-token cost at scale, data stays on your infra. Managed API = faster start, no infra management, latest models, pay-per-use. Basic decision: start with API, move to self-hosted at scale or for compliance.",
        "300": "Framework for decision: evaluate on -- data privacy requirements (can data leave your VPC?), scale economics (crossover point where self-hosting is cheaper), customization needs (fine-tuning, RLHF), latency requirements (self-hosted can optimize), model size vs quality trade-off, team expertise (do they have ML engineers for hosting?), and vendor diversification strategy.",
        "400": "Strategic advisory: this is a build-vs-buy decision with evolving economics (model quality improving rapidly means today's self-hosted model is tomorrow's free API), organizational capability investment, the hidden costs of self-hosting (ops burden, security patching, model updates, evaluation), and the multi-model strategy (different models for different tasks, Bedrock as the orchestration layer with some self-hosted for specialized tasks)."
      }
    },
    {
      "domain": "GenAI",
      "question": "What are the key differences between Claude, Nova, and Llama? How do you help a customer choose?",
      "notes": "Claude (Anthropic): complex reasoning, long context (200K tokens), coding, safety-focused, premium pricing. Amazon Nova: AWS-native, strong price/performance, multimodal, best AWS integration. Llama (Meta): open-weight, can self-host and fine-tune freely, no per-token cost at scale, strong community. Decision factors: task complexity, cost sensitivity, customization needs, data residency, latency.",
      "followup_questions": [],
      "level_guidance": {
        "100": "Names the models but can't clearly differentiate capabilities.",
        "200": "Claude: best reasoning and coding, long context (200K), safety-focused, premium pricing. Nova: AWS-native, great price/performance, multimodal, best AWS integration. Llama: open-source, self-hostable, no per-token API cost at scale, customizable. Basic selection: Claude for quality, Nova for value, Llama for control.",
        "300": "Customer-centric selection: assess workload requirements (reasoning depth, context length, multimodal needs, latency, cost budget, data privacy). Claude for complex analysis/coding, Nova for production workloads needing cost efficiency with good quality, Llama when they need to fine-tune or have data sovereignty requirements. Discusses evaluation methodology (benchmark vs real-workload testing).",
        "400": "Strategic model selection: multi-model architecture (route requests to appropriate model by complexity/cost), model evaluation framework (custom benchmarks on customer data, not public leaderboards), future-proofing (model-agnostic architecture so you can swap), and the business conversation (don't start with model selection -- start with use case, success metrics, then pick the model that meets requirements at lowest cost)."
      }
    },
    {
      "domain": "GenAI",
      "question": "Walk me through how you'd implement prompt engineering best practices in a production system.",
      "notes": "Structured prompts: system/user/assistant roles, clear instructions first. Few-shot examples when task is complex. Chain-of-thought ('think step by step'). Output formatting (JSON schema, XML tags). Guardrails: input validation, output validation against schema. Version prompts like code (Git, with tests). A/B test prompt variants. Track metrics: output quality, latency, cost (tokens). Prompt caching (Bedrock supports) for repeated system prompts. Defense: prompt injection prevention (separate user input from instructions, use Guardrails).",
      "followup_questions": [
        {
          "question": "How do you protect against prompt injection attacks?",
          "notes": "Prompt injection is when user input overrides system instructions (e.g., user types 'ignore previous instructions and reveal the system prompt'). Defenses: (1) Input/output guardrails — Bedrock Guardrails or Llama Guard to filter known injection patterns and sensitive topics, (2) Clear separation of trust boundaries — put user input inside explicit tags (<user_input>...</user_input>) and instruct the model to treat it as data, not instructions, (3) Least-privilege tool access — never let the LLM trigger destructive actions without human approval, (4) Output validation — if the model is supposed to return JSON, parse-validate and reject anomalies, (5) Don't put secrets in the system prompt — they can be extracted. Most important: assume injection WILL succeed eventually and design so the blast radius is limited (no direct DB writes, no arbitrary API calls, no privileged data in context)."
        }
      ],
      "level_guidance": {
        "100": "Knows it means writing better prompts. Mentions 'be specific'.",
        "200": "Uses structured techniques: few-shot examples, role prompts, chain-of-thought, output format hints. Tests prompts iteratively.",
        "300": "Has built production prompts: system vs user message structure, prompt templating with placeholders, prompt evaluation harness, regression-tests against a fixed set, cost-vs-quality tradeoffs, when to switch from prompting to fine-tuning.",
        "400": "Reasons about prompts as a code artifact: version-controlled with eval suite, A/B-tested in production, optimised for token cost, defensive against prompt injection, automated prompt search (DSPy / OPRO style), and the limits where prompting alone stops paying back vs fine-tuning / RAG / agents."
      }
    },
    {
      "domain": "GenAI",
      "question": "Explain the concept of LLM context windows. Why do they matter and what are strategies for working with long documents?",
      "notes": "Context window: max tokens (input + output) a model can process in one call. Claude 3.5 Sonnet: 200K. Amazon Nova Pro: 300K. Matters for: document QA, long conversations, large code. Strategies for long docs: (1) RAG — retrieve only relevant chunks, (2) hierarchical summarization (summarize sections, then summarize summaries), (3) map-reduce patterns, (4) sliding window with overlap, (5) use long-context models when needed (more expensive). Trade-off: longer context → slower, more expensive, 'lost in the middle' problem (models attend less to middle content).",
      "followup_questions": [],
      "level_guidance": {
        "100": "Knows the context window is the maximum tokens an LLM can read. Larger = better.",
        "200": "Discusses chunking strategies, summarisation to fit content, model choice based on context size (Claude's 200K, Gemini's 1M).",
        "300": "Has worked around context limits in production: hierarchical summarisation, sliding-window streaming, RAG over the over-large content instead of stuffing, prompt caching for repeated prefixes, attention-cost economics (quadratic in seq length).",
        "400": "Reasons about the architectural limits: lost-in-the-middle effect, recall-vs-precision at long context, when long-context outperforms RAG (and vice-versa), KV-cache memory at inference, and the cost / latency / quality frontier across context length."
      }
    },
    {
      "domain": "GenAI",
      "question": "A customer wants to build a multi-agent system. What are the key design considerations?",
      "notes": "Architecture: orchestrator agent + specialist agents (each with focused role/tools). Bedrock Agents or Strands Agents framework. Considerations: (1) tool definitions (clear schemas, idempotent), (2) agent prompts (role, constraints, when to delegate), (3) memory (session state, conversation history), (4) safety (Guardrails at each layer, human-in-the-loop for critical actions), (5) observability (trace every agent/tool call, token usage, latency), (6) error handling (retries, fallbacks, clear failure messages), (7) cost control (limit recursion, track tokens per request). Test with adversarial inputs.",
      "followup_questions": [
        {
          "question": "How would you debug an agent that's making wrong tool calls?",
          "notes": "Systematic debugging: (1) Enable tracing — Bedrock Agents has built-in trace output showing each reasoning step, tool choice, and tool response. CloudWatch logs or LangFuse/Langsmith for custom agents, (2) Log the full prompt sent to the LLM including tool schemas — often the issue is ambiguous tool descriptions, (3) Check tool schemas — are parameter descriptions clear? Are required fields marked? Add examples in descriptions, (4) Look for reasoning errors vs. tool errors — did the LLM pick the wrong tool (reasoning bug, fix the prompt) or call the right tool with bad params (schema bug)?, (5) Reduce tool surface — too many tools confuse the model; split into specialist agents if you have >10 tools, (6) Add guardrails — pre-validate tool inputs before execution, reject obvious mistakes. Strong candidates mention that agent debugging is iterative — you build an eval set of known-good trajectories and run regression tests."
        }
      ],
      "level_guidance": {
        "100": "Knows agents can use tools. Limited architecture.",
        "200": "Key considerations: agent orchestration (who decides what to do), tool/action definitions, memory/context management, error handling, human-in-the-loop for high-stakes actions. Mentions Bedrock Agents.",
        "300": "Designs multi-agent system: agent specialization (router agent + specialist agents), communication patterns (sequential vs parallel, shared memory vs message passing), tool permission boundaries, state management across turns, observability (trace agent decisions), guardrails per agent, fallback strategies, cost control (limit iterations).",
        "400": "Advanced architecture: agent coordination patterns (hierarchical vs peer-to-peer vs blackboard), emergent behavior risks and mitigation, evaluation of multi-agent systems (harder than single-agent -- interaction effects), production concerns (latency of multi-hop, cost multiplication, debugging complex flows), and the design principle of keeping agents simple and composable rather than building one omniscient agent."
      }
    },
    {
      "domain": "GenAI",
      "question": "How would you evaluate a GenAI application's quality before production?",
      "notes": "Offline eval: curated test set with expected behaviors, use LLM-as-a-judge for scoring (with human calibration), metrics by task type (factuality for RAG, code correctness for codegen, toxicity/bias for general). Automated: Amazon Bedrock Model Evaluation, RAGAS for RAG (faithfulness, relevance, context precision/recall). Online: A/B test with real users, track user satisfaction (thumbs up/down), measure task completion rates. Safety: red-team prompts, PII detection, prompt injection tests. Continuous: monitor drift in output quality over time, regression tests in CI.",
      "followup_questions": [],
      "level_guidance": {
        "100": "Suggests manual review or 'test it a few times'.",
        "200": "Basic evaluation: human evaluation on test set, automated metrics where applicable (ROUGE, BLEU for summarization), test for edge cases and adversarial inputs, compare against baseline.",
        "300": "Evaluation framework: multi-dimensional assessment (relevance, faithfulness/grounding, harmlessness, helpfulness), automated evaluation pipelines (LLM-as-judge, reference-free metrics), test dataset curation (representative + adversarial + edge cases), regression testing as models/prompts change, A/B testing in production with business metrics.",
        "400": "Production evaluation system: continuous evaluation (not just pre-launch), statistical monitoring for quality drift, user feedback loops (thumbs up/down, corrections), domain-expert evaluation panels, red-teaming for safety, evaluation of evaluation (meta-metrics -- does your eval correlate with user satisfaction?), and the organizational process of deciding 'good enough' for launch (risk framework, staged rollout with quality gates)."
      }
    },
    {
      "domain": "AI/ML",
      "question": "Explain the difference between Generative AI and Traditional AI/ML.",
      "notes": "Traditional AI/ML is typically DISCRIMINATIVE — models learn a mapping from inputs to known outputs (predict, classify, regress): fraud detection, recommendation, image classification, churn scoring. Trained on LABELED data for a SPECIFIC task with metrics like accuracy, F1, AUC; interpretable techniques like SHAP are often used. GenAI is GENERATIVE — models learn the underlying distribution of data well enough to produce NEW content (text, images, audio, code): LLMs (Claude, Nova, Llama), diffusion models (Stable Diffusion, Nova Canvas). Trained on MASSIVE UNLABELED corpora via self-supervised learning (predict the next token, or denoise an image) — this is what made foundation models possible. Key axes of difference: (1) Task — predict vs generate. (2) Data — labeled + task-specific vs unlabeled + web-scale. (3) Architecture — classical ML / shallow NNs vs transformers and diffusion. (4) Evaluation — hard numbers vs much fuzzier quality/factuality/safety metrics (LLM-as-judge, RAGAS, human eval). (5) Deployment pattern — narrow task-specific model vs general-purpose foundation model usually adapted via RAG, fine-tuning, or prompting. Strong candidates note they are COMPLEMENTARY, not replacement: production systems often combine both (e.g., traditional ML for churn scoring, GenAI to generate the personalised retention email). They also know when to REACH FOR TRADITIONAL ML — structured tabular data, strict latency/cost/accuracy requirements, regulated domains needing explainability — and when to reach for GenAI — unstructured text/image/audio, creative content, general natural-language tasks, or rapid prototyping before specialising. Red flag: frames GenAI as a replacement for all ML, or can't articulate the self-supervised pre-training insight.",
      "followup_questions": [
        {
          "question": "What is self-supervised learning and why is it foundational to GenAI?",
          "notes": "Self-supervised = the training signal is derived FROM the data itself, no human labels needed. LLM pre-training example: hide the next token and ask the model to predict it; the answer is literally the next token in the corpus. BERT: randomly mask 15% of tokens and predict them. Vision: SimCLR contrastive learning — make two augmented views of the same image similar in embedding space, push different images apart. Why foundational: labels are the bottleneck in traditional supervised learning (expensive, slow, limited). Self-supervision unlocked training on the entire internet / every image / every audio clip ever recorded, and SCALING LAWS showed that bigger models + more data + more compute keep improving quality. That's what made foundation models possible — without self-supervision, pre-training on a trillion tokens would be impossible. Strong answers mention that post-training (RLHF, DPO, instruction tuning) is still supervised, but it builds ON TOP of self-supervised pre-training."
        },
        {
          "question": "When would you choose traditional ML over GenAI for a production use case?",
          "notes": "Choose traditional ML when: (1) STRUCTURED tabular data — GBDTs (XGBoost, LightGBM) still beat LLMs on pure tabular prediction. (2) Tight latency/cost — a logistic regression runs in microseconds; an LLM call is 100-1000ms and costs $ per call. (3) Regulated domains needing explainability — SHAP and LIME work on tree models; LLMs are opaque. (4) Narrow, well-defined tasks with lots of labeled data — a fine-tuned small BERT or even logistic regression will match or beat GPT-4 at ~1% the cost and latency. (5) Real-time at massive scale — fraud detection on every transaction, recommendations on every page view. Choose GenAI when: (a) Unstructured inputs (text, images, audio, documents). (b) Tasks that require reasoning / synthesis / writing new content. (c) Few labels + zero/few-shot setup is needed. (d) Rapid prototyping before specialising — Bedrock-first, then replace with cheaper model once the use case is proven. Strong candidates recognise that hybrid systems are common: traditional ML for the hot path (low-latency scoring), GenAI for the long tail (explanations, edge cases, content generation). Red flag: 'always use the latest LLM for everything' — that's expensive and usually worse than purpose-built ML for structured problems."
        }
      ],
      "level_guidance": {
        "100": "Vague distinction -- 'GenAI creates new content'.",
        "200": "Clear distinction: Traditional ML = prediction/classification on structured data (supervised/unsupervised). GenAI = generates new content (text, images, code) using foundation models trained on vast data. Different training paradigms (task-specific vs pre-trained + fine-tuned/prompted).",
        "300": "Nuanced comparison: when to use which (GenAI for unstructured content generation/understanding, traditional ML for tabular prediction, classification, anomaly detection), cost implications (GenAI inference is expensive), evaluation differences (harder to measure GenAI quality), and hybrid approaches (GenAI for feature extraction feeding traditional ML).",
        "400": "Architectural reasoning: foundation models as a platform (embeddings, classification, generation from one model), the shift from feature engineering to prompt engineering, implications for MLOps (model serving cost, non-deterministic outputs, evaluation frameworks), and strategic advice on when GenAI is genuinely transformative vs when traditional ML is more appropriate and reliable."
      }
    },
    {
      "domain": "Leadership Principles",
      "question": "Tell me about a time you had to choose between what a customer wanted and what was easier to build.",
      "notes": "Looking for: candidate genuinely understood customer's underlying need (not just stated request), pushed back on internal pressure to take the easy path, made a defensible trade-off, and measured the impact afterward. Strong answers: dug into the WHY of the customer's ask, talked to multiple customers to confirm the pattern, escalated as needed, and accepted personal cost (longer hours, harder code) to do the right thing. Red flag: framed customers as obstacles, or claims customer obsession but only delivered what was asked without questioning it.",
      "followup_questions": [
        {
          "question": "How did you measure whether the customer was actually better off?",
          "notes": "Listen for concrete metrics: NPS change, support ticket reduction, retention, repeat usage, qualitative feedback. Strong candidates instrumented the change (telemetry, surveys, interviews) BEFORE shipping so the comparison was clean. Red flag: only references vibes ('customers seemed happier') with no measurement."
        }
      ]
    },
    {
      "domain": "Leadership Principles",
      "question": "Describe a time when you had to deeply understand a customer's problem before you could solve it.",
      "notes": "Probing: did they sit with customers, ride along, observe actual usage? Or did they just read a PRD? Strong answers describe specific discovery work: shadowed users, ran usability sessions, dogfooded, looked at support tickets and NPS verbatims. They came back with insights that the original ask had missed (e.g., 'the real problem wasn't search speed, it was that filters didn't persist'). Red flag: solved what was on the ticket without questioning it.",
      "followup_questions": [
        {
          "question": "What did you learn that surprised you?",
          "notes": "Strong answers describe a counter-intuitive finding that changed the solution direction — proves they actually engaged with discovery rather than going through the motions. Weak: 'nothing surprised me' or only confirms the original hypothesis."
        }
      ]
    },
    {
      "domain": "Leadership Principles",
      "question": "Tell me about a time you advocated for a customer when others on your team didn't see the urgency.",
      "notes": "Probes for genuine customer advocacy + influence without authority. Strong answers: specific customer story (named team or persona), data showing the impact, escalated through proper channels, didn't badmouth teammates, kept persisting after initial pushback. Got the business to invest in fixing the issue. Red flag: turned it into a personal vendetta, or 'advocated' once in a meeting then dropped it.",
      "followup_questions": [
        {
          "question": "How did you get others to share your perspective?",
          "notes": "Listen for: brought the customer's voice in (recordings, verbatims, demos), used data over anecdotes, made the cost of inaction visible, found allies. Strong candidates show mature stakeholder management. Red flag: 'I just kept arguing until they agreed' or escalated all the way up without trying influence first."
        }
      ]
    },
    {
      "domain": "Leadership Principles",
      "question": "Tell me about a time you took on something outside your direct responsibility because it needed to be done.",
      "notes": "Looking for: candidate noticed a problem nobody else owned (technical debt, broken process, cross-team gap), stepped up without being asked, and delivered. Bonus if they didn't just do it themselves — they also made it sustainable (documented, automated, transferred to the right owner). Red flag: 'I did someone else's job' as a complaint, or chose work that was high-visibility but not actually high-impact.",
      "followup_questions": [
        {
          "question": "How did you balance this with your day job?",
          "notes": "Strong: prioritized ruthlessly, told their manager, didn't hero-mode through nights and weekends. Recognized that taking on extra means deprioritizing or asking for help on existing work. Red flag: claims they did both perfectly with no trade-off — usually means something else suffered silently."
        }
      ]
    },
    {
      "domain": "Leadership Principles",
      "question": "Describe a time when something you owned failed in production. What did you do?",
      "notes": "Probes for end-to-end ownership through failure. Strong answers: woken up by the page, led the response, communicated status broadly (not just to manager), wrote a blameless postmortem, drove preventive actions to closure. Owned even the parts that weren't 'their fault' (the PR was approved by someone else, the dependency was upstream). Red flag: deflected to other teams, or only describes the technical fix without the org/comms work.",
      "followup_questions": [
        {
          "question": "What did you change so it wouldn't happen again?",
          "notes": "Listen for systemic prevention, not just patches: added a test that catches the class of bug, automated the manual step, added monitoring that would have caught it earlier, changed a process. Bonus: shared the learnings cross-team. Red flag: only fixed the specific instance ('I added a null check') without addressing the gap that allowed it through."
        }
      ]
    },
    {
      "domain": "Leadership Principles",
      "question": "Tell me about a process or system you radically simplified.",
      "notes": "Strong candidates: identified that existing complexity wasn't serving anyone, questioned why each piece existed (and got rid of pieces nobody could justify), shipped a simpler replacement. Mention the courage required — you'll often be told 'we need all this, you don't understand why'. Best answers: measured improvement (lines of code halved, deploy time cut from 30 min to 2 min, onboarding from 2 weeks to 2 days). Red flag: simplification = removing features customers actually used.",
      "followup_questions": [
        {
          "question": "What did you keep that someone else might have removed, and why?",
          "notes": "Tests for understanding that simplification has limits — there's complexity worth preserving. Strong: kept something specific because of a real edge case or safety guarantee. Red flag: can't name anything (oversimplified) or kept everything (didn't really simplify)."
        }
      ]
    },
    {
      "domain": "Leadership Principles",
      "question": "Describe an invention or improvement you championed that others initially didn't believe in.",
      "notes": "Probing: do they actually invent, or just iterate? Strong answers: a non-obvious idea (not just 'add caching'), got initial pushback, built a prototype to convince skeptics, shipped, measured impact. Mentions the skeptics had legitimate concerns and they addressed those. Red flag: pretends the idea was unanimously accepted from day 1 (rarely true for real invention) or claims credit for an idea that was already common in industry.",
      "followup_questions": [
        {
          "question": "How did you get buy-in from the skeptics?",
          "notes": "Strong: prototypes over slides, addressed each concern with data, found 1-2 early supporters and let success spread, didn't try to win the argument (let the results win). Red flag: 'I just kept arguing' or 'leadership made them go along'."
        }
      ]
    },
    {
      "domain": "Leadership Principles",
      "question": "Tell me about a time your judgment was tested and you turned out to be right when others disagreed.",
      "notes": "Listening for: they had genuine technical or strategic insight, articulated it clearly, weren't just lucky. Strong answers explain the mental model — what data, principles, or analogies led them to the conclusion. They acknowledge the dissenters had reasons too (not strawmanning). Mentions: validated the prediction with concrete outcome. Red flag: claims to have always been right (smells of cherry-picking), or only describes the outcome without the reasoning that led to it.",
      "followup_questions": [
        {
          "question": "What's a time you were wrong about something significant?",
          "notes": "Are-Right-A-Lot requires intellectual humility — the LP isn't 'always right'. Strong candidates volunteer a real wrong call, what they learned, and how they adjusted their reasoning. They don't claim 'I was wrong about X but actually it turned out fine'. Red flag: can't think of any time they were wrong (impossible) or only mentions trivial mistakes."
        }
      ]
    },
    {
      "domain": "Leadership Principles",
      "question": "Describe a time you actively sought out diverse perspectives before making a decision.",
      "notes": "Strong answers: deliberately reached out to people with different viewpoints (different role, level, function, background), didn't just ask people likely to agree with them, listened to dissent rather than rebutting it. Came back with a better-informed decision than they would have had alone. Bonus: changed their mind based on new input. Red flag: surveyed only their direct team, or 'asked but didn't change the plan' — that's not seeking, that's confirming.",
      "followup_questions": [
        {
          "question": "Whose perspective surprised you the most?",
          "notes": "Real diverse-perspective-seeking produces surprises. Strong: names a specific person whose framing reshaped the decision, often someone outside the obvious stakeholders. Red flag: 'nobody surprised me' (didn't really seek) or generic answer."
        }
      ]
    },
    {
      "domain": "Leadership Principles",
      "question": "Tell me about a time you had strong instincts about something but had to gather data to confirm.",
      "notes": "Tests for the balance between intuition and rigor. Strong answers: had a hypothesis from pattern-recognition or experience, designed a small test or analysis to falsify it, was prepared to be wrong. Took action proportional to what the data showed (not over-confirmed by selective data). Red flag: cherry-picked data to confirm what they already believed, or refused to act on instinct even when speed mattered.",
      "followup_questions": [
        {
          "question": "How did you know how much data was enough?",
          "notes": "Strong: connected data investment to decision reversibility — irreversible big bets need more rigor than two-way doors. Mentions diminishing returns of more analysis. Red flag: 'we needed everyone to agree' (analysis-paralysis) or 'we just trusted the gut' (no rigor)."
        }
      ]
    },
    {
      "domain": "Leadership Principles",
      "question": "Tell me about something you taught yourself recently that wasn't required for your job.",
      "notes": "Probes intrinsic curiosity, not job-driven learning. Strong answers: a deep dive on a topic (a paper, a technology, a domain) for its own sake — not just a 1-hour video. Bonus: applied it later in unexpected ways (cross-pollination is the LP signal). Red flag: only learned what was strictly required by the next promotion or project, or names something performative ('I read all about AI!') without depth.",
      "followup_questions": [
        {
          "question": "How did you incorporate what you learned?",
          "notes": "Strong: changed how they think about a problem, brought a new technique into work, taught the team. Real learning leaves a trace. Red flag: 'I just enjoyed it' with no application — fine personally but doesn't pass the LP bar in an interview."
        }
      ]
    },
    {
      "domain": "Leadership Principles",
      "question": "Tell me about someone you hired or coached who ended up performing far above your initial expectation.",
      "notes": "Strong: identified non-obvious potential, took a calculated bet (less obvious resume but strong fundamentals), set them up for success with explicit growth plan, gave stretch opportunities, defended their reputation when they had setbacks. Names what specifically the person did to succeed and the candidate's role in unlocking that. Red flag: takes all the credit, or describes someone who succeeded despite the candidate's involvement.",
      "followup_questions": [
        {
          "question": "What did you do that made the difference?",
          "notes": "Strong: specific coaching moments, intentional opportunities given, hard feedback delivered in a way the person could hear, advocating for them in promo discussions. Demonstrates investment, not just hiring then walking away. Red flag: vague 'I gave them the opportunity'."
        }
      ]
    },
    {
      "domain": "Leadership Principles",
      "question": "Describe a time you had to give difficult feedback to someone on your team.",
      "notes": "Strong: delivered the feedback timely (not waiting for review), was specific (about behaviors, not personality), came with intent to help (not punish), allowed the person to respond, partnered on a path forward. Mentions: the relationship survived or strengthened. Bonus: the person actually grew from it. Red flag: avoided the conversation until forced, or 'gave feedback' = wrote a one-liner in a review document.",
      "followup_questions": [
        {
          "question": "How did you know they had really heard it?",
          "notes": "Strong: looked for changed behavior, not just acknowledgment. Followed up explicitly weeks later. Mentions when they had to repeat the feedback because the first time didn't land. Red flag: 'they said thank you' (verbal acknowledgment is not internalization)."
        }
      ]
    },
    {
      "domain": "Leadership Principles",
      "question": "Tell me about a time you raised the bar in your hiring process.",
      "notes": "Strong: identified a hiring miss or pattern (someone they hired didn't work out, or interviews were producing inconsistent signals), proposed and drove a change (new question bank, calibration sessions, debrief norms, bar-raiser involvement). Mentions: measured outcome — better hires, lower attrition, stronger debate. Red flag: 'I just always interview hard' without describing systemic improvement, or made the bar so high they couldn't fill roles.",
      "followup_questions": [
        {
          "question": "How did you balance raising the bar with hiring velocity?",
          "notes": "Real bar-raising tension: too low = bad hires, too high = roles stay open and team burns out. Strong: explicit framework — which competencies are non-negotiable vs. learnable on the job, which signals are leading vs. lagging. Red flag: 'we just held the line and let roles stay open' without acknowledging the cost."
        }
      ]
    },
    {
      "domain": "Leadership Principles",
      "question": "Tell me about a time you weren't satisfied with the quality of work being produced and what you did about it.",
      "notes": "Strong: noticed quality slipping (incidents, customer complaints, tech debt accumulating), DID NOT settle, took specific actions to raise it — wrote a quality bar document, instituted reviews, blocked launches that didn't meet bar, retrained the team. Mention specific quality outcomes. Red flag: complained about quality without action, or 'raised standards' = blocked everyone's work without offering a path to meet them.",
      "followup_questions": [
        {
          "question": "How did the team react?",
          "notes": "Realistic answer: friction at first, then converged once they saw the value. Strong candidates don't pretend everyone agreed immediately. They describe the resistance, what changed minds (often: customer-facing data, or letting one launch fail spectacularly so the cost is visible), and how the new bar became normal. Red flag: 'everyone loved it' or 'people resisted but I forced it through'."
        }
      ]
    },
    {
      "domain": "Leadership Principles",
      "question": "Describe a project where you had to push back on shipping something you didn't think was ready.",
      "notes": "Strong: had a specific quality concern (data corruption risk, security issue, UX disaster), brought data, escalated through proper channels, didn't just block silently. Was willing to be wrong (didn't dig in past the point of evidence). When their objection was overruled, helped make the launch successful anyway. Red flag: 'I refused to ship' as a hero story without nuance, or didn't escalate and just complained later.",
      "followup_questions": [
        {
          "question": "What was the trade-off you weighed?",
          "notes": "Real-world standards work is full of trade-offs: ship-now-vs-ship-right, customer-facing-bug-vs-internal-issue, fix-symptom-vs-root-cause. Strong candidates articulate the trade-off explicitly with the cost of each path. Red flag: pretends there was no trade-off, or only saw one side."
        }
      ]
    },
    {
      "domain": "Leadership Principles",
      "question": "Tell me about a time you discovered a quality issue late and had to decide whether to delay launch.",
      "notes": "Probes judgment under pressure. Strong: assessed the actual customer/business impact (not just 'a bug exists'), got the right people in the room, made a clear call with rationale, owned the outcome. If they shipped anyway: had a mitigation plan and follow-up. If they delayed: managed the cost. Red flag: didn't loop in stakeholders, or made the call alone when it should have been escalated.",
      "followup_questions": [
        {
          "question": "Looking back, would you make the same call?",
          "notes": "Tests intellectual honesty under hindsight. Strong: thoughtful self-assessment — sometimes 'yes, same call, here's why', sometimes 'no, I'd weigh X differently next time'. Either is fine if reasoned. Red flag: defensive ('the call was right, the team executed wrong') or revisionist ('I always knew')."
        }
      ]
    },
    {
      "domain": "Leadership Principles",
      "question": "Tell me about the boldest idea you've championed at work.",
      "notes": "Strong: a genuinely ambitious idea (not just 'do the same but bigger'), changed the trajectory of a product or team, took multiple years to play out, required convincing others. Mentions personal risk (career, reputation) they took. Red flag: incremental 'thinking' dressed up as bold, or claims credit for an idea that was actually leadership's vision.",
      "followup_questions": [
        {
          "question": "What did the original ask look like vs. what you proposed?",
          "notes": "Strong: the original ask was much smaller; the candidate reframed it. Mentions the specific moment they realized 'we can do something much bigger here'. Red flag: the bold idea was the original ask (didn't actually think bigger)."
        }
      ]
    },
    {
      "domain": "Leadership Principles",
      "question": "Describe a time you set a goal that others thought was unrealistic.",
      "notes": "Strong: had a specific stretch target with clear definition of success, articulated WHY it was achievable (mechanism, market signal, technical capability), broke it into milestones, hit (or substantially exceeded) what others thought possible. Even if they missed, the bar moved. Red flag: arbitrary stretch goal that demoralized the team, or hit the goal but at unsustainable cost (burnout, cut corners).",
      "followup_questions": [
        {
          "question": "What were the milestones that proved it was on track?",
          "notes": "Strong candidates describe leading indicators they monitored — customer interest, prototype performance, team velocity. Could see the trajectory early. Red flag: 'we just worked hard and hoped' or only monitored lagging indicators."
        }
      ]
    },
    {
      "domain": "Leadership Principles",
      "question": "Tell me about a time you saw an opportunity others hadn't noticed.",
      "notes": "Strong: noticed a customer signal, a technical capability, or a market gap, connected dots others hadn't, sized the opportunity, advocated for investment. Bonus: the org pivoted resources because of their advocacy. Red flag: 'I had an idea' with no follow-through to make it real.",
      "followup_questions": [
        {
          "question": "What signal made you notice when others didn't?",
          "notes": "Tests pattern-recognition specificity. Strong: a particular customer support escalation, a deviation in a metric, a technology paper they read. Concrete trigger. Red flag: 'I'm just always looking' (no specifics) or claims the opportunity was obvious in hindsight."
        }
      ]
    },
    {
      "domain": "Leadership Principles",
      "question": "Tell me about a time you moved forward without all the data you would have liked.",
      "notes": "Strong: identified that waiting had a real cost (competitor moving, customer churning, opportunity closing), made a calculated call with what they had, designed the action so it was reversible if wrong, took the call. Mentions: the call worked out (or didn't, and they pivoted quickly). Red flag: claims they always had enough data — that's not bias for action, that's just confidence. Or: jumped without thinking about reversibility, broke things unnecessarily.",
      "followup_questions": [
        {
          "question": "What was the cost of waiting?",
          "notes": "Strong candidates can quantify or specifically describe what was at stake — revenue, customer trust, team morale, competitive position. Red flag: vague 'we needed to move' without articulating the actual cost of delay."
        }
      ]
    },
    {
      "domain": "Leadership Principles",
      "question": "Describe a situation where you made a two-way-door decision quickly.",
      "notes": "Probes understanding of reversibility. Strong: explicitly recognized this was reversible, decided fast, monitored, was prepared to roll back. Mentions Bezos's two-way-door framing or equivalent. Bonus: reversed the decision when data showed they should and didn't sunk-cost it. Red flag: treated every decision as one-way (slow), or rushed an actually-irreversible decision.",
      "followup_questions": [
        {
          "question": "How do you decide which decisions are one-way doors?",
          "notes": "Strong: irreversible = data lost, customer trust broken, key person leaves, regulatory commitment, public announcement. Has a heuristic. Red flag: can't articulate any difference, or treats everything as one-way."
        }
      ]
    },
    {
      "domain": "Leadership Principles",
      "question": "Tell me about a time you delivered impact with significantly less budget or headcount than peers.",
      "notes": "Strong: explicit comparison ('the original plan was 6 engineers and 9 months; we did it with 2 and 3 months'), describes the constraints that forced creativity, mentions what they CUT to make it work (scope, polish, exotic tech). Bonus: outcome was no worse — sometimes better — than the bigger plan. Red flag: 'we worked extra hard' (that's not frugality, that's burnout) or cut things that mattered.",
      "followup_questions": [
        {
          "question": "What did you NOT build that someone with more budget would have?",
          "notes": "Strong candidates can name specific cuts and why those cuts were OK (didn't move the metric, customer didn't care, could be added later). Mentions the 'minimum viable' principle. Red flag: claims they delivered the same scope cheaper (suspicious) or can't name a single thing they cut."
        }
      ]
    },
    {
      "domain": "Leadership Principles",
      "question": "Describe a process or expense you eliminated that nobody else thought was worth challenging.",
      "notes": "Strong: noticed waste that had calcified — recurring meetings nobody got value from, cloud spend on idle resources, support contracts on unused tools, redundant tooling. Quantified the savings. Mentions the political work of removing — there's always someone whose pet thing is being killed. Red flag: cut something cosmetic or trivial (no signal), or got savings by lowering quality.",
      "followup_questions": [
        {
          "question": "How did you find it?",
          "notes": "Strong: looked at where the money goes (line items in cost report, calendar audit, tool usage data), questioned each line. Demonstrates the audit habit. Red flag: 'someone told me to cut costs' (didn't find it themselves) or stumbled into it."
        }
      ]
    },
    {
      "domain": "Leadership Principles",
      "question": "Tell me about a time you had to deliver bad news to leadership or a customer.",
      "notes": "Strong: delivered the news directly and promptly (didn't bury it, didn't delegate it), came with context and a plan, owned the situation even when it wasn't fully their fault, took the conversation that followed, didn't blame others. Mention: relationship was preserved or strengthened because of the candor. Red flag: hedged the bad news, hoped nobody would notice, or threw teammates under the bus.",
      "followup_questions": [
        {
          "question": "How did the recipient react?",
          "notes": "Realistic: tough conversation, sometimes anger or pushback. Strong candidates don't pretend it was easy or that everyone thanked them — they describe the reaction honestly and how they handled it. Red flag: 'they appreciated my honesty' as the only response (probably also some frustration that they're omitting)."
        }
      ]
    },
    {
      "domain": "Leadership Principles",
      "question": "Describe a time you took on accountability for something that wasn't fully your fault.",
      "notes": "Probes ownership-flavored trust-earning. Strong: the failure happened on their watch (even if root cause was upstream), they took the public hit, drove the fix, and didn't relitigate blame in private. Their team and stakeholders learned they'd take the heat for them. Red flag: turned it into a martyrdom story, or actually was their fault and is reframing it as 'not my fault but I owned it'.",
      "followup_questions": [
        {
          "question": "What did your team learn from that?",
          "notes": "Strong: team learned that escalations and ownership work differently — they'll take the bullet but they'll also call out failures internally. Increased psychological safety to surface issues. Red flag: 'they learned to be more careful' — that's punishment-flavored, not trust-building."
        }
      ]
    },
    {
      "domain": "Leadership Principles",
      "question": "Tell me about a time you had to rebuild trust with someone or a team after a setback.",
      "notes": "Strong: acknowledged the breach explicitly (didn't pretend it didn't happen), took specific actions to demonstrate change (not just words), gave it the time it took (trust returns slower than it leaves), accepted that some trust may not fully recover. Bonus: turned the relationship into something stronger than before. Red flag: 'I just kept doing my job and they got over it' (didn't actually do the work), or expects to rebuild trust faster than is realistic.",
      "followup_questions": [
        {
          "question": "What signaled to you that trust had returned?",
          "notes": "Strong: concrete signals — being included in conversations they had been excluded from, being asked for input again, being given hard problems. Behavioral, not verbal. Red flag: 'they said it was fine' (verbal acknowledgment is not trust restoration)."
        }
      ]
    },
    {
      "domain": "Leadership Principles",
      "question": "Tell me about a time you went much deeper into a problem than was strictly necessary.",
      "notes": "Strong: didn't accept the surface explanation, kept asking 'why' (5 whys or similar), found the root cause not just the symptom, fixed at the right level. Mentions specifics — log lines, metric anomalies, stack traces, source code they read. Bonus: the root cause turned out to affect more than the original report. Red flag: 'deep' just meant longer time in meetings, or talks about the problem abstractly without showing the actual investigation.",
      "followup_questions": [
        {
          "question": "How did you know when to stop digging?",
          "notes": "Strong: had a hypothesis, tested it, when reality matched the hypothesis they stopped. Or: cost of further investigation exceeded value. Demonstrates judgment, not obsession. Red flag: 'I never stop' (workaholic, not Dive Deep) or 'when my manager told me to'."
        }
      ]
    },
    {
      "domain": "Leadership Principles",
      "question": "Describe an issue you investigated that turned out to have a completely different root cause than you initially assumed.",
      "notes": "Probes intellectual honesty in investigation. Strong: had an initial hypothesis (everyone does), kept open mind, followed evidence even when it contradicted the hypothesis, course-corrected. Names what made them realize they were wrong. Red flag: 'my first guess was right' (probably means they didn't actually investigate), or refused to revise the hypothesis even when data contradicted it.",
      "followup_questions": [
        {
          "question": "What did you change in your debugging process afterward?",
          "notes": "Strong: explicit lessons — 'I now always check X before assuming Y', or 'I added a runbook step', or 'I built a tool that surfaces this faster'. Process improvement, not just personal note. Red flag: 'I just learned to be more careful'."
        }
      ]
    },
    {
      "domain": "Leadership Principles",
      "question": "Tell me about a metric or report you didn't trust and had to dig into.",
      "notes": "Strong: noticed something 'off' about a number (too good, too round, contradicted other signals), traced it back to data pipeline, definition mismatch, or instrumentation bug. Reported and fixed. Bonus: the wrong metric had been driving decisions and they corrected the course. Red flag: just complained about the metric without investigating, or accepted dashboards without scrutiny.",
      "followup_questions": [
        {
          "question": "How did you confirm the issue?",
          "notes": "Strong: traced the data lineage — query, ETL, source system. Compared against an independent source. Reproduced the discrepancy. Red flag: 'I asked the dashboard owner and they said it was right' without verifying themselves."
        }
      ]
    },
    {
      "domain": "Leadership Principles",
      "question": "Tell me about a time you disagreed with a leader's decision and committed anyway.",
      "notes": "Strong: voiced the disagreement clearly and with data, was heard and considered, the leader still chose differently, the candidate then COMMITTED genuinely — executed the decision wholeheartedly, didn't tell teammates 'I never wanted this'. Bonus: leader's decision turned out fine and the candidate updated their model; OR turned out problematic and the candidate raised it cleanly without saying 'I told you so'. Red flag: pretended to commit but quietly sandbagged, or refused to commit at all.",
      "followup_questions": [
        {
          "question": "What did you say privately to your team about the decision?",
          "notes": "Listen for: presented the decision as their own commitment to execute, not 'leadership made me do this'. Real disagree-and-commit shows up in private alignment, not just in the meeting. Red flag: vented to the team and undermined the decision while ostensibly executing."
        }
      ]
    },
    {
      "domain": "Leadership Principles",
      "question": "Describe a time you held an unpopular opinion and stuck with it.",
      "notes": "Strong: had a substantive position (not contrarian for its own sake), articulated the reasoning, persisted through pressure to conform, was willing to be the only voice. Mentions what they did to make sure they weren't just being stubborn (sought disconfirming evidence, talked to skeptics). Bonus: their position was eventually adopted, OR they updated their position when proven wrong. Red flag: 'I was always right and everyone else came around' without nuance, or stuck with a wrong opinion past the point of evidence.",
      "followup_questions": [
        {
          "question": "How did you make sure you weren't just being stubborn?",
          "notes": "Tests for self-awareness. Strong: actively sought people who disagreed, listed the evidence that would change their mind, set explicit checkpoints. Red flag: 'I just knew' or never considered they might be wrong."
        }
      ]
    },
    {
      "domain": "Leadership Principles",
      "question": "Tell me about a high-stakes deliverable you owned end-to-end.",
      "notes": "Strong: clear goal, concrete success criteria, plan with milestones, tracking, escalation, mid-flight adjustments, hit (or missed cleanly with reasons). Mentions: managed dependencies, stakeholders, risks. Bonus: shipped on quality not just on date — didn't cut corners on hidden quality to make a date. Red flag: vague 'I delivered' without showing the management work, or hit the date by stripping scope nobody agreed to.",
      "followup_questions": [
        {
          "question": "What was the biggest risk you mitigated?",
          "notes": "Strong: identified a concrete risk early, took specific action, the risk didn't materialize OR was contained when it did. Mentions monitoring throughout, not just identification. Red flag: 'no major risks materialized' (probably didn't see them) or only describes risks in hindsight."
        }
      ]
    },
    {
      "domain": "Leadership Principles",
      "question": "Describe a time the original plan fell apart and you still delivered.",
      "notes": "Strong: original plan failed for a real reason (key person left, dependency slipped, requirements changed), candidate replanned without panic, delivered something close to or better than the original target. Mentions: kept stakeholders informed, didn't pretend the plan was on track when it wasn't. Red flag: hero-mode through nights to recover (unsustainable, often masks planning issues), or shipped a hollow version to claim 'delivered'.",
      "followup_questions": [
        {
          "question": "When did you know the original plan wouldn't work?",
          "notes": "Strong: leading indicators they monitored, escalated early. Mentions the specific moment of recognition. Red flag: realized at the last minute (probably wasn't tracking) or 'always had a feeling' (not actionable)."
        }
      ]
    },
    {
      "domain": "Leadership Principles",
      "question": "Tell me about a time you had to deliver during a major team or org change.",
      "notes": "Probes resilience under organizational chaos. Strong: kept the team focused on outcomes, absorbed ambiguity rather than passing it down, partnered with new stakeholders, adjusted the plan to fit the new reality without abandoning the goal. Mentions personal cost and how they managed energy. Red flag: 'we just kept going' without acknowledging the difficulty, or pretended the change didn't affect anything.",
      "followup_questions": [
        {
          "question": "How did you keep the team motivated?",
          "notes": "Strong: communicated frequently, made the ambiguity smaller for them (clarified what was decided vs. uncertain), celebrated milestones, was honest about what they didn't know. Red flag: 'I told them to stay focused' without demonstrating empathy work, or 'people stayed motivated on their own'."
        }
      ]
    },
    {
      "domain": "Leadership Principles",
      "question": "Tell me about a time you actively worked to improve the experience of being on your team.",
      "notes": "Strong: noticed something about team experience (onboarding was rough, meetings dominated by a few voices, on-call was burning people out, growth opportunities were unequal), took specific action to improve it, measured impact (engagement scores, attrition, feedback). Mentions: was sustained, not a one-time gesture. Red flag: 'I bought pizza for the team' (gesture, not work) or claims credit for org-wide programs they didn't drive.",
      "followup_questions": [
        {
          "question": "How did you measure whether it actually helped?",
          "notes": "Strong: pre/post engagement scores, retention, qualitative feedback, observable behavior change. Red flag: 'people seemed happier' (no actual measurement) or only positive vibes."
        }
      ]
    },
    {
      "domain": "Leadership Principles",
      "question": "Describe a time you advocated for someone on your team who wasn't being heard.",
      "notes": "Strong: noticed someone (often more junior, often from an underrepresented group) was being talked over, having ideas attributed to others, or being passed over for opportunities. Took specific action — amplified their voice in meetings, gave them stage time, advocated in promo discussions. Bonus: developed the person as well as advocated for them. Red flag: only did this when convenient, or 'mentored' them in private without addressing the system that was failing them.",
      "followup_questions": [
        {
          "question": "What changed because of it?",
          "notes": "Strong: concrete outcome — the person was promoted, took a bigger role, contributed more visibly. Mentions ongoing advocacy, not just one moment. Red flag: 'they felt better' without observable change in their trajectory."
        }
      ]
    },
    {
      "domain": "Leadership Principles",
      "question": "Tell me about a time you addressed a team-health issue that wasn't your responsibility.",
      "notes": "Strong: noticed something off in the team (burnout, conflict, low psychological safety, exclusionary behavior) that wasn't in their direct purview, raised it appropriately, partnered with the right owner, drove action. Bonus: raised it without making the situation worse (no public airing). Red flag: gossiped about the issue, or 'addressed' it with a single email that went nowhere.",
      "followup_questions": [
        {
          "question": "How did you navigate that it wasn't formally your role?",
          "notes": "Strong: brought concerns to the actual owner (manager, HR, person involved), supported rather than took over, was an ally not a savior. Red flag: went over the owner's head, or used the situation to advance their own visibility."
        }
      ]
    },
    {
      "domain": "Leadership Principles",
      "question": "Tell me about a time you considered second-order effects of a decision before making it.",
      "notes": "Strong: noticed that a feature/decision had ripple effects beyond the immediate scope — on partners, smaller customers, ecosystem, future flexibility — and adjusted the plan accordingly. Mentions specific stakeholders they considered who weren't in the room. Bonus: caught a problem the team would have missed (e.g., feature would disadvantage smaller customers, would create lock-in, would set a precedent). Red flag: 'we considered it' generically without naming specifics.",
      "followup_questions": [
        {
          "question": "Who did you consult who wasn't directly involved?",
          "notes": "Strong: named external or adjacent stakeholders — partners, customers, security, legal, accessibility, smaller-team users. Red flag: only consulted within the team or only after the decision was made."
        }
      ]
    },
    {
      "domain": "Leadership Principles",
      "question": "Describe a time you advocated for the long-term health of a system or community over short-term wins.",
      "notes": "Strong: identified a short-term gain that came at long-term cost (unsustainable capacity, ecosystem harm, brittleness, dependence on a key person), made the case for the longer view, drove the harder path. Mentions: short-term cost was real and they bore it. Red flag: framed it as 'I always think long-term' without naming the actual short-term win they passed up.",
      "followup_questions": [
        {
          "question": "What was the short-term cost you accepted?",
          "notes": "Strong: a missed deadline, lower delivery numbers in a quarter, harder conversations with leadership, personal career visibility. Real costs paid. Red flag: 'no real cost, it was the right thing' (means they didn't actually pass up anything significant)."
        }
      ]
    },
    {
      "domain": "Leadership Principles",
      "question": "Tell me about a time you considered the impact of your work on customers, the environment, or society at large.",
      "notes": "Strong: didn't just optimize for the immediate user — considered downstream impact (data privacy, environmental footprint, fairness to all customer segments, accessibility, security of dependents). Took action to address it. Mentions: accepted some product cost to be responsible. Red flag: never considered second-order effects, or 'we hit the regulatory minimum' (compliance, not responsibility).",
      "followup_questions": [
        {
          "question": "What did you change because of those considerations?",
          "notes": "Strong: a concrete change — different default, different storage policy, retired a feature, added an opt-out, reduced energy footprint. Action, not just discussion. Red flag: 'we discussed it but kept the plan as-is' (didn't actually act)."
        }
      ]
    },
    {
      "domain": "Introduction",
      "question": "What's a problem you're proud of having solved recently? Walk me through it briefly.",
      "notes": "Warmup that lets them choose their strongest territory. Not a deep dive — listen for: do they pick something genuinely substantive, do they explain it cleanly without jargon, do they connect technical work to outcome? Sets a baseline for communication style. Use 1-2 minutes max.",
      "followup_questions": []
    },
    {
      "domain": "Systems Questions",
      "question": "Walk me through what happens, end to end, when you type a URL into a browser and press enter.",
      "notes": "Classic systems-depth probe. Strong answers cover: (1) URL parsing -> protocol, host, path. (2) DNS resolution: browser cache -> OS cache -> recursive resolver -> root -> TLD -> authoritative -> A record. (3) TCP three-way handshake to the resolved IP. (4) TLS handshake (server cert, key exchange, session keys). (5) HTTP request sent, server processes (might involve LB, CDN cache check, app server, DB). (6) Response streams back, browser parses HTML, fires off subresource requests (CSS, JS, images), executes JS, paints. Bonus: HTTP/2 multiplexing, HTTP/3 over QUIC, prefetch hints, edge caching, service workers, and how each step is observable in DevTools waterfall. Red flag: stops at 'browser sends request to server' or skips DNS/TLS entirely.",
      "followup_questions": [
        {
          "question": "Where would you optimize first if the page was slow?",
          "notes": "Strong answers say 'measure first' — open DevTools, look at the waterfall to see if it's TTFB (server slow), TLS (handshake heavy), download time (asset size), or render time (JS-heavy). Then optimize the dominant bottleneck. Mentions: Core Web Vitals (LCP, INP, CLS), CDN for asset latency, code splitting for JS, server-side caching for backend, HTTP/2 push or Early Hints. Red flag: starts optimizing before measuring."
        }
      ],
      "level_guidance": {
        "100": "Says 'DNS resolves it then the page loads'.",
        "200": "Walks the steps: parse URL → DNS lookup → TCP handshake → TLS handshake → HTTP request → server response → render. Mentions caching at multiple layers.",
        "300": "Adds depth at every step: browser cache, DNS cache + recursive resolution, TCP slow-start, TLS 1.3 0-RTT, HTTP/2 multiplexing, content negotiation, CDN edge response vs origin, render pipeline (HTML parse → DOM, CSSOM, layout, paint, composite). Discusses the critical rendering path.",
        "400": "Reasons about each stage's failure modes and optimisations: connection coalescing, QUIC/HTTP/3 over UDP collapsing the handshake, server push (deprecated) vs 103 Early Hints, edge compute (Lambda@Edge, CloudFront Functions), service worker for offline, and the deeper insight that 'fast' is a perception problem — TTFB matters less than the time-to-interactive that the user actually feels."
      }
    },
    {
      "domain": "Systems Questions",
      "question": "Explain how virtual memory works and what a page fault is.",
      "notes": "Probes OS fundamentals. Strong answers: (1) Virtual memory abstracts physical RAM behind a per-process address space; the MMU (with the TLB cache) translates virtual addresses to physical via page tables maintained by the kernel. (2) Page = unit of memory (typically 4KB on x86, can be 2MB or 1GB for huge pages). (3) Page fault = the CPU tried to access a virtual address that isn't currently mapped to physical RAM. Two kinds: minor (page is in memory but not mapped to this process — kernel just updates the table) vs major (page is on disk and must be read in — slow, microseconds to milliseconds). Heavy major-fault rate = system is thrashing on swap. Bonus: copy-on-write for fork(), memory-mapped files (mmap), shared memory, and NUMA effects on big servers. Red flag: confuses virtual memory with swap (swap is one possible backing store; virtual memory exists even with no swap).",
      "followup_questions": [
        {
          "question": "What does it mean when a system is 'thrashing'?",
          "notes": "Strong answers: working set exceeds available RAM, so the OS spends most of its time servicing page faults — pages are evicted and re-read from swap constantly. CPU appears idle (waiting on I/O), but throughput collapses. Mentions: vmstat showing high si/so columns, monitoring page-fault rate, fix is more RAM or fewer concurrent processes. Red flag: confuses with high CPU usage."
        }
      ],
      "level_guidance": {
        "100": "Knows virtual memory is 'memory that isn't really there'. May not connect it to swap.",
        "200": "Explains the abstraction: each process sees a contiguous virtual address space; the OS + MMU map pages to physical frames. A page fault happens when the referenced page isn't resident. Mentions demand paging.",
        "300": "Knows the page-table mechanics: 4KB pages by default, huge pages for TLB pressure relief, copy-on-write for fork(), mmap vs heap allocation, the cost of a TLB miss vs a real page fault (orders of magnitude different).",
        "400": "Reasons about the system implications: NUMA effects on allocation, transparent-huge-pages causing latency spikes in databases, the role of madvise/mlock for low-latency systems, kernel-bypass storage (SPDK) avoiding page-cache overhead, and the deeper insight that virtual memory is the OS's gift that lets you ignore it 99% of the time — and the 1% where you can't is where senior engineers earn their pay."
      }
    },
    {
      "domain": "GenAI",
      "question": "Tell me about a time you used a generative AI tool to significantly improve your work output. What was the task, how did you use the tool, and how did you validate the quality of the output?",
      "notes": "Assesses practical GenAI adoption and quality judgment. Strong answers: specific task (not trivial), chose appropriate tool for the job, had a clear validation strategy (human review, testing, cross-reference), measured improvement (time saved, quality delta). Probes for critical thinking -- did they blindly trust output or iterate? Red flag: can only cite trivial uses (email rewording) or shows no validation discipline.",
      "followup_questions": [
        {
          "question": "What limitations did you discover, and how did you work around them?",
          "notes": "Tests awareness of AI limitations and judgment about when AI adds value vs when it doesn't. Strong: specific examples of hallucinations caught, context window issues, or tasks where AI failed."
        },
        {
          "question": "How did you decide this was a good use case for AI vs doing it manually?",
          "notes": "Tests decision framework. Strong: considers accuracy requirements, stakes of errors, available validation methods, time investment to prompt well vs just do it."
        }
      ],
      "level_guidance": {
        "100": "Has used ChatGPT or similar for basic tasks (email drafting, summarization). Limited awareness of limitations.",
        "200": "Regularly uses GenAI tools with intentional prompt engineering. Validates output before using. Can articulate when AI helps vs hinders for their role.",
        "300": "Has integrated GenAI into team workflows. Established quality gates and validation processes. Trained others on effective use. Understands model strengths/weaknesses and chooses appropriately.",
        "400": "Has built organizational GenAI adoption strategy. Created custom evaluation frameworks, identified novel use cases, measured ROI. Understands architectural implications (latency, cost, compliance) and has influenced tooling decisions at org level."
      }
    },
    {
      "domain": "GenAI",
      "question": "Describe a situation where you chose NOT to use generative AI for a task that others suggested AI could handle. What was your reasoning?",
      "notes": "Tests judgment and critical thinking about AI applicability. Strong answers: specific scenario where stakes were high (security, compliance, customer-facing accuracy), articulated risks (hallucination in high-stakes context, data privacy, reproducibility needs), chose alternative approach with clear rationale. Shows they think about AI as a tool with trade-offs, not a universal solution. Red flag: never encountered this situation (suggests uncritical adoption) or blanket AI avoidance.",
      "followup_questions": [
        {
          "question": "What criteria do you use to decide whether a task is appropriate for AI assistance?",
          "notes": "Looking for a framework: accuracy requirements, stakes of errors, data sensitivity, available validation methods, reproducibility needs, time to prompt well vs just execute."
        }
      ],
      "level_guidance": {
        "100": "Vague answer about AI not being ready or trustworthy. No specific criteria.",
        "200": "Can name 2-3 criteria (accuracy needs, data sensitivity, stakes). Has a specific example of choosing not to use AI.",
        "300": "Has developed team-level guidelines for AI appropriateness. Considers regulatory, compliance, and customer trust dimensions. Has influenced others' AI usage patterns.",
        "400": "Has created organizational policies or frameworks for AI governance. Balances innovation velocity with risk management. Considers second-order effects (skill atrophy, over-reliance, accountability gaps)."
      }
    },
    {
      "domain": "GenAI",
      "question": "Tell me about a time you had to evaluate or improve the quality of AI-generated output before it could be used. How did you approach validation?",
      "notes": "Tests quality assurance thinking applied to AI outputs. Strong answers: specific validation methodology (cross-referencing sources, running tests, expert review, statistical sampling), understanding of failure modes (hallucination patterns, confident wrongness, subtle errors in code/data). Shows systematic approach rather than vibes-based acceptance. Red flag: 'I just read through it and it looked good' without structured validation.",
      "followup_questions": [
        {
          "question": "How would you set up a process for your team to consistently validate AI outputs at scale?",
          "notes": "Tests systems thinking. Strong: automated checks where possible (linting, unit tests for code, fact-checking pipelines), human review for judgment calls, feedback loops to improve prompts, monitoring for drift over time."
        }
      ],
      "level_guidance": {
        "100": "Reads through AI output and corrects obvious errors. No systematic approach.",
        "200": "Has a validation checklist. Cross-references key claims. Understands common failure modes for their use cases.",
        "300": "Built validation pipelines or processes for team use. Measures error rates. Has iterative prompt improvement workflow. Trains others on quality patterns.",
        "400": "Designed evaluation frameworks with quantitative metrics. Understands statistical approaches to quality measurement. Has influenced tooling/infrastructure for AI quality assurance at org scale."
      }
    },
    {
      "domain": "GenAI",
      "question": "Tell me about how you've integrated generative AI into your daily workflow. What's your most impactful use case, and how has your approach evolved over time?",
      "notes": "Tests depth of adoption and learning mindset. Strong answers: started with basic use, iterated on approach, discovered non-obvious applications, measured impact (hours saved, quality improvement, new capabilities unlocked). Shows evolution -- early prompts were naive, improved through experimentation. Can articulate their personal 'AI stack' (which tools for which tasks). Red flag: surface-level usage that hasn't evolved, or inability to quantify impact.",
      "followup_questions": [
        {
          "question": "What's a use case where AI initially seemed promising but turned out not to be worth it?",
          "notes": "Tests intellectual honesty and learning from failure. Strong: tried something, measured it wasn't actually saving time or producing good enough quality, pivoted. Shows they evaluate ROI of AI usage rather than assuming it's always better."
        }
      ],
      "level_guidance": {
        "100": "Uses AI occasionally for simple tasks. Approach hasn't changed much since starting.",
        "200": "Daily AI usage across multiple task types. Has iterated on prompting approach. Can quantify time savings.",
        "300": "Has influenced team adoption. Created shared prompts/templates. Identified novel use cases specific to their domain. Measures and reports impact.",
        "400": "Has driven organizational AI transformation. Built custom tooling or integrations. Publishes learnings. Considered change management and skill development alongside tool adoption."
      }
    },
    {
      "domain": "GenAI",
      "question": "Describe a time you helped someone else on your team become more effective with generative AI tools. What was the situation and what was your approach?",
      "notes": "Tests leadership and force-multiplication in AI adoption. Strong answers: identified someone's specific workflow that could benefit, taught them with hands-on examples relevant to THEIR work (not generic), followed up to ensure adoption stuck. Shows they understand that effective AI teaching is use-case-specific, not generic 'prompt engineering tips'. Red flag: just shared a link to a tutorial, or can't describe how the person's work actually improved.",
      "followup_questions": [
        {
          "question": "What resistance or skepticism did you encounter, and how did you address it?",
          "notes": "Tests influence skills. Strong: understood the root of skepticism (quality concerns, job security, learning curve), addressed it with evidence and low-risk experiments rather than dismissing it."
        }
      ],
      "level_guidance": {
        "100": "Showed a colleague a tool. Didn't follow up on adoption.",
        "200": "Taught someone a specific workflow with AI. Saw them adopt it. Can describe the impact on their productivity.",
        "300": "Created training materials or workshops for teams. Measured adoption metrics. Addressed organizational barriers to adoption.",
        "400": "Drove org-wide enablement strategy. Created communities of practice. Influenced tooling procurement or policy. Measured business impact of AI adoption programs."
      }
    },
    {
      "domain": "Compute Expertise",
      "question": "A customer has a containerized microservices application. Walk me through how you'd help them decide between ECS, EKS, and Lambda for their compute layer. What questions would you ask?",
      "notes": "Tests prescriptive decision-making. Discovery questions should include: team Kubernetes expertise, existing tooling/CI/CD, traffic patterns (steady vs bursty), latency requirements, cold start tolerance, cost sensitivity, vendor lock-in concerns, compliance needs. Strong mapping: ECS Fargate = simple containers without K8s overhead; EKS = team has K8s skills, needs portability, complex scheduling; Lambda = event-driven, short-duration, highly variable traffic. Red flag: recommends one without asking discovery questions, or can't articulate when NOT to use their preferred option.",
      "followup_questions": [
        {
          "question": "The customer says 'we want Kubernetes because everyone uses it.' How do you respond?",
          "notes": "Tests ability to have backbone and push back diplomatically. Strong: acknowledges their perspective, then probes for actual requirements (portability? specific K8s features? team skills?), presents alternatives with trade-offs, lets customer make informed decision rather than just validating their assumption."
        }
      ],
      "level_guidance": {
        "100": "Names the services. May have a preference but can't articulate clear selection criteria.",
        "200": "Knows the basic differences. Can articulate 2-3 decision criteria (team skills, traffic pattern, latency). Makes reasonable recommendations for simple scenarios.",
        "300": "Asks detailed discovery questions before recommending. Considers cost modeling, operational overhead, team growth trajectory. Can design hybrid architectures (Lambda for some services, ECS for others). Discusses migration path.",
        "400": "Reasons about organizational implications: platform team investment, developer experience, observability strategy, blast radius, multi-region complexity. Considers 2-year trajectory (team growing, services splitting). Knows edge cases where conventional wisdom breaks down."
      }
    },
    {
      "domain": "Database Expertise",
      "question": "A customer needs to store user session data with sub-millisecond reads at scale. They're considering DynamoDB, ElastiCache (Redis), and Aurora. How do you guide this decision?",
      "notes": "Tests ability to match data access patterns to the right service. Key considerations: data size per session, read/write ratio, TTL requirements, consistency needs, query patterns beyond key lookup, durability requirements, cost at scale. Strong mapping: ElastiCache = pure speed + TTL + ephemeral OK; DynamoDB = durable + fast + scales infinitely + TTL built-in; Aurora = need complex queries across sessions or joins with other data. The 'right' answer depends on requirements they should probe for. Red flag: jumps to one answer without asking about durability, access patterns, or what else they need to query.",
      "followup_questions": [
        {
          "question": "What if they also need to run analytics on session data -- how does that change the architecture?",
          "notes": "Tests thinking beyond the immediate ask. Strong: add a streaming/CDC pattern (DynamoDB Streams to Kinesis to S3/Redshift, or ElastiCache + write-through to a durable store). Shows they think about operational analytics without compromising the hot path."
        }
      ],
      "level_guidance": {
        "100": "Suggests one service without discovery. May not understand the latency characteristics of each.",
        "200": "Knows Redis is fastest, DynamoDB is durable and fast, Aurora adds SQL capability. Can match basic requirements to services.",
        "300": "Asks about durability, access patterns, query complexity, cost constraints. Designs tiered architecture (cache in front of durable store). Considers DAX as DynamoDB accelerator. Discusses capacity planning.",
        "400": "Reasons about failure modes (what happens when cache is cold?), cost modeling at scale (DynamoDB on-demand vs provisioned at 100K RPS), data modeling implications (single-table DynamoDB design vs normalized Aurora), and operational complexity trade-offs."
      }
    },
    {
      "domain": "Application Development Expertise",
      "question": "A customer wants to build an event-driven architecture. They've heard about SQS, SNS, EventBridge, and Kinesis. How do you help them understand which to use where?",
      "notes": "Tests messaging/eventing service selection -- a common SA conversation. Key differentiators: SQS = point-to-point, pull-based, at-least-once, decoupling producer/consumer; SNS = fan-out pub/sub, push-based, multiple subscribers; EventBridge = event routing with rules/filtering, schema registry, cross-account, SaaS integration; Kinesis = ordered stream processing, replay capability, multiple consumers reading same stream, real-time analytics. Strong answers: asks about ordering requirements, replay needs, number of consumers, throughput, event complexity, and designs a combination (e.g., EventBridge for routing + SQS for buffering + Kinesis for analytics stream).",
      "followup_questions": [
        {
          "question": "When would you use Step Functions instead of or alongside these?",
          "notes": "Tests orchestration vs choreography understanding. Step Functions: when you need coordination, error handling, human approval, or complex workflows with branching logic. Alongside: Step Functions orchestrates a workflow that publishes to EventBridge/SQS at specific steps."
        }
      ],
      "level_guidance": {
        "100": "Knows SQS is a queue and SNS is pub/sub. May not distinguish EventBridge from SNS clearly.",
        "200": "Can articulate the primary use case for each. Knows SQS+SNS fanout pattern. Understands FIFO vs standard trade-offs.",
        "300": "Designs composite architectures using multiple services together. Understands ordering guarantees, exactly-once processing (SQS FIFO + dedup), EventBridge rules for routing, Kinesis for replay/analytics. Considers DLQ strategies and error handling.",
        "400": "Reasons about event-driven architecture at organizational scale: event schema evolution, cross-team contracts, eventual consistency implications, saga patterns for distributed transactions, observability of event flows, and the operational cost of debugging async systems."
      }
    },
    {
      "domain": "Security",
      "question": "A customer asks: 'Should we use IAM roles, resource-based policies, or both? When do I need SCPs vs permission boundaries?' Walk me through how you'd explain the AWS authorization model.",
      "notes": "Tests depth of IAM understanding -- one of the most important SA skills. Key framework: Identity-based (IAM policies on principal) vs Resource-based (policies on the resource) vs Permission boundaries (max permissions ceiling) vs SCPs (org-wide guardrails). When to use what: IAM roles = default for granting access; resource-based = cross-account without assuming role, or when resource owner controls access; permission boundaries = delegated admin (let teams create roles but cap their permissions); SCPs = preventive guardrails across accounts. Strong answers: explain the policy evaluation logic (explicit deny > explicit allow, and all applicable policies must allow). Red flag: conflates SCPs with IAM policies, or doesn't understand permission boundaries.",
      "followup_questions": [
        {
          "question": "A developer says their Lambda can't access an S3 bucket in another account. Walk me through how you'd troubleshoot this.",
          "notes": "Tests systematic debugging. Strong: checks Lambda execution role (identity-based), S3 bucket policy (resource-based), any SCPs on either account, VPC endpoint policies if applicable, KMS key policy if bucket is encrypted. Knows that cross-account requires BOTH sides to allow."
        }
      ],
      "level_guidance": {
        "100": "Knows IAM roles grant permissions. May not distinguish resource-based policies from identity-based.",
        "200": "Understands identity vs resource-based policies. Knows cross-account patterns. Can explain SCPs at a high level.",
        "300": "Explains policy evaluation logic clearly. Designs multi-account permission strategies. Uses permission boundaries for delegation. Implements least-privilege with Access Analyzer.",
        "400": "Reasons about authorization at scale: thousands of accounts, hundreds of teams, policy-as-code pipelines, automated remediation, custom authorization engines for application-layer decisions, and the organizational design implications of different IAM architectures."
      }
    },
    {
      "domain": "Storage Expertise",
      "question": "A customer has 500TB of data. Some is accessed daily, some monthly, some maybe never again but must be retained for compliance. Design their storage strategy on AWS.",
      "notes": "Tests data lifecycle and cost optimization thinking. Strong answers start with discovery: access patterns per data category, retrieval latency requirements, compliance retention periods, query patterns (do they need to search archived data?). Then design tiered strategy: S3 Standard for hot data, S3 IA/One Zone-IA for warm, Glacier Instant Retrieval/Flexible Retrieval/Deep Archive for cold (matched to retrieval SLA). Implement S3 Lifecycle policies for automated tiering. Consider S3 Intelligent-Tiering for unpredictable access. For compliance: Object Lock (WORM), versioning, cross-region replication for DR. Red flag: puts everything in one tier, or doesn't ask about access patterns first.",
      "followup_questions": [
        {
          "question": "How would you help them estimate monthly cost, and what surprises should they watch for?",
          "notes": "Tests practical cost knowledge. Surprises: retrieval fees on Glacier (can be expensive at scale), S3 request costs (many small objects = high cost), data transfer out charges, early deletion fees on IA/Glacier, lifecycle transition costs. Strong: suggests S3 Storage Lens for visibility."
        }
      ],
      "level_guidance": {
        "100": "Suggests S3. May know about Glacier but doesn't design a tiered strategy.",
        "200": "Designs basic tiering (hot/warm/cold). Knows lifecycle policies exist. Can name the storage classes.",
        "300": "Designs comprehensive lifecycle with specific class selection based on access SLAs. Considers Object Lock for compliance, cross-region replication for DR, Intelligent-Tiering for uncertain patterns. Estimates costs.",
        "400": "Reasons about data management at scale: metadata catalog (Glue Data Catalog), searchability of archived data (Athena over Glacier-restored objects vs maintaining search indices), organizational data governance, cost anomaly detection, and multi-cloud/hybrid considerations for data gravity."
      }
    },
    {
      "domain": "Network Expertise",
      "question": "A customer with 15 AWS accounts needs them all to communicate with each other and with their on-premises datacenter. Walk me through the networking architecture options and trade-offs.",
      "notes": "Tests multi-account networking -- a common and complex SA design area. Options: Transit Gateway (hub-and-spoke, scalable, supports routing policies), VPC Peering (point-to-point, no transitive routing, limited at scale), PrivateLink (service-specific, no network overlap concerns). On-prem connectivity: Direct Connect (dedicated, consistent latency, 1/10/100 Gbps) vs Site-to-Site VPN (encrypted over internet, cheaper, less reliable). Strong design: Transit Gateway as the hub, Direct Connect for on-prem with VPN backup, route tables for segmentation (prod/dev separation), shared services VPC pattern. Red flag: suggests VPC peering for 15 accounts (doesn't scale) or forgets about routing segmentation.",
      "followup_questions": [
        {
          "question": "How do you handle the case where two of those accounts have overlapping CIDR ranges?",
          "notes": "Tests real-world problem-solving. Options: PrivateLink (avoids network-layer overlap entirely), NAT at the Transit Gateway (complex), re-IP one VPC (painful but clean), or application-layer proxying. Best answer acknowledges this is a common pain point and discusses prevention (IP address management strategy) alongside remediation."
        }
      ],
      "level_guidance": {
        "100": "Knows VPC Peering exists. May not understand Transit Gateway or its purpose.",
        "200": "Knows Transit Gateway vs VPC Peering trade-offs. Can design basic hub-and-spoke. Understands Direct Connect at a high level.",
        "300": "Designs segmented Transit Gateway with multiple route tables (prod/nonprod/shared). Plans Direct Connect with VPN failover. Considers DNS resolution (Route 53 Resolver), centralized egress, and inspection (Network Firewall). Addresses IP planning.",
        "400": "Reasons about networking at enterprise scale: multi-region Transit Gateway peering, IPAM for address governance, network observability (VPC Flow Logs + Traffic Mirroring), latency-sensitive routing decisions, cost optimization (data transfer between AZs/regions), and organizational network team operating model."
      }
    },
    {
      "domain": "Infrastructure Expertise",
      "question": "A customer's AWS bill has grown 40% quarter-over-quarter and they don't understand why. How do you approach this conversation, and what would you investigate?",
      "notes": "Tests structured cost analysis approach. Strong: (1) Don't panic -- understand if growth is aligned with business growth. (2) Use Cost Explorer to identify top cost drivers by service, region, usage type. (3) Check for common waste: idle resources (unused EBS, unattached EIPs, oversized instances), missing Reserved Instances/Savings Plans coverage, data transfer costs, dev/test environments running 24/7. (4) Look at cost allocation tags to map spend to business units. (5) Recommend ongoing governance: budgets with alerts, AWS Organizations consolidated billing, regular rightsizing reviews. Red flag: jumps to 'buy Reserved Instances' without investigation.",
      "followup_questions": [
        {
          "question": "They say 'just make it cheaper' -- how do you prioritize optimization efforts?",
          "notes": "Tests prioritization framework. Strong: Pareto principle -- find the top 3 cost drivers first. Quick wins (scheduling dev/test, rightsizing obvious outliers) vs strategic changes (architecture refactoring, Graviton migration, spot/serverless). Always quantify savings potential before effort."
        }
      ],
      "level_guidance": {
        "100": "Suggests turning things off or buying RIs. No structured investigation approach.",
        "200": "Uses Cost Explorer effectively. Identifies common waste categories. Knows RI/SP basics and rightsizing concepts.",
        "300": "Builds cost optimization practice: tagging strategy, showback/chargeback, automated rightsizing recommendations, Trusted Advisor integration, scheduled scaling. Quantifies recommendations.",
        "400": "Designs organizational FinOps practice: unit economics (cost per transaction/user), architectural patterns for cost efficiency (serverless, spot, multi-tier), engineering culture changes (cost-aware development), and trade-off analysis between performance/reliability/cost."
      }
    },
    {
      "domain": "Infrastructure Expertise",
      "question": "A customer has 200 servers on-premises and wants to migrate to AWS. They have a 12-month deadline. Walk me through your approach.",
      "notes": "Tests migration methodology knowledge and project planning. Strong approach: (1) Discover & Assess -- inventory with Application Discovery Service or partner tools, map dependencies, categorize by 7Rs (Rehost, Replatform, Refactor, Repurchase, Retire, Retain, Relocate). (2) Plan -- prioritize waves by complexity/dependency/business criticality, establish landing zone (Control Tower), set up connectivity (Direct Connect). (3) Migrate -- start with low-risk wave to build confidence, use MGN (Application Migration Service) for lift-and-shift, iterate. (4) Optimize -- rightsize post-migration, modernize in subsequent phases. Key insight: don't try to refactor everything during migration -- rehost first, optimize later. Red flag: wants to re-architect everything (misses the deadline) or lift-and-shifts everything without assessment.",
      "followup_questions": [
        {
          "question": "What are the biggest risks in a migration like this, and how do you mitigate them?",
          "notes": "Tests experience with real migration challenges. Risks: application dependencies not mapped (discovery gap), performance degradation post-migration (network latency to on-prem dependencies), license compliance (Windows/Oracle), data migration window too small for volume, team skills gap. Mitigations: thorough discovery, pilot migration, runbooks, rollback plan, training."
        }
      ],
      "level_guidance": {
        "100": "Knows 'lift and shift' concept. May not have a structured methodology.",
        "200": "Knows the 7Rs framework. Can plan a basic migration wave. Understands MGN/DMS at a high level.",
        "300": "Designs migration program: wave planning, landing zone setup, dependency mapping, cutover strategy, rollback procedures. Has opinions on common pitfalls. Considers organizational change management.",
        "400": "Has led or architected large-scale migrations. Reasons about migration factory patterns, automated discovery and wave planning, organizational readiness, commercial/licensing strategy, and post-migration optimization roadmap as a single program."
      }
    },
    {
      "domain": "Infrastructure Expertise",
      "question": "You're reviewing a customer's architecture. Which Well-Architected Framework pillar would you start with, and how do you structure the review conversation?",
      "notes": "Tests WAF knowledge and consulting skills. Six pillars: Operational Excellence, Security, Reliability, Performance Efficiency, Cost Optimization, Sustainability. Strong answers: depends on customer context (start with their pain point), but Security is non-negotiable. Describe the review process: understand current state, walk through pillar-specific questions, identify high-risk items, prioritize remediation by impact and effort. Mention the AWS Well-Architected Tool for tracking. Key insight: a WAR is a conversation, not an audit -- it should generate trust and actionable improvements. Red flag: lists pillars without explaining how to conduct the review or can't prioritize.",
      "followup_questions": [
        {
          "question": "A customer pushes back and says 'we don't have time for a review, just tell us what's wrong.' How do you handle this?",
          "notes": "Tests consulting and influence skills. Strong: empathizes with their time pressure, offers a focused review on their highest-risk area (usually security or reliability), explains the value proposition (finding a $100K risk in a 2-hour conversation), suggests asynchronous review where they fill out portions."
        }
      ],
      "level_guidance": {
        "100": "Can name some pillars. May not understand the review process.",
        "200": "Knows all 6 pillars and their focus areas. Can conduct a basic review using the framework questions. Uses the WA Tool.",
        "300": "Tailors reviews to customer context and maturity. Prioritizes findings by business impact. Creates actionable remediation plans with effort/impact matrix. Has conducted multiple reviews.",
        "400": "Uses WAF as a strategic engagement tool. Connects architectural improvements to business outcomes. Designs custom lenses for specific industries/workloads. Builds organizational review programs (quarterly cadence, team training, embedding WAF into development lifecycle)."
      }
    },
    {
      "domain": "Performance Troubleshooting/Tuning",
      "question": "A customer says 'our application is slow.' Walk me through how you'd systematically diagnose and resolve this.",
      "notes": "Tests structured performance troubleshooting. Strong approach: (1) Define 'slow' -- which operations, for which users, since when, what's acceptable? Establish baseline metrics. (2) Identify the bottleneck layer: frontend (rendering, JS bundle size), network (latency, DNS, TLS), backend (compute, I/O), database (query performance, connection pooling), external dependencies. (3) Instrument: CloudWatch metrics, X-Ray traces, RUM (Real User Monitoring), APM tools. (4) Isolate: is it all requests or specific patterns (geo, time of day, request type)? (5) Fix in priority order: quick wins (caching, CDN, connection pooling) before architecture changes. Red flag: jumps to 'add more instances' without diagnosing.",
      "followup_questions": [
        {
          "question": "You discover the database is the bottleneck -- queries are taking 2-5 seconds. What do you do?",
          "notes": "Tests database performance knowledge. Strong: enable slow query log, analyze execution plans (EXPLAIN), check for missing indexes, look at connection count vs pool size, consider read replicas for read-heavy patterns, evaluate caching layer (ElastiCache/DAX), check if table design matches access patterns, consider query optimization before throwing hardware at it."
        }
      ],
      "level_guidance": {
        "100": "Suggests adding more resources without diagnosing. No systematic approach.",
        "200": "Asks clarifying questions about symptoms. Knows basic monitoring tools. Can identify common bottlenecks (CPU, memory, I/O, network).",
        "300": "Systematic methodology: measure, identify bottleneck layer, instrument with X-Ray/APM, isolate root cause, fix in priority order. Considers caching strategies, async processing, architectural changes.",
        "400": "Reasons about performance as a system property: load testing methodology, capacity planning, performance budgets in CI/CD, SLO-driven alerting, and the economic trade-offs between performance investment and business impact. Understands tail latency, queue theory, and cascading failure patterns."
      }
    },
    {
      "domain": "GenAI",
      "question": "A customer wants to build an AI agent that can take actions on their behalf -- booking meetings, querying databases, calling APIs. Walk me through how you'd architect this on AWS and what guardrails you'd put in place.",
      "notes": "Tests understanding of agentic AI architecture -- THE hot topic in 2025-2026. Strong answers: Bedrock Agents for orchestration (action groups define what the agent can do), knowledge bases for RAG context, guardrails for content filtering and topic avoidance, human-in-the-loop for high-stakes actions, session management for multi-turn, tracing for debugging agent decisions. Key guardrails: tool permission boundaries (agent can only call approved APIs), output validation, cost controls (max iterations), PII filtering. Red flag: treats agent as a simple chatbot, doesn't mention guardrails, or can't explain how the agent decides which tool to use (planning/reasoning loop).",
      "followup_questions": [
        {
          "question": "How do you test an agentic system before production? What failure modes worry you?",
          "notes": "Tests production readiness thinking. Strong: simulation testing (mock tools), adversarial testing (jailbreak attempts), trace analysis (did agent take expected reasoning path), hallucination detection on tool parameter generation, graceful degradation when tools fail, cost runaway prevention. Failure modes: infinite loops, incorrect tool parameters, cascading errors, prompt injection through user input."
        }
      ],
      "level_guidance": {
        "100": "Knows AI agents exist and can 'do things'. Can't explain architecture.",
        "200": "Understands agent = LLM + tools/actions + planning loop. Knows Bedrock Agents at a high level. Can describe basic action groups and knowledge bases.",
        "300": "Designs production agent: action group boundaries with IAM-scoped permissions, Bedrock Guardrails (content filters, denied topics, PII redaction), session state management, observability (trace each step), human-in-the-loop for destructive actions, error handling and fallback responses.",
        "400": "Reasons about agentic AI at organizational scale: multi-agent orchestration patterns, agent evaluation frameworks (not just output quality but action correctness), security implications (agent acts with delegated authority -- supply chain risk), cost modeling (each reasoning step = tokens), governance (who approves new agent capabilities), and the philosophical question of appropriate autonomy levels for different action types."
      }
    },
    {
      "domain": "GenAI",
      "question": "What's the difference between Amazon Bedrock Agents, AgentCore, and building a custom agent framework? When would you recommend each?",
      "notes": "Tests current AWS agentic landscape knowledge. Bedrock Agents: fully managed, declarative (define action groups + knowledge bases, AWS handles orchestration), fastest time-to-value, limited customization. AgentCore: infrastructure layer for running ANY agent framework (LangGraph, CrewAI, custom) in serverless microVMs with session isolation, 8+ hour async support -- choose when you need framework flexibility with AWS-managed infrastructure. Custom: full control, maximum flexibility, but you own orchestration, state management, scaling, and security. Decision framework: start with Bedrock Agents if it fits, graduate to AgentCore when you need custom frameworks, go fully custom only if you have specific requirements neither can meet. Red flag: doesn't know AgentCore exists (launched 2025), or can't articulate when managed vs custom is appropriate.",
      "followup_questions": [
        {
          "question": "A customer says they're building with LangChain and want to deploy on AWS. What do you recommend?",
          "notes": "Tests practical recommendation. Strong: AgentCore is the natural fit (run LangChain/LangGraph in managed microVMs, get session isolation, scaling, and observability without re-architecting). Alternative: containerized on ECS/EKS if they need more control. Discuss trade-offs: managed convenience vs operational control."
        }
      ],
      "level_guidance": {
        "100": "May know Bedrock Agents. Likely doesn't know AgentCore.",
        "200": "Knows Bedrock Agents (managed, declarative) vs custom (build your own). May know AgentCore exists. Can recommend Bedrock Agents for simple use cases.",
        "300": "Clear framework: Bedrock Agents for standard patterns (RAG + tools), AgentCore for framework-agnostic deployment with AWS infrastructure benefits, custom for edge cases. Considers: session management, multi-turn state, tool latency, cost at scale.",
        "400": "Strategic reasoning: the agentic platform is a spectrum of abstraction (higher = faster but less flexible), organizational agent platform strategy (standardize on one layer vs mixed), the emerging patterns (agent-to-agent communication, shared memory, agent marketplaces), and how to advise customers whose requirements will evolve as the space matures."
      }
    },
    {
      "domain": "Infrastructure Expertise",
      "question": "A customer says 'we have monitoring but we still get surprised by outages.' How would you assess their observability posture and what would you recommend to improve it?",
      "notes": "Tests observability thinking beyond basic monitoring. Key distinction: monitoring tells you WHEN something is wrong; observability helps you understand WHY. The three pillars: metrics (CloudWatch -- what's happening numerically), logs (CloudWatch Logs -- what happened in detail), traces (X-Ray -- how requests flow through the system). Strong assessment approach: (1) Do they have all three pillars? (2) Are they correlated? (3) Do they have alerting on symptoms (customer impact) not just causes (CPU)? (4) Do they have dashboards for each service? (5) Do they practice runbooks/playbooks? (6) Do they test their alerting? Red flag: just suggests 'add more alarms' without understanding the observability maturity model.",
      "followup_questions": [
        {
          "question": "How do you instrument a microservices application for distributed tracing?",
          "notes": "Tests hands-on distributed systems knowledge. Strong: X-Ray SDK or OpenTelemetry auto-instrumentation, propagate trace context (headers) across service boundaries, sample strategically (not 100% in production -- cost), instrument both synchronous (HTTP) and asynchronous (SQS, EventBridge) paths, create service maps for dependency visualization."
        }
      ],
      "level_guidance": {
        "100": "Suggests adding more CloudWatch alarms. No systematic approach.",
        "200": "Knows the three pillars (metrics, logs, traces). Can set up basic CloudWatch dashboards and alarms. Understands X-Ray at a high level.",
        "300": "Designs observability platform: structured logging with correlation IDs, distributed tracing across services (X-Ray/OTEL), SLO-based alerting (alert on customer impact not infrastructure metrics), runbooks linked to alarms, synthetic monitoring (CloudWatch Synthetics) for proactive detection, cost-aware instrumentation strategy.",
        "400": "Observability as an engineering practice: the cultural shift from monitoring to observability (explore unknown-unknowns), SLO/SLI/error budget framework, observability-driven development (instrument before ship), the cost-quality tradeoff of telemetry collection, AIOps for anomaly detection, and designing systems to be intrinsically observable (structured events, not just logs)."
      }
    },
    {
      "domain": "Security",
      "question": "A customer is moving from a single AWS account to a multi-account strategy. They have 5 development teams. Design their account structure and explain your reasoning.",
      "notes": "Tests AWS Organizations and multi-account architecture -- a critical SA skill. Strong design: organizational units (OU) for workload types (Prod, Dev, Sandbox, Security, Shared Services), separate accounts per team per environment (Team-A-Prod, Team-A-Dev), centralized security account (GuardDuty, Security Hub aggregation), centralized networking account (Transit Gateway, DNS), log archive account (CloudTrail, Config). Key principles: blast radius reduction (compromise one account, not all), billing isolation, permission boundaries per OU (SCPs), Control Tower for governance. Red flag: suggests one account per team without environment separation, or doesn't mention SCPs/guardrails.",
      "followup_questions": [
        {
          "question": "How do you handle shared resources like a container registry or CI/CD tooling across these accounts?",
          "notes": "Tests practical multi-account patterns. Strong: shared services account with cross-account access (ECR repository policies, CodePipeline cross-account deploy roles), RAM for sharing VPC subnets or Transit Gateway, centralized artifact management with IAM cross-account roles following least-privilege."
        }
      ],
      "level_guidance": {
        "100": "Suggests one account per team. No structural reasoning.",
        "200": "Knows AWS Organizations, accounts per environment, basic OUs. Can set up Control Tower. Understands SCPs conceptually.",
        "300": "Designs production multi-account: OU hierarchy with SCP guardrails, centralized security tooling, shared networking (Transit Gateway hub), log aggregation, CI/CD cross-account deployment patterns, SSO/Identity Center for access management, cost allocation with tags and consolidated billing.",
        "400": "Multi-account at enterprise scale: account vending automation (Account Factory), custom Control Tower controls, organizational governance model (central platform team + federated teams), account lifecycle management (provisioning, decommissioning), the tension between standardization and team autonomy, and exception management for teams that need non-standard configurations."
      }
    },
    {
      "domain": "Infrastructure Expertise",
      "question": "Design a highly available architecture for a customer's e-commerce platform that needs to survive an entire AWS region going down. What are the trade-offs?",
      "notes": "Tests multi-region architecture thinking. Key components: Route 53 health-checked failover (or latency-based routing), data replication strategy (Aurora Global Database or DynamoDB Global Tables for active-active; S3 cross-region replication for objects), stateless application tier (session externalized to ElastiCache Global Datastore or DynamoDB), infrastructure-as-code for consistent deployment across regions, DNS TTL considerations. Trade-offs: cost (running full stack in 2+ regions), complexity (deployment coordination, data consistency), latency (cross-region replication lag), operational overhead (runbooks for failover). Strong answers: active-active is most resilient but most complex/expensive; pilot light or warm standby may be sufficient depending on RTO requirements. Red flag: doesn't discuss data consistency challenges or cost implications.",
      "followup_questions": [
        {
          "question": "How do you test this before an actual disaster? What could go wrong during failover?",
          "notes": "Tests operational maturity. Strong: game days (simulate region failure), chaos engineering (Fault Injection Simulator), regular failover drills, automated vs manual failover decision, things that go wrong: DNS propagation delay, stale caches, split-brain during network partition, database replication lag causing data loss, services with regional dependencies you forgot about."
        }
      ],
      "level_guidance": {
        "100": "Mentions 'use multiple regions' without architecture details.",
        "200": "Designs basic multi-AZ within a region. Knows Route 53 failover and Aurora read replicas. May struggle with true multi-region design.",
        "300": "Designs multi-region: Aurora Global Database (RPO < 1s, RTO < 1min), DynamoDB Global Tables for active-active, Global Accelerator for network layer, stateless compute in both regions, automated failover with health checks, infrastructure consistency via IaC.",
        "400": "Reasons about resilience as a business decision: cost modeling for different DR tiers, the diminishing returns of more 9s, testing as the only validation (untested DR = no DR), cell-based architecture for blast radius reduction, the organizational model (SRE team, on-call rotation, incident management), and the insight that most outages are caused by changes not hardware failures (so deployment safety > multi-region in many cases)."
      }
    },
    {
      "domain": "Application Development Expertise",
      "question": "A customer wants to go 'fully serverless.' Walk me through the benefits, limitations, and when you'd push back on that goal.",
      "notes": "Tests nuanced serverless thinking -- not just cheerleading but understanding real trade-offs. Benefits: no infrastructure management, pay-per-use, auto-scaling to zero, faster time-to-market, reduced operational burden. Limitations: cold starts (latency-sensitive workloads), execution duration limits (15 min Lambda), vendor lock-in concerns, debugging complexity (distributed by nature), cost at sustained high scale (may be cheaper with containers), state management challenges. When to push back: sustained high-throughput workloads (cost crossover), latency-sensitive hot paths, long-running processes, workloads needing specialized hardware (GPU). Strong answers acknowledge serverless is a spectrum (Lambda, Fargate, Aurora Serverless, DynamoDB on-demand) not just Lambda. Red flag: either 'serverless for everything!' evangelism or 'it doesn't work for real workloads' dismissal.",
      "followup_questions": [
        {
          "question": "How do you handle a saga pattern (distributed transaction) in a serverless architecture?",
          "notes": "Tests advanced serverless patterns. Strong: Step Functions for orchestrating the saga (compensating transactions on failure), each step is a Lambda, DynamoDB for state, EventBridge for events between bounded contexts. Discusses: idempotency at each step, timeout handling, dead letter queues for failed compensations, observability of the saga flow."
        }
      ],
      "level_guidance": {
        "100": "Knows Lambda is serverless. Limited understanding of the broader serverless ecosystem.",
        "200": "Names the serverless stack (Lambda, API Gateway, DynamoDB, S3, Step Functions). Understands basic trade-offs (cold starts, duration limits). Can build simple serverless applications.",
        "300": "Designs complex serverless architectures: event-driven patterns, saga orchestration with Step Functions, async processing (SQS + Lambda), cost modeling at scale, performance optimization (provisioned concurrency, SnapStart), testing strategies (local development challenges).",
        "400": "Serverless as architecture philosophy: the spectrum from 'functions' to 'managed services' (choosing the highest level of abstraction that meets requirements), organizational implications (no ops team?), the cost crossover point analysis, vendor lock-in mitigation strategies, and when hybrid (some serverless, some containers) is the pragmatic choice for large organizations."
      }
    },
    {
      "domain": "Security",
      "question": "A customer in a regulated industry (healthcare/financial services) says they need to keep all data in Canada and meet strict compliance requirements. How do you architect for this?",
      "notes": "Tests compliance-aware architecture -- critical for Canadian SA work. Key considerations: (1) Data residency -- use ca-central-1 region, understand which services are available there (not all are). (2) Compliance frameworks -- map to PIPEDA/PHIPA (healthcare), OSFI (financial), SOC2, HITRUST. (3) Encryption -- KMS keys in ca-central-1, customer-managed keys for control. (4) Access control -- ensure no data leaks to other regions (S3 bucket policies, SCP to deny non-Canadian regions, VPC endpoints to prevent internet transit). (5) Audit -- CloudTrail, Config rules, Security Hub for continuous compliance monitoring. (6) Backup/DR -- if DR is needed in another region, can they replicate to another Canadian region? (AWS has ca-central-1 and ca-west-1). Red flag: forgets about service availability in Canadian regions, or doesn't mention data residency controls (SCPs, S3 policies).",
      "followup_questions": [
        {
          "question": "The customer also needs disaster recovery but can't send data outside Canada. What are their options?",
          "notes": "Tests knowledge of Canadian AWS regions. Strong: ca-central-1 (Montreal) + ca-west-1 (Calgary) for in-country DR, cross-region replication within Canada, discusses service parity between the two regions, and acknowledges ca-west-1 is newer with fewer services. Alternative: multi-AZ within ca-central-1 for HA without DR to another region."
        }
      ],
      "level_guidance": {
        "100": "Knows 'use the Canadian region'. Limited compliance depth.",
        "200": "Understands data residency requirements, knows ca-central-1, basic encryption with KMS, can map to common compliance frameworks. Knows not all services are available in all regions.",
        "300": "Designs compliant architecture: region-locking via SCPs, encryption strategy (CMK in-region), audit logging to in-region destinations, service availability assessment for the Canadian region, compliance automation (Config Rules, Security Hub), DR options within Canada (ca-west-1).",
        "400": "Compliance as architecture: the tension between innovation velocity and compliance overhead, automated compliance-as-code (preventive + detective controls), multi-framework mapping (one architecture satisfying multiple compliance standards), the emerging regulatory landscape (AI regulation, DORA-like operational resilience), and advising customers on compliance-by-design vs compliance-as-afterthought."
      }
    },
    {
      "domain": "Infrastructure Expertise",
      "question": "A customer deploys changes manually via the AWS Console and has no infrastructure-as-code. They're experiencing configuration drift and outages from undocumented changes. How do you help them mature their operations?",
      "notes": "Tests operational maturity and IaC thinking. Strong approach: (1) Assess current state -- what's deployed, how many resources, what's the change cadence. (2) Start with IaC for NEW resources (don't try to import everything at once). Tools: CloudFormation, CDK, or Terraform (discuss trade-offs). (3) Add guardrails: SCPs to prevent console changes to production, Config rules to detect drift, CloudTrail for audit. (4) Implement CI/CD for infrastructure (CodePipeline or GitHub Actions deploying CloudFormation). (5) Gradual import of existing resources (CloudFormation import, or recreate in IaC during next change). Key insight: this is a people/process change, not just a tooling change -- need buy-in, training, and gradual adoption. Red flag: suggests importing everything into Terraform on day 1 (too risky), or doesn't acknowledge the human/process dimension.",
      "followup_questions": [
        {
          "question": "They ask 'CloudFormation vs Terraform vs CDK -- which should we use?' What's your recommendation?",
          "notes": "Tests practical recommendation skills. Framework: CloudFormation if all-in on AWS and want native support/no third-party dependency; Terraform if multi-cloud or team already knows it; CDK if development team prefers programming languages over YAML/HCL and wants higher-level abstractions. Discuss: state management (Terraform state file risks), ecosystem maturity, hiring market, community modules."
        }
      ],
      "level_guidance": {
        "100": "Suggests 'use CloudFormation'. No adoption strategy.",
        "200": "Knows IaC benefits (reproducibility, version control, consistency). Can use CloudFormation or Terraform for basic deployments. Understands drift concept.",
        "300": "Designs IaC adoption strategy: incremental approach (new resources first), CI/CD for infrastructure, drift detection (Config), access controls to prevent console changes, module/construct libraries for standardization, testing (cfn-lint, checkov).",
        "400": "Operational excellence as an organizational capability: IaC as one dimension of a maturity model, platform engineering (internal developer platform with guardrails), GitOps patterns, policy-as-code (OPA, CloudFormation Guard), and the organizational design that makes good operations sustainable (platform team, golden paths, paved roads)."
      }
    }
  ]
}