DevOps Engineer Interview Questions

Direct Answer

A process is a running program with its own identifier and resources. Signals request actions such as graceful termination, reload, interruption, or forced termination.

Detailed Explanation

# Inspect the service and recent logs.
systemctl status resumeloop-api.service
journalctl   --unit resumeloop-api.service   --since "15 minutes ago"

# Ask the service manager to stop it gracefully.
sudo systemctl stop resumeloop-api.service

# Inspect an unmanaged process.
ps -fp 2481
cat /proc/2481/status
ls -l /proc/2481/fd

# Request graceful termination.
kill -TERM 2481

# Force termination only after investigation
# and an appropriate grace period.
kill -KILL 2481

Common Interview Pitfalls

• Sending SIGKILL before attempting a graceful service shutdown.
• Killing a process without confirming its PID and ownership.
• Assuming every application interprets SIGHUP as a configuration reload.
• Restarting a failing service repeatedly without reviewing logs or dependencies.
• Managing a systemd service by killing its child process directly.
• Ignoring active requests and cleanup requirements during termination.

Direct Answer

Ownership and permission bits control filesystem access, while file descriptors are per-process references to open files, sockets, pipes, and other I/O resources.

Detailed Explanation

# Inspect ownership and permissions.
ls -ld /opt/resumeloop
ls -l /opt/resumeloop/config
namei -l /opt/resumeloop/config/app.env

# Inspect the service identity.
systemctl show   resumeloop-api.service   --property=User   --property=Group

# Inspect open descriptors.
ls /proc/2481/fd | wc -l
lsof -p 2481

# Inspect process limits.
cat /proc/2481/limits

# Apply narrow ownership and permissions.
sudo chown root:resumeloop   /opt/resumeloop/config/app.env

sudo chmod 640   /opt/resumeloop/config/app.env

Common Interview Pitfalls

• Using chmod 777 instead of correcting ownership and required access.
• Checking only the target file while ignoring permissions on parent directories.
• Assuming an application runs as the interactive shell user.
• Ignoring leaked file descriptors during connection failures.
• Making private keys or environment files readable by every user.
• Changing permissions without considering container or service-manager identities.

Direct Answer

IP identifies hosts, ports identify transport endpoints, sockets combine addressing and protocol state, TCP provides reliable streams, and UDP sends independent datagrams.

Detailed Explanation

# Show listening TCP and UDP sockets.
ss -lntup

# Confirm the service binding.
ss -lntp '( sport = :3000 )'

# Inspect local addresses and routes.
ip address show
ip route show

# Test transport connectivity.
nc -vz api.example.com 443

# Test the application and TLS path.
curl   --verbose   --connect-timeout 5   https://api.example.com/health

# Test the application directly on the host.
curl --verbose http://127.0.0.1:3000/health

Common Interview Pitfalls

• Assuming an open TCP port proves that the application is healthy.
• Binding a service only to loopback when remote traffic must reach it.
• Testing only with ping and concluding that every TCP service is unavailable.
• Confusing a connection refusal with a connection timeout.
• Ignoring outbound network rules for dependency connections.
• Troubleshooting HTTP before confirming DNS, route, and transport connectivity.

Direct Answer

Verify DNS records and resolver behavior, network reachability, server identity, certificate validity, trust chains, hostname matching, and protocol negotiation in sequence.

Detailed Explanation

# Resolve through the configured system resolver.
getent ahosts api.example.com

# Inspect DNS records directly.
dig api.example.com A
dig api.example.com AAAA
dig api.example.com CNAME

# Inspect the HTTPS request path.
curl   --verbose   --resolve api.example.com:443:203.0.113.10   https://api.example.com/health

# Inspect the certificate chain and SNI behavior.
openssl s_client   -connect api.example.com:443   -servername api.example.com   -showcerts

# Show key certificate fields.
openssl s_client   -connect api.example.com:443   -servername api.example.com   </dev/null 2>/dev/null |
openssl x509   -noout   -subject   -issuer   -dates   -ext subjectAltName

Common Interview Pitfalls

• Assuming every hostname-resolution failure is an authoritative DNS problem.
• Testing an HTTPS endpoint by IP without sending the expected hostname.
• Disabling TLS verification permanently instead of correcting the trust problem.
• Checking only certificate expiration while ignoring hostname and chain validation.
• Forgetting that DNS caches can preserve an old result until expiration.
• Ignoring differences between IPv4 and IPv6 connection paths.
• Troubleshooting the application before confirming TLS termination and routing.

Direct Answer

Subnets divide address space, route tables choose next hops, security groups filter resource traffic statefully, and network ACLs filter subnet traffic using ordered stateless rules.

Detailed Explanation

# Example conceptual routing:

# Public application subnet
10.0.10.0/24 -> local
0.0.0.0/0   -> internet-gateway

# Private worker subnet
10.0.20.0/24 -> local
0.0.0.0/0   -> nat-gateway

# Database security policy:
# Inbound TCP 5432 only from application
# and worker security groups.

# Troubleshooting from a Linux workload:
ip address show
ip route show
ss -lntp
curl --connect-timeout 5 https://dependency.example.com
traceroute dependency.example.com

Common Interview Pitfalls

• Assuming a subnet is public solely because of its name.
• Opening a security group to every source instead of the required workload.
• Ignoring return-path routing and stateless network ACL rules.
• Confusing a resource security group with a subnet-level network ACL.
• Allowing direct public access to a database that only application workloads require.
• Checking cloud firewall rules without confirming that the process is listening.
• Adding a default route without understanding which gateway receives the traffic.

Direct Answer

Define the symptom and timeline, follow one failing request across layers, compare healthy and unhealthy instances, gather evidence, test one hypothesis at a time, and preserve recovery options.

Detailed Explanation

# Process and service evidence.
systemctl status resumeloop-api.service
journalctl   --unit resumeloop-api.service   --since "2026-08-05 13:30:00"

# Kernel and resource evidence.
dmesg --time-format iso | tail -n 100
free -m
df -h
df -i
cat /proc/2481/limits

# Socket and connection evidence.
ss -s
ss -lntp
ss -tan state time-wait
ss -tan state established

# Process-level evidence.
ps -o   pid,ppid,state,%cpu,%mem,etime,cmd   -p 2481

lsof -p 2481 | wc -l

# Network-path evidence.
dig api.example.com
curl --verbose https://api.example.com/health

# Use system-call tracing only for a defined,
# controlled diagnostic question.
sudo strace   -f   -p 2481   -e trace=network,file

Common Interview Pitfalls

• Restarting the affected instance before preserving logs and runtime evidence.
• Changing several network and application settings simultaneously.
• Assuming correlation with a deployment proves the deployment caused the failure.
• Investigating only average metrics that hide instance-specific failures.
• Treating a temporary mitigation as the confirmed root cause.
• Running invasive tracing or packet capture without scope and data-handling controls.
• Ignoring kernel logs during memory, disk, or network resource failures.
• Troubleshooting each layer independently without following one request end to end.

Direct Answer

Continuous integration verifies frequent code changes, continuous delivery keeps verified releases deployable, and continuous deployment automatically releases every qualifying change.

Detailed Explanation

name: Application pipeline

on:
  pull_request:
  push:
    branches:
      - main

jobs:
  verify:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@<pinned-commit>

      - name: Install dependencies
        run: npm ci

      - name: Verify
        run: |
          npm run lint
          npx tsc --noEmit
          npm test -- --runInBand
          npm run build

  deploy-production:
    if: github.ref == 'refs/heads/main'
    needs: verify
    environment: production
    runs-on: ubuntu-latest

    steps:
      - name: Deploy verified release
        run: ./scripts/deploy.sh

Common Interview Pitfalls

• Using continuous delivery and continuous deployment as interchangeable terms.
• Calling a workflow continuous integration when developers integrate infrequently.
• Deploying automatically without adequate testing or recovery controls.
• Rebuilding the release differently after continuous-integration verification.
• Treating pipeline automation as a substitute for deployment observability.
• Creating one large change that is difficult to test, review, and roll back.

Direct Answer

Building once ensures that staging and production use the same verified bytes, while immutable versions provide traceability, reproducibility, rollback, and auditability.

Detailed Explanation

# Build one immutable image.
docker build   --tag registry.example.com/resumeloop-api:${GITHUB_SHA}   .

# Push the content-addressed artifact.
docker push   registry.example.com/resumeloop-api:${GITHUB_SHA}

# Capture its immutable digest.
IMAGE_DIGEST="$(
  docker inspect     --format='{{index .RepoDigests 0}}'     registry.example.com/resumeloop-api:${GITHUB_SHA}
)"

echo "Deploying ${IMAGE_DIGEST}"

# Use the same digest in staging and production.
./deploy.sh staging "${IMAGE_DIGEST}"
./deploy.sh production "${IMAGE_DIGEST}"

Common Interview Pitfalls

• Rebuilding the application independently for staging and production.
• Deploying mutable latest tags without recording the image digest.
• Embedding production secrets into the build artifact.
• Allowing published artifacts to be overwritten under the same version.
• Retaining no known-good release for rollback.
• Recording the source commit without recording the actual deployed artifact.

Direct Answer

Stages produce progressively stronger evidence, required checks block unsafe merges, branch protection controls source changes, and environments gate sensitive deployments and secrets.

Detailed Explanation

name: Verified production release

on:
  push:
    branches:
      - main

permissions:
  contents: read

jobs:
  build-and-test:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@<pinned-commit>

      - run: npm ci
      - run: npm run lint
      - run: npx tsc --noEmit
      - run: npm test -- --runInBand
      - run: npm run build

  deploy:
    needs: build-and-test
    environment: production

    permissions:
      contents: read
      id-token: write

    runs-on: ubuntu-latest

    steps:
      - name: Authenticate using workload identity
        run: ./scripts/authenticate.sh

      - name: Deploy
        run: ./scripts/deploy-production.sh

Common Interview Pitfalls

• Allowing protected branches to merge while required verification is failing.
• Giving every pipeline job production-level credentials.
• Exposing production secrets to untrusted pull-request workflows.
• Adding manual approvals that provide no meaningful risk decision.
• Running expensive end-to-end tests before quick validation checks.
• Allowing the person initiating a sensitive deployment to be its only approver.
• Failing to pin or control third-party pipeline dependencies.

Direct Answer

Rolling replaces instances gradually, blue-green switches between complete environments, and canary exposes a new version to a small audience before increasing traffic.

Detailed Explanation

apiVersion: apps/v1
kind: Deployment
metadata:
  name: resumeloop-api
spec:
  replicas: 6

  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 0
      maxSurge: 1

  selector:
    matchLabels:
      app: resumeloop-api

  template:
    metadata:
      labels:
        app: resumeloop-api
        version: v2
    spec:
      containers:
        - name: api
          image: registry.example.com/api@sha256:<digest>
          readinessProbe:
            httpGet:
              path: /health/ready
              port: 3000

Common Interview Pitfalls

• Using rolling deployment when old and new versions cannot coexist.
• Calling a deployment canary without measuring the exposed cohort separately.
• Assuming blue-green automatically rolls back database changes.
• Switching all traffic before the new environment passes readiness checks.
• Keeping no capacity for healthy traffic during a rolling update.
• Increasing canary exposure without predefined success and stop criteria.
• Selecting a deployment strategy without considering stateful dependencies.

Direct Answer

Use backward-compatible expand-and-contract changes, separate long data migrations, verify constraints later, and keep old and new application versions compatible during rollout.

Detailed Explanation

-- Release 1: expand.
ALTER TABLE job_applications
ADD COLUMN archived_at TIMESTAMPTZ;

-- Deploy code that understands both the old status
-- representation and the new archived_at column.

-- Backfill in bounded, resumable batches.
UPDATE job_applications
SET archived_at = updated_at
WHERE id IN (
  SELECT id
  FROM job_applications
  WHERE status = 'archived'
    AND archived_at IS NULL
  ORDER BY id
  LIMIT 1000
);

-- Release 2: switch reads and writes.

-- Release 3: validate and contract only after
-- all application versions use the new model.

Common Interview Pitfalls

• Dropping a column in the same release that stops writing to it.
• Adding a required field before old application versions can supply it.
• Running a large backfill in one long production transaction.
• Assuming application rollback also reverses data mutations safely.
• Creating an index or constraint without evaluating production locking.
• Removing old schema while background workers still depend on it.
• Writing to old and new fields without defining conflict behavior.

Direct Answer

Protect source changes, isolate builds, create one signed artifact, use short-lived credentials, enforce environment gates, deploy progressively, verify health, and retain rollback evidence.

Detailed Explanation

name: Production release

on:
  workflow_dispatch:

concurrency:
  group: production-release
  cancel-in-progress: false

permissions:
  contents: read

jobs:
  verify-and-package:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@<pinned-commit>

      - run: npm ci
      - run: npm run lint
      - run: npx tsc --noEmit
      - run: npm test -- --runInBand
      - run: npm run build

      - name: Build immutable artifact
        run: ./scripts/build-image.sh "${GITHUB_SHA}"

      - name: Scan and sign artifact
        run: ./scripts/verify-artifact.sh "${GITHUB_SHA}"

  deploy-canary:
    needs: verify-and-package
    environment: production
    runs-on: ubuntu-latest

    permissions:
      contents: read
      id-token: write

    steps:
      - name: Obtain temporary deployment identity
        run: ./scripts/cloud-auth-oidc.sh

      - name: Deploy verified digest to canary
        run: ./scripts/deploy-canary.sh

      - name: Verify canary
        run: ./scripts/check-canary.sh

      - name: Expand production exposure
        run: ./scripts/promote-release.sh

Common Interview Pitfalls

• Giving untrusted pull-request workflows production credentials.
• Using permanent administrator credentials for routine deployments.
• Rebuilding the artifact after its verification or approval.
• Allowing pipeline actions to float to uncontrolled mutable versions.
• Declaring success before readiness and production smoke checks complete.
• Running overlapping production deployments without coordination.
• Deploying an incompatible database or event schema without rollback planning.
• Recording the commit but not the actual deployed artifact digest.
• Giving one pipeline identity unrestricted access to every environment.

Direct Answer

An image is an immutable packaged filesystem and metadata, while a container is a running process created from that image with isolated namespaces and controlled resources.

Detailed Explanation

# Pull an image by an immutable digest.
docker pull   registry.example.com/resumeloop-api@sha256:<digest>

# Inspect image metadata and layers.
docker image inspect   registry.example.com/resumeloop-api@sha256:<digest>

docker image history   registry.example.com/resumeloop-api@sha256:<digest>

# Run with reduced privileges and bounded resources.
docker run   --rm   --read-only   --user 10001:10001   --cap-drop ALL   --memory 512m   --cpus 1   --publish 3000:3000   registry.example.com/resumeloop-api@sha256:<digest>

Common Interview Pitfalls

• Using the terms image and container as though they describe the same object.
• Deploying mutable tags without recording the resolved image digest.
• Running application containers as root without a demonstrated requirement.
• Treating containers as equivalent to virtual machines with separate kernels.
• Giving a container privileged mode or broad host mounts unnecessarily.
• Embedding credentials or private keys into image layers.
• Assuming a deleted file is removed from every earlier image layer.

Direct Answer

Order stable build steps before frequently changing files, minimize build context and layers, and copy only required runtime artifacts into a small final stage.

Detailed Explanation

# syntax=docker/dockerfile:1

FROM node:22-alpine AS dependencies
WORKDIR /app

COPY package.json package-lock.json ./
RUN npm ci

FROM dependencies AS build
WORKDIR /app

COPY . .
RUN npm run build
RUN npm prune --omit=dev

FROM node:22-alpine AS runtime
WORKDIR /app

ENV NODE_ENV=production

RUN addgroup       --system       --gid 10001 appgroup   && adduser       --system       --uid 10001       --ingroup appgroup appuser

COPY --from=build   --chown=appuser:appgroup   /app/package.json   /app/package-lock.json   ./

COPY --from=build   --chown=appuser:appgroup   /app/node_modules   ./node_modules

COPY --from=build   --chown=appuser:appgroup   /app/.next   ./.next

COPY --from=build   --chown=appuser:appgroup   /app/public   ./public

USER appuser

EXPOSE 3000

CMD ["npm", "start"]

Common Interview Pitfalls

• Copying the complete source tree before installing unchanged dependencies.
• Including source-control history, local secrets, and build output in the build context.
• Installing compilers and development tools in the final runtime image.
• Using build arguments to pass secrets that can remain in image metadata.
• Running the final image as root without a specific requirement.
• Using a mutable base-image tag without an update and verification policy.
• Deleting sensitive files in a later layer after copying them into an earlier layer.
• Installing packages without cleaning temporary package-manager data in the same layer.

Direct Answer

Pods run colocated containers, Deployments manage desired replica and rollout state through ReplicaSets, and Services provide stable networking to selected Pods.

Detailed Explanation

apiVersion: apps/v1
kind: Deployment
metadata:
  name: resumeloop-api
spec:
  replicas: 3
  selector:
    matchLabels:
      app: resumeloop-api
  template:
    metadata:
      labels:
        app: resumeloop-api
    spec:
      containers:
        - name: api
          image: registry.example.com/api@sha256:<digest>
          ports:
            - name: http
              containerPort: 3000
---
apiVersion: v1
kind: Service
metadata:
  name: resumeloop-api
spec:
  selector:
    app: resumeloop-api
  ports:
    - name: http
      port: 80
      targetPort: http
  type: ClusterIP

Common Interview Pitfalls

• Deploying standalone Pods for workloads that require automated recovery and rollout.
• Treating Pod names or Pod IP addresses as stable service identities.
• Creating a Service selector that does not match the workload labels.
• Placing unrelated application services inside one Pod.
• Assuming a Deployment sends traffic directly without a Service or other routing layer.
• Changing Deployment selector labels after creation without understanding immutability rules.
• Assuming a running Pod is automatically ready to receive traffic.

Direct Answer

Use ConfigMaps for non-sensitive configuration, Secrets for confidential values, distinct probes for startup, readiness, and liveness, and measured requests and limits.

Detailed Explanation

apiVersion: apps/v1
kind: Deployment
metadata:
  name: resumeloop-api
spec:
  replicas: 3
  selector:
    matchLabels:
      app: resumeloop-api
  template:
    metadata:
      labels:
        app: resumeloop-api
    spec:
      containers:
        - name: api
          image: registry.example.com/api@sha256:<digest>

          envFrom:
            - configMapRef:
                name: resumeloop-api-config
            - secretRef:
                name: resumeloop-api-secrets

          resources:
            requests:
              cpu: 250m
              memory: 256Mi
            limits:
              cpu: "1"
              memory: 512Mi

          startupProbe:
            httpGet:
              path: /health/startup
              port: 3000
            periodSeconds: 2
            failureThreshold: 30

          readinessProbe:
            httpGet:
              path: /health/ready
              port: 3000
            periodSeconds: 5
            failureThreshold: 3

          livenessProbe:
            httpGet:
              path: /health/live
              port: 3000
            periodSeconds: 10
            failureThreshold: 3

Common Interview Pitfalls

• Storing confidential credentials in a ConfigMap.
• Assuming Kubernetes Secrets are automatically encrypted and inaccessible to cluster users.
• Using the same dependency-heavy endpoint for startup, readiness, and liveness.
• Making liveness fail when an optional external service is temporarily unavailable.
• Setting memory limits below normal peak usage and causing repeated termination.
• Omitting requests and allowing the scheduler to overcommit a node unknowingly.
• Setting requests far above measured usage and leaving workloads unschedulable.
• Expecting environment-variable configuration to update inside an existing process.

Direct Answer

The scheduler places Pods from resource and placement rules, autoscalers adjust capacity, disruption budgets limit voluntary loss, and rollout settings preserve healthy replicas.

Detailed Explanation

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: resumeloop-api
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: resumeloop-api
  minReplicas: 3
  maxReplicas: 20
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 65
---
apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: resumeloop-api
spec:
  minAvailable: 2
  selector:
    matchLabels:
      app: resumeloop-api
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: resumeloop-api
spec:
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 0
      maxSurge: 1
  template:
    spec:
      topologySpreadConstraints:
        - maxSkew: 1
          topologyKey: topology.kubernetes.io/zone
          whenUnsatisfiable: ScheduleAnyway
          labelSelector:
            matchLabels:
              app: resumeloop-api

Common Interview Pitfalls

• Configuring CPU-based autoscaling without defining meaningful CPU requests.
• Assuming the Horizontal Pod Autoscaler adds node capacity automatically.
• Using a PodDisruptionBudget as protection against every involuntary failure.
• Running several replicas on the same node without considering failure domains.
• Creating required anti-affinity rules that make the workload impossible to schedule.
• Setting maxUnavailable too high for a small critical workload.
• Scaling on a delayed metric without considering queue age or startup time.
• Expecting autoscaling to compensate for an inefficient or failing application instantly.

Direct Answer

Use immutable images, least privilege, measured resources, health and disruption controls, safe rollouts, observability, and a layered troubleshooting process from workload to cluster.

Detailed Explanation

# Deployment and rollout evidence.
kubectl   --namespace production   rollout status   deployment/resumeloop-api

kubectl   --namespace production   describe deployment resumeloop-api

# Pod state, placement, events, and restarts.
kubectl   --namespace production   get pods   --selector app=resumeloop-api   --output wide

kubectl   --namespace production   describe pod <pod-name>

# Current and previous container logs.
kubectl   --namespace production   logs <pod-name>   --container api

kubectl   --namespace production   logs <pod-name>   --container api   --previous

# Confirm Service selectors and endpoints.
kubectl   --namespace production   get service resumeloop-api   --output yaml

kubectl   --namespace production   get endpointslices   --selector kubernetes.io/service-name=resumeloop-api

# Inspect resource use where metrics are available.
kubectl   --namespace production   top pod   --selector app=resumeloop-api

# Add an ephemeral debugging container when permitted.
kubectl   --namespace production   debug   --interactive   <pod-name>   --image=busybox:1.36

Common Interview Pitfalls

• Restarting Pods before capturing events, previous logs, and termination reasons.
• Troubleshooting only the container while ignoring Service selectors and endpoints.
• Giving workloads cluster-admin access for operational convenience.
• Using liveness checks that restart Pods during temporary dependency failures.
• Setting no resource requests and then relying on predictable scheduling.
• Running all replicas in one failure domain.
• Using mutable image tags that make unhealthy releases difficult to identify.
• Treating CrashLoopBackOff as a root cause rather than a retry condition.
• Assuming NetworkPolicy rules are enforced without confirming network-plugin support.
• Using privileged debugging containers without authorization and data-handling controls.

Direct Answer

Regions are separate geographic areas, Availability Zones are isolated locations within a Region, and customers remain responsible for securing and operating their workloads.

Detailed Explanation

# Conceptual regional architecture

Region: us-east-1

Availability Zone A:
  - Application replica
  - Private subnet
  - Database standby

Availability Zone B:
  - Application replica
  - Private subnet
  - Database primary

Availability Zone C:
  - Application replica
  - Private subnet

Regional services:
  - Load balancer
  - Object storage
  - Monitoring
  - Managed DNS

Customer responsibilities:
  - IAM policies
  - Network rules
  - Encryption settings
  - Backup policy
  - Application security
  - Capacity and failover testing

Common Interview Pitfalls

• Running several replicas in one Availability Zone and calling the design highly available.
• Assuming a multi-zone deployment protects against every regional or application failure.
• Treating managed cloud services as though the provider owns all security configuration.
• Failing to reserve enough remaining capacity for an Availability Zone failure.
• Deploying across zones without testing failover and dependency behavior.
• Ignoring data backup because the database already has redundant infrastructure.

Direct Answer

Infrastructure as Code represents infrastructure in versioned files, while declarative configuration describes the desired end state instead of every procedural step.

Detailed Explanation

terraform {
  required_version = ">= 1.8.0"

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
}

resource "aws_s3_bucket" "application_artifacts" {
  bucket = "resumeloop-production-artifacts"

  tags = {
    Environment = "production"
    ManagedBy   = "terraform"
  }
}

resource "aws_s3_bucket_versioning" "artifacts" {
  bucket = aws_s3_bucket.application_artifacts.id

  versioning_configuration {
    status = "Enabled"
  }
}

Common Interview Pitfalls

• Treating Infrastructure as Code as a generated backup of manual console work.
• Applying infrastructure changes without reviewing the execution plan.
• Assuming declarative configuration prevents destructive changes automatically.
• Making emergency manual changes without reconciling them into source control.
• Copying complete environment configurations instead of creating reusable boundaries.
• Storing credentials directly inside infrastructure configuration files.
• Using infrastructure code without validation or policy checks in continuous integration.

Direct Answer

State maps configuration to real resources, plans preview proposed operations, modules package reusable infrastructure, locking prevents concurrent writes, and drift records external changes.

Detailed Explanation

terraform {
  backend "s3" {
    bucket         = "resumeloop-terraform-state"
    key            = "production/network/terraform.tfstate"
    region         = "us-east-1"
    encrypt        = true
    dynamodb_table = "terraform-state-locks"
  }
}

module "application_network" {
  source = "../../modules/application-network"

  environment = "production"
  vpc_cidr    = "10.20.0.0/16"

  availability_zones = [
    "us-east-1a",
    "us-east-1b",
    "us-east-1c"
  ]
}

# Pipeline sequence:
# terraform fmt -check
# terraform init
# terraform validate
# terraform plan -out=tfplan
# review and approve tfplan
# terraform apply tfplan

Common Interview Pitfalls

• Committing Terraform state files to a public or broadly accessible repository.
• Running concurrent applies against one state without locking.
• Applying a newly generated plan after reviewing a different saved plan.
• Editing Terraform state JSON manually.
• Using one state file for unrelated infrastructure with different ownership.
• Ignoring drift until an unrelated deployment attempts destructive reconciliation.
• Creating modules that expose every provider argument without adding a meaningful boundary.
• Moving resource addresses without preserving the state association.

Direct Answer

Grant only required actions on required resources, prefer assumable roles and temporary credentials, constrain trust policies, and give each workload a distinct identity.

Detailed Explanation

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "ReadOnlyRequiredSecret",
      "Effect": "Allow",
      "Action": [
        "secretsmanager:GetSecretValue"
      ],
      "Resource": [
        "arn:aws:secretsmanager:us-east-1:123456789012:secret:production/resumeloop/database-*"
      ]
    },
    {
      "Sid": "DecryptThroughSecretsManager",
      "Effect": "Allow",
      "Action": [
        "kms:Decrypt"
      ],
      "Resource": [
        "arn:aws:kms:us-east-1:123456789012:key/example-key-id"
      ],
      "Condition": {
        "StringEquals": {
          "kms:ViaService": "secretsmanager.us-east-1.amazonaws.com"
        }
      }
    }
  ]
}

Common Interview Pitfalls

• Granting wildcard actions and resources for deployment convenience.
• Using one long-lived access key across several workloads.
• Restricting permission policies while leaving the role trust policy overly broad.
• Giving every Kubernetes Pod the node or cluster-wide cloud permissions.
• Keeping temporary emergency permissions after the incident ends.
• Assuming a role name alone proves which workload assumed it.
• Granting read access to every secret when one named secret is required.
• Creating policies without reviewing actual access patterns later.

Direct Answer

Store secrets in managed systems, encrypt them with controlled keys, grant narrow access, audit retrieval, rotate credentials safely, and keep plaintext out of code and logs.

Detailed Explanation

async function loadDatabaseCredentials(): Promise<{
  username: string;
  password: string;
}> {
  const response = await secretsManager.getSecretValue({
    secretId:
      'production/resumeloop/database'
  });

  if (!response.secretString) {
    throw new Error(
      'Database secret has no string value'
    );
  }

  const parsed: unknown =
    JSON.parse(response.secretString);

  if (
    typeof parsed !== 'object' ||
    parsed === null ||
    !('username' in parsed) ||
    !('password' in parsed) ||
    typeof parsed.username !== 'string' ||
    typeof parsed.password !== 'string'
  ) {
    throw new Error(
      'Database secret has an invalid structure'
    );
  }

  return {
    username: parsed.username,
    password: parsed.password
  };
}

// Never log the returned credentials.

Common Interview Pitfalls

• Committing secrets to source control and deleting them only in a later commit.
• Embedding secrets in container-image layers or build arguments.
• Granting every workload permission to retrieve every production secret.
• Rotating a stored secret without updating the target database or service.
• Logging complete configuration objects containing secret values.
• Confusing encryption-key rotation with credential rotation.
• Allowing applications to use permanent credentials when workload identity is available.
• Changing a secret without considering already running application instances.

Direct Answer

Separate trust boundaries, reuse versioned modules, isolate state and credentials, enforce policy in pipelines, promote reviewed plans, and verify security and resilience continuously.

Detailed Explanation

infrastructure/
├── modules/
│   ├── application-network/
│   ├── kubernetes-platform/
│   ├── managed-database/
│   └── workload-identity/
└── environments/
    ├── development/
    │   ├── backend.tf
    │   ├── main.tf
    │   └── variables.tf
    ├── staging/
    │   ├── backend.tf
    │   ├── main.tf
    │   └── variables.tf
    └── production/
        ├── backend.tf
        ├── main.tf
        └── variables.tf

# Example pipeline sequence:
terraform fmt -check
terraform init
terraform validate
terraform plan -out=tfplan
security-policy-check tfplan
manual-production-approval tfplan
terraform apply tfplan
post-deployment-verification

Common Interview Pitfalls

• Keeping development and production in one unrestricted administrative trust boundary.
• Using one Terraform state file for every environment and platform component.
• Copying modules into each environment and allowing them to diverge.
• Giving the planning job the same broad write permissions as the apply job.
• Allowing production to consume an unpinned module branch.
• Assuming environment parity requires production-scale development infrastructure.
• Sharing production secrets or customer data with lower environments.
• Applying infrastructure plans without preserving the reviewed plan and audit record.
• Using multi-zone infrastructure without testing failure and recovery behavior.
• Treating drift detection as a substitute for preventing unauthorized manual changes.

Direct Answer

Metrics reveal system trends, logs record discrete events, and traces follow requests across services; together they help engineers explain production behavior.

Detailed Explanation

from opentelemetry import trace

tracer = trace.get_tracer(
    "job-service"
)

def get_job(job_id: str):
    with tracer.start_as_current_span(
        "get_job"
    ) as span:
        span.set_attribute(
            "job.id",
            job_id
        )

        return repository.find(job_id)

Common Interview Pitfalls

• Treating monitoring and observability as exactly the same concept.
• Collecting logs without correlation identifiers across distributed services.
• Using metrics with extremely high-cardinality labels such as user IDs.
• Logging secrets or unnecessary sensitive data.
• Collecting traces without preserving useful service and deployment context.
• Assuming one telemetry signal can answer every troubleshooting question.
• Keeping unlimited telemetry without considering storage and operational cost.
• Instrumenting infrastructure while ignoring user-facing application behavior.

Direct Answer

An SLI measures service behavior, an SLO defines its target, an SLA is a service commitment, and an error budget quantifies allowable unreliability.

Detailed Explanation

from dataclasses import dataclass

@dataclass
class ReliabilityWindow:
    eligible_requests: int
    successful_requests: int

def success_ratio(
    window: ReliabilityWindow,
) -> float:
    if window.eligible_requests == 0:
        return 1.0

    return (
        window.successful_requests
        / window.eligible_requests
    )

def remaining_error_budget(
    actual_success_ratio: float,
    target_success_ratio: float,
) -> float:
    allowed_failure = (
        1.0 - target_success_ratio
    )

    actual_failure = (
        1.0 - actual_success_ratio
    )

    return allowed_failure - actual_failure

Common Interview Pitfalls

• Defining an SLO using infrastructure metrics that users do not experience.
• Treating an SLO and an SLA as interchangeable concepts.
• Setting every service objective to 100 percent without evaluating cost.
• Defining availability while ignoring latency and correctness.
• Creating an error budget but never using it in release decisions.
• Changing SLO definitions whenever the service performs poorly.
• Counting requests that should be excluded from the SLI denominator.
• Using a single SLO for services with several important user-facing behaviors.

Direct Answer

Alert on meaningful user impact or imminent risk, define ownership and response actions, deduplicate noise, and continuously tune alerts using incident evidence.

Detailed Explanation

groups:
  - name: api-reliability
    rules:
      - alert: HighRequestFailureRate
        expr: |
          (
            sum(rate(http_requests_total{
              status=~"5.."
            }[5m]))
            /
            sum(rate(http_requests_total[5m]))
          ) > 0.05
        for: 10m
        labels:
          severity: page
        annotations:
          summary: >
            API error rate above 5%
          runbook: >
            https://internal/runbooks/api-errors

Common Interview Pitfalls

• Paging on every infrastructure threshold regardless of user impact.
• Creating alerts with no owner or documented response procedure.
• Using the same urgency level for every operational event.
• Allowing one dependency failure to produce dozens of duplicate pages.
• Keeping noisy alerts because they might become useful someday.
• Alerting on instantaneous spikes without considering persistence.
• Using email as the primary mechanism for urgent operational events.
• Measuring alert quantity without measuring whether alerts are actionable.

Direct Answer

Establish incident command, assess impact, mitigate quickly, communicate clearly, preserve evidence, verify recovery, and convert postmortem findings into owned improvements.

Detailed Explanation

from dataclasses import dataclass
from enum import Enum

class Severity(str, Enum):
    SEV1 = "sev1"
    SEV2 = "sev2"
    SEV3 = "sev3"

@dataclass
class Incident:
    severity: Severity
    user_impact: str
    commander: str
    mitigation: str | None = None
    resolved: bool = False

def can_close_incident(
    service_healthy: bool,
    backlog_recovered: bool,
    data_integrity_verified: bool,
) -> bool:
    return all([
        service_healthy,
        backlog_recovered,
        data_integrity_verified,
    ])

Common Interview Pitfalls

• Having every engineer make changes independently during a major incident.
• Focusing on perfect root-cause diagnosis before applying a safe mitigation.
• Declaring recovery from one infrastructure metric without checking user impact.
• Failing to maintain a shared incident timeline.
• Publishing speculative incident updates as established facts.
• Writing postmortems that focus primarily on individual blame.
• Creating corrective actions without owners or deadlines.
• Closing incidents while significant backlogs or integrity issues remain.

Direct Answer

Design for failure domains, maintain capacity headroom, automate safe failover, define recovery objectives, and test failure and disaster scenarios regularly.

Detailed Explanation

from dataclasses import dataclass

@dataclass
class RecoveryRequirements:
    recovery_time_minutes: int
    recovery_point_minutes: int
    survives_zone_failure: bool
    restore_tested: bool

def recovery_ready(
    requirements: RecoveryRequirements,
) -> bool:
    return all([
        requirements.recovery_time_minutes <= 30,
        requirements.recovery_point_minutes <= 5,
        requirements.survives_zone_failure,
        requirements.restore_tested,
    ])

Common Interview Pitfalls

• Calling a system highly available because it has several replicas in one failure domain.
• Running production at full capacity with no headroom for instance or zone failures.
• Assuming successful backups prove that disaster recovery works.
• Designing failover without testing dependency limits in the failover region.
• Requiring every noncritical feature to remain fully available during severe failures.
• Running chaos experiments without defining blast radius and abort conditions.
• Writing disaster recovery plans that are never exercised.
• Ignoring additional capacity required during deployments and failovers.

Direct Answer

Standardize telemetry and reliability primitives, assign service ownership, define tiered SLOs, automate safe operations, and preserve team accountability for production outcomes.

Detailed Explanation

from dataclasses import dataclass
from enum import Enum

class ServiceTier(str, Enum):
    CRITICAL = "tier-0"
    USER_FACING = "tier-1"
    STANDARD = "tier-2"
    INTERNAL = "tier-3"

@dataclass
class ServiceReliabilityMetadata:
    service_name: str
    owner: str
    tier: ServiceTier
    slo_defined: bool
    on_call_defined: bool
    runbook_defined: bool
    dashboard_defined: bool
    recovery_tested: bool

def production_ready(
    service: ServiceReliabilityMetadata,
) -> bool:
    base_requirements = all([
        bool(service.owner),
        service.dashboard_defined,
        service.runbook_defined,
    ])

    if not base_requirements:
        return False

    if service.tier in {
        ServiceTier.CRITICAL,
        ServiceTier.USER_FACING,
    }:
        return all([
            service.slo_defined,
            service.on_call_defined,
            service.recovery_tested,
        ])

    return True

Common Interview Pitfalls

• Making a central SRE team operationally responsible for every product service.
• Applying identical availability requirements to critical and low-impact workloads.
• Allowing teams to invent incompatible telemetry naming conventions.
• Creating dashboards without defining who owns the underlying services.
• Measuring infrastructure health while omitting user-centered SLOs.
• Defining error budgets without connecting them to release policy.
• Centralizing every incident decision and creating an operational bottleneck.
• Collecting unlimited telemetry without cardinality and retention controls.
• Automating remediation without rate limits, abort conditions, or escalation paths.
• Tracking incident counts without tracking repeated failures and corrective-action completion.