Deployed

• ✓ v0.1.16 images on Docker Hub (multi-arch amd64+arm64) ✓
• ✓ Chart bumped to 0.1.17 ✓
• ✓ Flux image policies resolved to 0.1.16 ✓
• ✓ Flux GitRepository Ready ✓
• ✓ Jane pods updated to 0.1.16 ✓

CI Status

• ✓ Speech tests: skip gracefully when torch absent ✓
• ✓ bump-chart step: fixed (git push origin HEAD:master) ✓
• ○ Known: opencode image cross-repo pull perms (OPENCODE_PAT needed for Flux deploy key, not CI)

Next

• ○ Verify jane.yash.abyssmallab.in after update
• ○ Sub-project: NetworkPolicies + PDBs

Nodes

Installed Components

Running Image Versions

• ○ bridge: jane-bridge:0.1.16 (with jane-opencode:0.1.16 sidecar)
• ○ pwa: jane-pwa:0.1.16
• ○ speech: jane-speech:0.1.16
• ○ nats: nats:2.14.2-alpine

  report-server: beingyash/report-server:v0.1.1 + git-sync:v4.2.1

Flux Status — RESOLVED ✓

• ○ Worker node oracle-cp had abysmallab.in in its DNS search domains from Tailscale MagicDNS.
• ○ *.abysmallab.in wildcard resolves to oracle-cp itself (140.238.165.62).
• ○ With ndots:5, pod queries like github.com were suffixed to github.com.abysmallab.in, resolving back to the worker node.
• ✗ This caused both tls: internal error (HTTPS) and knownhosts: key mismatch (SSH) because connections were hitting local Caddy/Traefik instead of GitHub.
• ○ Control-plane node was already protected by resolvConf: /etc/resolv.clean.conf in kubelet config; worker node was using /run/systemd/resolve/resolv.conf.

  Fix applied:

• ○ Created /etc/resolv.clean.conf on oracle-cp with only nameserver 127.0.0.53 and no search domains.
• ○ Updated /var/lib/kubelet/config.yaml on oracle-cp to use resolvConf: /etc/resolv.clean.conf.
• ○ Restarted kubelet; recreated source-controller pod; switched jane-chart back to SSH URL.
• ○ Updated ansible/add-worker.yml so future workers get the same clean resolv.conf.

Known Issues

• ○ Only traefik has LoadBalancer service (EXTERNAL-IP <pending>) — no MetalLB (bare-metal)
• ○ Traffic reaches node via NodePort: HTTP 32580, HTTPS 32305

  ### Flux GitRepository — resolved ✓

• ○ Cause: Tailscale MagicDNS search domain abysmallab.in + wildcard DNS caused pod DNS to resolve github.com back to worker node.
• ○ Fix: kubelet on worker uses /etc/resolv.clean.conf (no search domains).

  ### Monitoring — resolved ✓

• ○ metrics-server: working for both nodes
• ○ kube-prometheus-stack: 7/7 pods running
• ○ Grafana at grafana.yash.abysmallab.in

  ### Cross-node networking — resolved ✓

• ○ Pod-to-pod across nodes working
• ○ Routing fix: ip rule 5255 persisted via systemd

Sub-project 3: Security & Resilience — DONE ✓

• ○ Default-deny ingress applied to: jane, pocketid, traefik-system, cert-manager, default, monitoring, kube-system, sealed-secrets
• ○ flux-system skipped (already has its own policies)
• ○ Per-namespace allow rules tailored to app needs:

  - jane: traefik-system (80,3000,8000), intra-namespace, monitoring scraping - pocketid: traefik-system on :1411 - traefik-system: monitoring scraping on :9100, dashboard on :8080 (external traffic not restricted by NP) - cert-manager: webhook from all, monitoring scraping, intra-namespace - default: traefik-system on :3000 (report-server), monitoring scraping - monitoring: grafana from traefik, intra-namespace, prometheus API - kube-system: DNS from all, intra-namespace, monitoring scraping - sealed-secrets: API from all, monitoring scraping### PodDisruptionBudgets (minAvailable: 1)

• ○ jane/bridge-pdb, jane/pwa-pdb, jane/speech-pdb
• ○ pocketid/pocketid-pdb
• ○ traefik-system/traefik-pdb

  ### Verification

• ○ All app pods Running
• ○ Ingress endpoints reachable: id.yash.abysmallab.in (200), jane.yash.abysmallab.in (200), reports.yash.abysmallab.in (200), grafana.yash.abysmallab.in (302)
• ○ DNS resolution, external connectivity, Flux gitrepositories all Ready
• ○ Prometheus targets: 14 UP, 9 DOWN (down targets likely pre-existing — control-plane scraping auth)
• ○ Rollback: kubectl delete networkpolicy --all -n <ns> per namespace

  ### Manifests

• ○ netpol/ — 9 YAML files covering all namespaces
• ○ pdb/ — 3 YAML files (jane combined, pocketid, traefik)

Sub-project 4: Storage Durability (Longhorn) — DONE ✓

• ○ Longhorn v1.12.0 installed via Helm (longhorn/longhorn chart)
• ○ open-iscsi and nfs-common pre-installed on both nodes
• ○ Default StorageClass: longhorn (2 replicas, Immediate binding mode)
• ○ local-path de-prioritized (no longer default, kept as fallback)

  ### PVC Migration

• ○ pocketid/pocketid-data (5Gi SQLite):

  - Backed up /app/data/pocket-id.db → /tmp/pocketid-backup.sqlite3 - Scaled down, deleted old local-path PVC, created new with storageClassName: longhorn - Scaled up → pod healthy, ingress HTTP 200

• ○ jane/nats-js-nats-0 (10Gi JetStream):

  - Scaled down, deleted old local-path PVC, created new with storageClassName: longhorn - Scaled up → pod Running 1/1, NATS HTTP 200

• ○ monitoring/prometheus left on local-path (ephemeral per spec; migrate later if needed)

  ### Volume Health

• ○ Both Longhorn volumes attached, degraded robustness (3rd replica stopped — only 2 nodes available)
• ○ Replicas: 2 running per volume (one per node), 1 stopped pending third node
• ○ Acceptable for lab environment

  ### Known Issue: CP Node DiskPressure

• ○ CP node (marketgenie-market-genie) had transient DiskPressure after Longhorn install
• ○ Evicted Longhorn CSI/instance-manager pods on CP
• ○ Resolved by pruning containerd images + journal vacuum
• ○ CSI plugin now Running on both nodes
• ○ Root cause: 45GB root disk with 24GB containerd images. Monitor over time.

Next

• ○ ~~Sub-project 3: NetworkPolicies + PDBs~~ ✓
• ○ ~~Sub-project 4: Longhorn storage~~ ✓
• ○ Sub-project 5: Image multi-arch cleanup

Deployed

• ○ Chart: v0.1.3 deployed at reports.yash.abysmallab.in
• ○ App image: beingyash/report-server:v0.1.1
• ○ Sidecar: git-sync:v4.2.1
• ✓ TLS: Let's Encrypt cert issued (DNS-01 via Cloudflare), HTTPS 200 ✓
• ○ Helm status: deployed, revision 4
• ✓ Flux GitRepository: Ready ✓
• ✓ Flux HelmRelease: Ready ✓

New in v0.1.3

• ○ Global status dashboard at root / replaces old reports listing
• ○ Old reports listing moved to /reports
• ○ Design tokens: dark theme, card grid, responsive layout

Known Issues

• ○ (none)

Next

• ○ Verify overview dashboard renders correctly