Quantar Genomics
2.3 terabytes were queued to leave. Fourteen got out.
A compromised research notebook started staging a genome archive to an external bucket. Egress baselining caught it inside two minutes, at 14 GB of a 2.3 TB transfer.
- Sector
- Biotechnology
- Estate
- GCP · 640 workloads · GKE research clusters
- Regions
- europe-west4, us-central1
- Plan
- Business
- Customer since
- November 2024
- 14 GBExfiltrated before cut-offof 2,341 GB staged
- 1m 48sFirst byte to containmentautomated egress hold, then human confirm
- 3Prior tools that saw nothingCSPM, agent-based EDR, cloud-native SIEM
- £0Regulatory penaltyno identifiable participant data in the staged set
Every other product we ran was looking at configuration. The bucket was configured correctly. The notebook was allowed to read the archive. Nothing was misconfigured — someone was just doing something a researcher has never done at 4am.
Quantar sequences and stores genomic data for eleven research consortia. The crown jewel is not a database — it is a 400 TB object store of aligned reads, and the whole point of it is that researchers can read it. You cannot lock it down. You can only notice.
Nothing was misconfigured
At 03:52 on a Tuesday, a JupyterHub notebook belonging to a postdoctoral researcher began copying objects from the archive bucket to an external Cloud Storage bucket in a different organisation. The notebook had legitimate read access. The researcher’s credentials were valid. The destination bucket was not on any blocklist, because it had been created ninety minutes earlier.
Quantar’s cloud posture tool reported green. Their EDR agent reported green, correctly — no
process on that node was malicious; gsutil is not malware. Their SIEM had the logs and no
rule that cared.
What fired
Two detections correlated into one situation:
- Workload egress deviation.
research-prod/jupyter-pod-7f2moved 41× its 40-day p99 in a sixty-second window. Baselines are per-workload and per-hour-of-week, so 4am is compared against 4am. - Service account first-use. The destination project had never appeared in any Quantar audit log. First-use of an external principal is a weak signal on its own and generates noise; combined with the egress deviation it is not weak at all.
Neither rule alone would have paged. Together they opened situation #881 at 03:53:26.
The containment decision
Quantar runs automated egress hold on exactly one namespace: research-prod. The hold
applies a deny-all egress NetworkPolicy to the offending pod and posts to Slack with a
sixty-second undo button. It fired at 03:53:41. Tom confirmed at 03:54:12 from his phone.
Fourteen gigabytes left. Two-point-three terabytes did not.
Afterwards
The root cause was a phished session cookie, three weeks old, used from a residential proxy. It is a boring root cause. The interesting part is that three security products watching the same cluster saw a correctly-configured system doing a permitted thing, and the only question that produced an answer was: has this workload ever behaved like this before?