Testing-Farm on thrix's blog http://vadkerti.net/tags/testing-farm/ Recent content in Testing-Farm on thrix's blog Hugo -- 0.152.2 en-us Fri, 17 Jul 2026 00:00:00 +0000 The Dumb Git Protocol That Flooded Our Git Server http://vadkerti.net/posts/the-dumb-git-protocol-that-flooded-our-git-server/ Fri, 17 Jul 2026 00:00:00 +0000 http://vadkerti.net/posts/the-dumb-git-protocol-that-flooded-our-git-server/ A CI clone spike pulled terabytes off an internal git server and caused an outage. The cause was one URL pointing at git’s dumb HTTP protocol, which makes every fresh clone re-fetch a repo object by object. A DNS Record That Existed Everywhere Except CI http://vadkerti.net/posts/dns-negative-caching-broke-our-ephemeral-ci/ Tue, 30 Jun 2026 00:00:00 +0000 http://vadkerti.net/posts/dns-negative-caching-broke-our-ephemeral-ci/ A flaky CI failure where a freshly-created DNS record was unresolvable from inside the cluster, but resolved fine from my laptop a few minutes later. The culprit was negative DNS caching, driven by the zone’s SOA, poisoning lookups for our ephemeral per-job records. How a Single Regex Stalled 30 Testing Farm Jobs for Hours http://vadkerti.net/posts/how-a-single-regex-stalled-30-testing-farm-jobs/ Fri, 06 Mar 2026 00:00:00 +0000 http://vadkerti.net/posts/how-a-single-regex-stalled-30-testing-farm-jobs/ Tracing stuck Testing Farm jobs from process list through Python stack traces to a catastrophic regex backtracking bug in tmt — caused by a 1M-character line in test output.