GitHub Refines Layered Defenses to Reduce False Positives
These articles are AI-generated summaries. Please check the original sources for full details.
GitHub Reworks Layered Defenses after Legacy Protections Block Legitimate Traffic
GitHub engineers recently investigated user reports of unexpected “Too Many Requests” errors, tracing the issue to abuse-mitigation rules that had accidentally remained active long after the incidents that prompted them, with affected users making only a handful of normal requests. The investigation revealed that older incident rules were based on traffic patterns that were strongly associated with abuse at the time but later began matching some legitimate, logged-out requests, highlighting the importance of regularly reviewing and updating defensive controls.
Why This Matters
The technical reality of layered defenses is that they can provide robust protection against threats, but they can also lead to false positives and unnecessary blocking of legitimate traffic, resulting in unacceptable user impact, as seen in GitHub’s case, where even a small fraction of blocked requests can have significant consequences, emphasizing the need for careful management and monitoring of defensive controls to ensure they remain effective and do not outlive their usefulness.
Key Insights
- According to GitHub, only a small subset of requests that matched suspicious fingerprints were blocked, resulting in roughly 0.5-0.9% of fingerprint matches being blocked: https://www.infoq.com/news/2026/02/github-layered-def/
- Layered defenses can make attribution harder when something goes wrong, as each layer can legitimately rate-limit or block, and isolating which layer made the decision requires correlating logs across multiple systems with different schemas.
- Vercel’s published request lifecycle describes requests encountering “multiple stages” of its firewall protections, followed by an additional WAF stage for project-level policies, demonstrating the use of layered defenses in other large platforms: https://vercel.com/
Working Example
# Example of a simple rate limiter
import time
class RateLimiter:
def __init__(self, max_requests, time_window):
self.max_requests = max_requests
self.time_window = time_window
self.requests = []
def allow_request(self):
current_time = time.time()
self.requests = [request for request in self.requests if current_time - request < self.time_window]
if len(self.requests) < self.max_requests:
self.requests.append(current_time)
return True
return False
# Create a rate limiter with a maximum of 10 requests per minute
limiter = RateLimiter(10, 60)
# Test the rate limiter
for i in range(15):
if limiter.allow_request():
print(f"Request {i} allowed")
else:
print(f"Request {i} blocked")
Practical Applications
- Use Case: GitHub’s experience demonstrates the importance of regularly reviewing and updating defensive controls to ensure they remain effective and do not outlive their usefulness, highlighting the need for careful management and monitoring of layered defenses.
- Pitfall: A common anti-pattern is to introduce defensive controls without proper monitoring and maintenance, leading to unnecessary blocking of legitimate traffic and decreased system resilience, emphasizing the need for a structured approach to defense-in-depth.
References:
Continue reading
Next article
Java Explores Carrier Classes for Enhanced Data Modeling
Related Content
How a Mesh of Old Phones and LLM Agents Becomes a Self-Healing Organism
A self-hosted system uses old phones as bodies, LLMs as minds, and a shared log for coordination—no cloud middleware needed.
The 8 Fallacies of Distributed Computing: Why Your Assumptions Will Break Production
Dev.to article exposes 8 hard truths about distributed systems—from latency assumptions to topology myths—challenging developers.
"Nobody's Walking Over to a Desk": The Hidden Cost of Removing Humans from Software Spec Loops
Simon Schrottner argues that removing human confusion from the loop removes the only mechanism that caught planning mistakes before they shipped.