Browse past weeks of engineering reads.
How to handle 121 million concurrent gRPC connections from mobile devices establishing persistent connections simultaneously during live events without concentrating traffic onto a single origin endpoint.
Cloudflare needed to optimize cache tier selection for origins hosted across multiple public cloud providers (AWS, GCP, Azure, Oracle Cloud) to improve cache hit rates and reduce latency.
Network administrators needed deep, programmable control over how BGP routes are evaluated and propagated without requiring expensive third-party virtual appliances.
Netflix needed to simplify and standardize batch compute job queueing and scheduling while transitioning their container infrastructure to be more Kubernetes-native.
How can a CDN provider protect civil society organizations from cyberattacks while operating at scale across global infrastructure?
Enabling organizations to route public traffic to private applications without exposing private IP addresses or deploying additional connector infrastructure.
Ensuring high availability and service continuity when AI inference workloads fail in one region while maintaining access to the service across multiple regions.
Managing startup latencies up to 20 seconds for AI workloads on Cloud Run serverless GPUs, which causes poor user experience and is driving developers back to traditional container orchestration.
Migrating business-critical load balancer configurations from on-premises hardware solutions to Google Cloud while preserving existing traffic manipulation logic.
Building a multi-tenant architecture that isolates tenants without requiring separate AWS accounts while maintaining stateful service deployments.
Browser Run needed higher usage limits, better performance, and improved reliability while increasing development velocity for their browser automation service.
Netflix needed to design a domain-independent traffic routing system for their ML model serving infrastructure that could handle personalized experiences at scale across multiple domains while maintaining high availability.
Cloudflare needed to build an internal AI engineering stack that could handle massive scale (20 million requests, 241 billion tokens) while dogfooding their own platform products.
Cloudflare needed to improve request handling performance across its global network to maintain competitive advantage over other CDNs.
How to efficiently run inference for extra-large language models on edge infrastructure while maintaining low latency and high throughput across distributed Cloudflare servers.
Developers needed a unified way to access multiple AI model providers without managing separate integrations and API contracts for each one.
Cloudflare Workflows needed to support higher concurrency and creation rate limits to enable durable background agents at scale.
Simplifying the deployment and scheduling of machine learning inference workloads across multiple instances and instance types on Amazon SageMaker HyperPod.
How to scale a global content delivery and DDoS mitigation network to handle massive throughput (500 Tbps) while maintaining capacity to protect against record-breaking attacks.
Magic Transit customers needed the ability to define and enforce custom DDoS mitigation logic for proprietary and non-standard UDP protocols without being limited to Cloudflare's pre-built detection rules.
CDN cache systems were designed for human traffic patterns but struggle with the distinct access patterns of AI bot traffic, which now represents over 10 billion requests per week and threatens cache efficiency.
Meta needed to scale their ads ranking models to LLM-scale complexity and size while maintaining inference latency requirements for real-time ad serving.
Salesforce's Cluster Autoscaler could not efficiently scale and manage node provisioning across their fleet of 1,000+ EKS clusters, likely suffering from slow scaling decisions, suboptimal bin-packing, and operational complexity at massive scale.
Cloudflare's existing server fleet could not keep pace with rapidly growing global traffic demands, requiring a new generation of hardware with significantly higher compute and network throughput.