SKYLINE: Rethinking the Cloud-Native Stack From Hardware to Software

To democratize access to cloud computing systems, cloud providers have introduced new, cloud-native, computing paradigms. These emerging paradigms, including microservices and serverless computing, offer significantly simpler programming models alongside cost-efficient billing models. However, cloud-native services differ fundamentally from traditional monolithic applications. They exhibit short execution times, frequent context switching, bursty request patterns, and strict tail latency requirements. Hence, when such workloads run on conventional hardware and software systems, they end up having substantial performance, energy, and resource inefficiencies.
In this talk, I will present my research efforts to tackle these challenges by designing hardware platforms that deliver orders of magnitude improvements in the efficiency of cloud-native workloads. I will give an overview of research vision and focus on two examples. First, I will introduce μManycore, a processor architecture tailored to minimize the tail latency of cloud services. Then, I will integrate many domain-specific accelerators with the processor and introduce AccelFlow, an efficient mechanism for orchestrating these accelerators. Finally, I will show how many of the architectural principles developed for cloud-native workloads are becoming increasingly relevant for emerging agentic AI systems, providing a foundation for efficient execution of large-scale, multi-agent workloads.

Jovan Stojkovic is an incoming assistant professor in the Computer Science Department at the University of Texas at Austin. Currently, he is a visiting researcher at Meta. Jovan holds a Ph.D. from the University of Illinois at Urbana-Champaign. Jovan's research interests are in computer architecture and systems for cloud and datacenter computing. His work has been published at top-tier computer architecture conferences such as ISCA, HPCA, MICRO, and ASPLOS, and he holds six patents for his work with Microsoft and IBM on serverless computing, overclocking in the cloud, and AI datacenter design. His research has been recognized with accolades, including: David J. Kuck Outstanding PhD Thesis Award, HPCA Best Paper Award, IEEE Micro Top Pick Honorable Mentions, W. J. Poppelbaum Memorial Award, Kenichi Miura Award, and an invitation to speak at the Heidelberg Laureate Forum.

Cloud-native AI platforms are rapidly becoming the dominant consumers of datacenter resources, driving unprecedented demand for compute, power, and cooling. Yet today's cloud stacks remain largely siloed. As AI pushes physical infrastructure toward capacity limits, sustaining growth requires a fundamental shift: leveraging the extreme flexibility of modern cloud-native software stacks to optimize the rigid underlying hardware. In this talk, I will share lessons from building and deploying cross-stack optimization frameworks for hyperscale AI workloads. I will show how taking an end-to-end perspective allows us to exploit the elasticity of SaaS architectures and container orchestration to solve deep physical infrastructure constraints. I will present two examples of this philosophy in practice.
1. From Static Provisioning to Software-Driven Power Oversubscription. Traditional approaches manage power constraints through conservative provisioning and fixed oversubscription policies, leaving substantial capacity stranded. Exposing fine-grained infrastructure telemetry and integrating it with cloud-native control planes enables a fundamentally different model. By combining cross-stack telemetry sharing, power-aware placement, and request-level inference routing, infrastructure constraints can be dynamically incorporated into runtime decisions. Leveraging the elasticity of Kubernetes orchestration and SaaS serving architectures, these techniques reduce throttling, improve multi-tenant efficiency, and unlock significant serving capacity. 2. End-to-End Datacenter Lifecycle Co-Design. The same principles extend beyond runtime management to long-term infrastructure planning. As software systems become increasingly adaptive, decisions involving power delivery, cooling infrastructure, hardware refreshes, and operational policies can be jointly optimized with operational policies to dramatically lower Total Cost of Ownership (TCO) while supporting continued AI growth.
Together, these experiences suggest a broader evolution in cloud infrastructure: moving from optimizing isolated components toward a deeply coordinated end-to-end ecosystem. As AI workloads continues to scale, exploiting software-stack flexibility to orchestrate hardware resources will be essential for achieving sustainable performance, efficiency, and capacity growth.

Chaojie Zhang is a Senior Research SDE in the Azure Research - Systems Group at Microsoft. Her work focuses on improving large-scale cloud efficiency across the stack through hardware-software co-design. Her research spans datacenter resource management, efficient systems for AI, and infrastructure planning. Chaojie's work has been published in leading computer architecture and systems venues, including ISCA, ASPLOS, and HPCA. Beyond research, she works closely with engineering teams to translate research innovations into production systems across Microsoft's AI and cloud platforms. She holds a Ph.D. from the University of Chicago.

As the impact of artificial intelligence (AI) continues to proliferate, computer architects must assess and mitigate its demands for energy and infrastructure. This talk presents a vision for sustainable datacenter computing and surveys recent advances from an NSF Expedition in Computing, a multi-university effort that spans hardware, systems, algorithms, and policy. The talk will draw on foundational academic research as well as data and experiences from industrial, hyperscale systems. We begin by quantifying AI's environmental impacts and fundamental trade-offs between capital and operating costs. We then traverse the datacenter stack, discussing topics such as the fair attribution of power and carbon with game theory; infrastructure design with carbon-efficient energy generation and storage; and emerging interfaces between the datacenter and electricity grid. The talk provides a broad perspective on sustainable computing and outlines the many directions for future work.

Benjamin C. Lee is a Professor of Electrical and Systems Engineering and of Computer and Information Science at the University of Pennsylvania. He is also a visiting researcher in AI and Infrastructure at Google. Dr. Lee’s research focuses on computer architecture (microprocessors, memories, datacenters), energy efficiency, and environmental sustainability. He builds interdisciplinary links to machine learning and algorithmic economics to better design and manage computer systems. He received his post-doctorate from Stanford, Ph.D. from Harvard, and B.S. from the University of California at Berkeley. He has also held visiting positions at Meta AI, Microsoft Research, Intel Labs, and Lawrence Livermore National Lab. He is an IEEE Fellow and ACM Distinguished Scientist.

In 2017, AlphaZero revolutionized strategy games by discovering counterintuitive tactics entirely free from human bias. A similar inflection point is now emerging in computer systems, where evolutionary coding agents are autonomously discovering novel optimization heuristics. This talk explores this paradigm shift across the computing stack, starting with how AI generates new microarchitectural hardware prefetchers. We then examine an agentic framework that synthesizes compiler optimization passes that outperform decades of manual tuning, before exploring how similar techniques scale to high-level code optimization for warehouse-scale datacenters. Finally, I will discuss my thoughts on this inflection point's trajectory, the major evaluation and simulation challenges we face, and why I believe a true AlphaZero moment for system design is inevitable.

Amir Yazdanbakhsh is a Research Scientist at Google DeepMind, where his work bridges the critical intersection of machine learning and computer architecture. His primary research interests encompass machine learning for systems, automated code generation, and developer efficiency. Prior to DeepMind, Dr. Yazdanbakhsh completed the Google Brain AI Residency, where he developed scalable reinforcement learning systems for complex real-world tasks and applied machine learning to accelerator design. His work in hardware-software co-design has driven significant industry innovation, directly shaping multiple headline hardware features across several generations of Google’s Tensor Processing Units (TPUs). Dr. Yazdanbakhsh is an ISCA Hall of Fame inductee, and his research has earned multiple IEEE Micro Top Picks distinctions, CACM Research Highlights nominations, and Distinguished Paper Awards. He received his PhD in Computer Science from Georgia Tech and was awarded the Microsoft Research PhD Fellowship and the Qualcomm Innovation Fellowship.

The software stack of cloud-native software is complex and different from that of monolithic cloud applications. However, we can innovate by adapting known techniques from computer architecture to the distributed systems software of these environments. An analysis of the bottlenecks in cloud-native software suggests using concepts such as speculation and data coherence support to speed-up micro services and serverless functions. This talk will examine the software overheads of cloud-native workloads and present some ideas to mitigate them.

Josep Torrellas is the Thomas M. Siebel Chair in Computer Science at the University of Illinois at Urbana-Champaign (UIUC). He is the Director of the ACE Center for Evolvable Computing (an SRC/DARPA JUMP 2.0 Center), past Co-Leader of an Intel Strategic Research Alliance (ISRA) on Computer Security, and past Director of the Illinois-Intel Parallelism Center (I2PC). His research interests are multiprocessor computer architectures and parallel computing. Some of his contributions include thread-level speculation (TLS) architectures, the Bulk Multiprocessor concept, deterministic record and replay mechanisms, process variation mitigation techniques, and hardware defenses against speculative execution attacks. In addition, he has contributed to several experimental multiprocessor designs such as IBM’s PERCS Multiprocessor, Intel’s Runnemede Extreme-Scale Multiprocessor, Illinois Cedar, and Stanford DASH. Torrellas has received several research awards including the IEEE CS Harry H. Goode Memorial Award. He is a Fellow of IEEE, ACM, and AAAS. He received a PhD from Stanford University.

The rise of agentic AI is transforming the software landscape of modern datacenters. This panel brings together experts from academia and industry to discuss how the cloud-native stack—from datacenter infrastructure and hardware platforms to operating systems, runtimes, and orchestration frameworks—must evolve to support these emerging workloads. We will also explore how agentic AI can become a powerful tool for designing, optimizing, and managing future cloud infrastructure itself.