Publications

As machine learning models scale in size and complexity, their computational
requirements become a significant barrier. Mixture-of-Experts (MoE) models
alleviate this issue by selectively activating relevant experts. Despite this,
MoE models are hindered by high communication overhead from all-to-all
operations, low GPU utilization due to the synchronous communication
constraint, and complications from heterogeneous GPU environments.
This paper presents Aurora, which optimizes both model deployment and
all-to-all communication scheduling to address these challenges in MoE
inference. Aurora achieves minimal communication times by strategically
ordering token transmissions in all-to-all communications. It improves GPU
utilization by colocating experts from different models on the same device,
avoiding the limitations of synchronous all-to-all communication. We analyze
Aurora's optimization strategies theoretically across four common GPU cluster
settings: exclusive vs. colocated models on GPUs, and homogeneous vs.
heterogeneous GPUs. Aurora provides optimal solutions for three cases, and for
the remaining NP-hard scenario, it offers a polynomial-time sub-optimal
solution with only a 1.07x degradation from the optimal.
Aurora is the first approach to minimize MoE inference time via optimal model
deployment and communication scheduling across various scenarios. Evaluations
demonstrate that Aurora significantly accelerates inference, achieving speedups
of up to 2.38x in homogeneous clusters and 3.54x in heterogeneous environments.
Moreover, Aurora enhances GPU utilization by up to 1.5x compared to existing
methods.

Optical data center networks (DCNs) are emerging as a promising design for
cloud infrastructure. However, existing optical DCN architectures operate as
closed ecosystems, tying software solutions to specific optical hardware. We
introduce Lighthouse, an open research framework that decouples software from
hardware, allowing them to evolve independently. Central to Lighthouse is the
time-flow table abstraction, serving as a common interface between optical
hardware and software. We develop Lighthouse on programmable switches,
achieving a minimum optical circuit duration of 2 {\mu}s, the shortest duration
realized by commodity devices to date. We demonstrate Lighthouse's generality
by implementing six optical architectures on an optical testbed and conducted
extensive benchmarks on a 108-ToR setup, highlighting system efficiency.
Additionally, we present case studies that identify potential research topics
enabled by Lighthouse.

Optical data center networks (DCNs) are renovating the infrastructure design
for the cloud in the post Moore's law era. The fact that optical DCNs rely on
optical circuits of microsecond-scale durations makes nanosecond-precision time
synchronization essential for the correct functioning of routing on the network
fabric. However, current studies on optical DCNs neglect the fundamental need
for accurate time synchronization. In this paper, we bridge the gap by
developing Nanosecond Optical Synchronization (NOS), the first
nanosecond-precision synchronization solution for optical DCNs general to
various optical hardware. NOS builds clock propagation trees on top of the
dynamically reconfigured circuits in optical DCNs, allowing switches to seek
better sync parents throughout time. It predicts drifts in the tree-building
process, which enables minimization of sync errors. We also tailor today's sync
protocols to the needs of optical DCNs, including reducing the number of sync
messages to fit into short circuit durations and correcting timestamp errors
for higher sync accuracy. Our implementation on programmable switches shows
28ns sync accuracy in a 192-ToR setting.

Optical data center networks (DCNs) are emerging as a promising solution for
cloud infrastructure in the post-Moore's Law era, particularly with the advent
of 'fast-switched' optical architectures capable of circuit reconfiguration at
microsecond or even nanosecond scales. However, frequent reconfiguration of
optical circuits introduces a unique challenge: in-flight packets risk loss
during these transitions, hindering the deployment of many mature optical
hardware designs due to the lack of suitable routing solutions. In this paper,
we present Unified Routing for Optical networks (URO), a general routing
framework designed to support fast-switched optical DCNs across various
hardware architectures. URO combines theoretical modeling of this novel routing
problem with practical implementation on programmable switches, enabling
precise, time-based packet transmission. Our prototype on Intel Tofino2
switches achieves a minimum circuit duration of 2us, ensuring end-to-end,
loss-free application performance. Large-scale simulations using production DCN
traffic validate URO's generality across different hardware configurations,
demonstrating its effectiveness and efficient system resource utilization.