Scenarios / Robotaxi rebalancing

AI for Real-Time Rebalancing of Robotaxi Fleets

Where should the idle robotaxis go?

Shared objective

Find a robotaxi for as many riders as possible, with few empty trips

Each district observes

Idle robotaxis, ride requests, riders without a car and robotaxis on the way, in every district

Each district decides

Where each of its idle robotaxis goes, every 5 minutes

Six hours of shifting demand

0:00

Bars: riders without a car in each 5 minutes, with planned rebalancing above the line and with AI rebalancing below. Click to jump to a moment.

With planned rebalancing

Fixed plan

Every 5 minutes, idle robotaxis are sent towards district targets planned once from the long-run average demand.

With AI rebalancing

Adaptive

Every 5 minutes, each district decides where its idle robotaxis go, from what it observes across the city.

How to train such an AI efficiently?

Why conventional multi-agent training struggles at scale

One agent's share of the shared reward

Illustration with equal contributions: the blue slice is what agent i's own action causes.

Noise in each agent's learning signal

With conventional multi-agent reinforcement learning, every district learns from the same reward, in which its own effect shrinks as the city grows.

How our algorithm trains the districts, step by step

Once the demand pattern is known, a fluid model of the fleet tells how many robotaxis each district needs so that departures and arrivals balance everywhere. Since a robotaxi sent now arrives up to 20 minutes later, what matters is where the fleet will be 20 minutes ahead. Could this knowledge be internalized into how the districts are trained?

Referenceoffline, centralized

{\tilde{x}}_{j} (t) = \sum_{k} (λ_{k j}^{p} + r_{k j}^{p}) τ_{k j} + b_{j}, p = pattern at t + 20 min

Robotaxis that should be in or on their way to district j: λ^p are the ride rates of the pattern, r^p the empty-car flows that balance every district's departures and arrivals in the fluid model of Pavone et al. (2012), τ the travel times and b_j an idle buffer.

Guidanceper district

g_{i} (t) = \frac{1}{2} \sum_{j} {(\frac{x_{j} (t) - {\tilde{x}}_{j} (t)}{s})}^{2} - \frac{1}{2} \sum_{j} {(\frac{x_{j}^{- i} (t) - {\tilde{x}}_{j} (t)}{s})}^{2}

x_j(t): robotaxis in district j 20 minutes after this dispatch, counting the idle ones kept there and those arriving within 20 minutes; x⁻ⁱ: the same had district i kept all its idle robotaxis; s: mean requests per district and 5 minutes. Negative when district i's dispatch brings the fleet closer to the reference.

Learning signalper district

{\hat{A}}_{i} (t) = (1 - α) A (t) - α {\hat{g}}_{i} (t)

Shared advantage A of the reward r = −(riders without a car + 0.1 × empty robotaxi-steps) / 100, combined with the normalized guidance; α = 0.9.

Policy updateinternalizes the reference

\nabla_{θ} J \approx \sum_{t} \sum_{i} {\hat{A}}_{i} (t) \sum_{j} n_{i j} (t) \nabla_{θ} \log p_{i j} (o (t))

District i sends each idle robotaxi to district j with probability p_ij, and n_ij robotaxis go there; the reference and the guidance are not needed at run time.

At its core, our algorithm uses prior knowledge of the system, in the form of a reference state, to guide the training of the districts and give each district a clear optimization signal. Through training, each district internalizes in its policy where to send its idle robotaxis as demand shifts across the city.