On-demand ride services or ride-sourcing services have been experiencing fast development and steadily reshaping the way people travel in the past decade. Various optimization algorithms, including reinforcement learning approaches, have been developed to help ride-sourcing platforms design better operational strategies to achieve higher efficiency. However, due to cost and reliability issues, it is commonly infeasible to validate these models and train/test these optimization algorithms within real-world ride-sourcing platforms. Acting as a proper test bed, a simulation platform for ride-sourcing systems will thus be essential for both researchers and industrial practitioners. While previous studies have established simulators for their tasks, they lack a fair and public platform for comparing the models/algorithms proposed by different researchers. In addition, the existing simulators still face many challenges, ranging from their closeness to real environments of ride-sourcing systems to the completeness of tasks they can implement. To address the challenges, researchers from the Hong Kong University of Science and Technology, the University of Hong Kong, and the Hong Kong Polytechnic University propose a novel simulation platform for ride-sourcing systems on real transportation networks. It provides a few accessible portals to train and test various optimization algorithms, especially reinforcement learning algorithms, for a variety of tasks, including on-demand matching, idle vehicle repositioning, and dynamic pricing. Evaluated on real-world data-based experiments, the simulator is demonstrated to be an efficient and effective test bed for various tasks related to on-demand ride service operations.
They published their study in Communications in Transportation Research .
"We develop a multi-functional simulation platform for on-demand ride service operations, which maintains pricing module, matching module, repositioning module, etc." says Jintao Ke, an Assistant Professor at the Department of Civil Engineering at the University of Hong Kong.
Simulation Framework
The simulation framework is represented in Fig. 2, which is comprised of five major components, including input data, agent properties, platform operation process, tasks, and visualization. The input data represents the fundamental setting of the simulation environment, under which all the other modules are implemented. Basically, these settings do not change once determined before experiments, unless there are special requirements.
"In addition to input data, we also define properties for passengers, drivers, and platforms within the simulation framework. These properties may change with the evolution of the state of the ride-sourcing market." says Siyuan Feng, a Research Assistant Professor at the Department of Logistics and Maritime Studies at the Hong Kong Polytechnic University.
With the input data and agent properties given, the operation process of the platform can be constructed. During each time interval for simulation, the platform first conducts order dispatching by collecting idle vehicles and waiting passengers. Afterward, the matching result is presented to both passengers and drivers for their reaction. In this step, some matched passengers and drivers may find the pick-up time too long, and thus abandon the matching. On the other hand, the unmatched drivers may choose to continue waiting or drop the request based on their maximum waiting time. Simultaneously, new orders will be generated and priced based on the given demand patterns, the current time interval, and the pricing rule by the platform. In addition, the unmatched drivers can also chose to idle, cruising to some other areas for new orders, or accept guidance from platforms to reposition to some given areas. In addition, the drivers may also choose to drop out of the platform based on the set patterns. Finally, all the properties for passengers, drivers, and platforms are updated for the next time interval, and the whole process is repeated until the end of the simulation period. Specifically, they also build a routing module to compute road node-based routes given OD pairs frequently utilized by other simulated operations, as shown in Fig. 2.
"The proposed framework provides a simulation environment to interact with different research or industry practice tasks. As a result, the simulator is designed to flexibly accommodate the needs of the tasks, such as RL, for operations." says Taijie Chen, a Ph.D. candidate at the Department of Civil Engineering at the University of Hong Kong.
Moreover, they also develop a visualization module to represent the market status from different perspectives of demand and supply. In summary, a simulation framework is proposed to depict the whole process of the ride-sourcing market. The framework is highly complete, modularized, and user-friendly for task implementation and result representation. All the details of the simulator are open-source.
"Researchers can use the simulator as a shared test bed for model validation, algorithm training, testing, comparison, and demonstration. For industrial practitioners (e.g., ride-sourcing platforms and other on-demand service providers), the simulator is a free and flexible tool for algorithm pre-training for A/B tests or result representation to non-experts, such as government administration or business investors." says Hai Yang, a Chair Professor at the Department of Civil and Environmental Engineering at the Hong Kong University of Science and Technology.
Validations of the simulation platform
This research conducts an experiment based on the proposed simulator to further validate its capability of approximating the real-world ride-sourcing market.
They select and process a taxi dataset of Hong Kong for the experiment, with complete information of drivers and passengers in the market, such as order details and the continuous records of drivers' tracks. Based on the historical information, some metrics can be formulated to evaluate the accuracy of the proposed simulation. They employ the average utilization rate of vehicles as metric.
With the metrics determined, the utilization rate is first calculated based on the real track records and then generated via the simulation. For some specifications in the simulation, the average vehicle speed is set to 23km/h, and the maximal pick-up distance is 5km. The fleet size and spatial distribution are also calibrated based on real taxi data. For cruising rules, when drivers are not matched, they set them to randomly cruise to nearby areas with larger historical demand to mimic the drivers' experience for order searching. In addition, they utilize Ur– UsUr as the error between real and simulated utilization rates of vehicles, which are respectively represented by Ur and Us. The error is only around 0.17. The result validates that the proposed simulator can effectively fit the real-world data after calibration, which is fundamental for the following applications in the next section.
Experiments for reinforcement learning-based tasks
"To show the proposed simulator's capability to implement different reinforcement learning (RL) tasks for ride-sourcing services. We mainly focus on matching and idle vehicle repositioning RL tasks." says Siyuan Feng, a Research Assistant Professor at the Department of Logistics and Maritime Studies at the Hong Kong Polytechnic University.
For the matching task, the tested RL method outperforms the best of the other two baselines in platform revenue and occupancy rates by 10.8% and 6.4%, respectively. Although the RL method's matching time and pickup time metrics are not the best, the excessive part of these two metrics is still acceptable for passengers. For the repositioning task, the tested RL method is superior to the baseline for all the metrics. In particular, it outperforms the baseline in platform revenue and occupancy rate by 12.9% and 6.1%, demonstrating its capability to guide vehicle movement more effectively. In summary, users can train, test, and analyze the RL algorithms based on the proposed simulator in an efficient way. It is worth mentioning that other RL approaches can also be trained and tested quickly on our open-sourced simulation platform for different operations of ride-sourcing services.
The above research is published in Communications in Transportation Research (COMMTR), which is a fully open access journal co-published by Tsinghua University Press and Elsevier. COMMTR publishes peer-reviewed high-quality research representing important advances of significance to emerging transport systems. COMMTR is also among the first transportation journals to make the Replication Package mandatory to facilitate researchers, practitioners, and the general public in understanding and advancing existing knowledge. At its discretion, Tsinghua University Press will pay the open access fee for all published papers from 2024 to 2025.
About Communications in Transportation Research
Communications in Transportation Research was launched in 2021, with academic support provided by Tsinghua University and China Intelligent Transportation Systems Association. The Editors-in-Chief are Professor Xiaobo Qu, a member of the Academia Europaea from Tsinghua University and Professor Shuai'an Wang from Hong Kong Polytechnic University. The journal mainly publishes high-quality, original research and review articles that are of significant importance to emerging transportation systems, aiming to become an international platform and window for showcasing and exchanging innovative achievements in transportation and related fields, to promote the exchange and development of transportation research between China and the international academic community. It has been indexed in ESCI, Ei Compendex, Scopus, DOAJ, TRID and other databases. In 2022, it was selected as a high-starting-point new journal project of the "China Science and Technology Journal Excellence Action Plan". This year, it received the first impact factor of 12.5. The 2023 IF is 12.5, ranking in the Top1 (1/57, Q1) among all journals in "TRANSPORTATION" category. At its discretion, Tsinghua University Press will pay the open access fee for all published papers from 2024 to 2025.
