Scheduling online repartitioning in OLTP systems

TY - CHAP

T1 - Scheduling online repartitioning in OLTP systems

AU - Chen, Kaiji

AU - Zhou, Yongluan

AU - Cao, Yu

N1 - Conference code: 15

PY - 2014/12/8

Y1 - 2014/12/8

N2 - Previous studies on automatic database partitioning mostly focus on optimizing the (re)partitioning scheme for a given database and its query workload, while overseeing the problem about how to efficiently deploy the partition scheme onto the database system, which is, however, often non-trivial and challenging, especially in a distributed OLTP system where repartitioning is expected to take place online without interfering the user transactions. In this paper, we propose SOAP, a system framework for scheduling online database repartitioning for OLTP workloads. SOAP aims to minimize the time frame of executing the repartition operations while guaranteeing the correctness and performance of user transactions. It models and groups the repartition operations into repartition transactions, and then mixes them with the normal transactions for holistic scheduling optimization. SOAP utilizes a cost-based approach to prioritize the repartition transactions, and leverages a feedback model in control theory to determine in which order and at which frequency the repartition transactions should be scheduled for execution. When the system is under heavy workloads, selected repartition operations would piggyback onto the normal transactions to mitigate the repartitioning overhead. We have built a SOAP prototype on top of PostgreSQL and running at Amazon EC2, and conducted a comprehensive experimental study validating SOAP's significant performance advantages.

AB - Previous studies on automatic database partitioning mostly focus on optimizing the (re)partitioning scheme for a given database and its query workload, while overseeing the problem about how to efficiently deploy the partition scheme onto the database system, which is, however, often non-trivial and challenging, especially in a distributed OLTP system where repartitioning is expected to take place online without interfering the user transactions. In this paper, we propose SOAP, a system framework for scheduling online database repartitioning for OLTP workloads. SOAP aims to minimize the time frame of executing the repartition operations while guaranteeing the correctness and performance of user transactions. It models and groups the repartition operations into repartition transactions, and then mixes them with the normal transactions for holistic scheduling optimization. SOAP utilizes a cost-based approach to prioritize the repartition transactions, and leverages a feedback model in control theory to determine in which order and at which frequency the repartition transactions should be scheduled for execution. When the system is under heavy workloads, selected repartition operations would piggyback onto the normal transactions to mitigate the repartitioning overhead. We have built a SOAP prototype on top of PostgreSQL and running at Amazon EC2, and conducted a comprehensive experimental study validating SOAP's significant performance advantages.

U2 - 10.1145/2676727.2676731

DO - 10.1145/2676727.2676731

M3 - Book chapter

SN - 978-1-4503-3219-4

BT - Middeware’14 Industry Short Papers

PB - Association for Computing Machinery

T2 - 15th ACM/IFIP/USENIX International Conference on Middleware

Y2 - 11 December 2014 through 11 December 2014

ER -