Progressive recovery of correlated failures in distributed stream processing engines

TY - GEN

T1 - Progressive recovery of correlated failures in distributed stream processing engines

AU - Su, Li

AU - Zhou, Yongluan

N1 - Conference code: 20

PY - 2017

Y1 - 2017

N2 - Correlated failures in large-scale clusters have significant effects on systems’ availability, especially for streaming data applications that run continuously and require low processing latency. Most state-of-the-art distributed stream processing engines (DSPEs) adopt a blocking recovery paradigm, which, upon correlated failure, would block the progress of recovery until sufficient new resources for recovery are available. As the arrival of new resources is usually progressive, a blocking paradigm fails to minimize the recovery latency. To address this problem, we propose a progressive and query-centric recovery paradigm where the recovery of the failed operators would be carefully scheduled to progressively recover the outputs of queries as early as possible based on the current availability of resources. In this work, we propose and implement a fault-tolerance framework which supports progressive recovery after correlated failures with minimum overhead during the system’s normal execution. We also formulate the new problem of recovery scheduling under correlated failures and design effective algorithms to optimize the recovery latency. The proposed methods are implemented on Apache Storm and preliminary experiments are conducted to verify their validity.

AB - Correlated failures in large-scale clusters have significant effects on systems’ availability, especially for streaming data applications that run continuously and require low processing latency. Most state-of-the-art distributed stream processing engines (DSPEs) adopt a blocking recovery paradigm, which, upon correlated failure, would block the progress of recovery until sufficient new resources for recovery are available. As the arrival of new resources is usually progressive, a blocking paradigm fails to minimize the recovery latency. To address this problem, we propose a progressive and query-centric recovery paradigm where the recovery of the failed operators would be carefully scheduled to progressively recover the outputs of queries as early as possible based on the current availability of resources. In this work, we propose and implement a fault-tolerance framework which supports progressive recovery after correlated failures with minimum overhead during the system’s normal execution. We also formulate the new problem of recovery scheduling under correlated failures and design effective algorithms to optimize the recovery latency. The proposed methods are implemented on Apache Storm and preliminary experiments are conducted to verify their validity.

U2 - 10.5441/002/edbt.2017.59

DO - 10.5441/002/edbt.2017.59

M3 - Article in proceedings

T3 - Advances in Database Technology

SP - 518

EP - 521

BT - Advances in Database Technology

A2 - Markl, Volker

A2 - Orlando, Salvatore

A2 - Mitschang, Bernhard

A2 - Andritsos, Periklis

A2 - Sattler, Kai-Uwe

A2 - Breß, Sebastian

PB - OpenProceedings.org

T2 - 20th International Conference on Extending Database Technology

Y2 - 21 March 2017 through 24 March 2017

ER -