Materialized view selection in feed following systems

TY - GEN

T1 - Materialized view selection in feed following systems

AU - Chen, Kaiji

AU - Zhou, Yongluan

PY - 2016

Y1 - 2016

N2 - Recently emerging feed-following applications generate personalized event streams from various feeds and deliver them to a large number of users. To provide a low-latency service, a feed-following system has to buffer the events in a number of tables, called materialized views, and choosing views to materialize is critical to the system performance. State-of-the-art solutions only consider view selections for each individual user. Due to the existence of very popular feeds and social communities, users often share a lot of feeds that they follow and hence performing a global optimization by considering all the users can significantly enhance the system performance. However, performing such a global optimization needs to choose views for materialization from an exponential number of possible ones. To solve the issue, we propose an effective method to generate candidate views that are potentially beneficial. We then propose several cost-based algorithms to solve the global view selection problem, which adopt a cost model that captures the cost of both user query processing and view maintenance and make use of the containment relationships among the sets of feeds followed by the individual users. We implement the complete approach in a prototype system and perform experiments on a computing cluster using both real and synthetic data. The results indicate that our approach outperforms the state-of-the-art approaches significantly.

AB - Recently emerging feed-following applications generate personalized event streams from various feeds and deliver them to a large number of users. To provide a low-latency service, a feed-following system has to buffer the events in a number of tables, called materialized views, and choosing views to materialize is critical to the system performance. State-of-the-art solutions only consider view selections for each individual user. Due to the existence of very popular feeds and social communities, users often share a lot of feeds that they follow and hence performing a global optimization by considering all the users can significantly enhance the system performance. However, performing such a global optimization needs to choose views for materialization from an exponential number of possible ones. To solve the issue, we propose an effective method to generate candidate views that are potentially beneficial. We then propose several cost-based algorithms to solve the global view selection problem, which adopt a cost model that captures the cost of both user query processing and view maintenance and make use of the containment relationships among the sets of feeds followed by the individual users. We implement the complete approach in a prototype system and perform experiments on a computing cluster using both real and synthetic data. The results indicate that our approach outperforms the state-of-the-art approaches significantly.

U2 - 10.1109/BigData.2016.7840635

DO - 10.1109/BigData.2016.7840635

M3 - Article in proceedings

SN - 978-1-4673-9006-4

SP - 442

EP - 451

BT - Proceedings of the 2016 IEEE International Conference on Big Data

PB - IEEE Press

T2 - 2016 IEEE International Conference on Big Data

Y2 - 5 December 2015 through 8 December 2015

ER -