Shuhao Zhang

Tel: (65)64997153

About Me (CV): I have been an Assistant Professor at SUTD since 2021. Before that, I did a postdoc from 2020 to 2021 with Prof. Volker Markl (founder of Apache Flink) at TU Berlin. I obtained my PhD and Bachelor’s degrees from NUS (2019) and NTU (2014), respectively. My research emphasizes the design of database systems and big data processing frameworks, with a special interest in high-performance stream processing systems. I regularly serve on the program committees of HPC venues such as SC, ICDCS, and ICPP, and DB/DM venues like ICDE, KDD, and EDBT. I will join SCSE at NTU as an assistant professor in November 2023. I work on Parallel and Distributed Computing, Database & ML systems, and Data-Centric Machine Learning, with a key theme of “event stream processing.”

Important Annoucement (13/Sep/2023): Since announced in early August, we have received few hundres of emails querying PhD positions for 2024 Intake. We are now pleased to announce that we have identified a sufficient number of qualified candidates. These selected applicants have been invited to participate in a visiting student program at our Lab this coming November. Upon successful completion of this program, they are expected to formally join our Lab as PhD students. We would like to express our gratitude to all who have shown interest in our program. Please be advised that we have ceased accepting new PhD applications for this intake cycle, except in the event that any of the already-identified candidates decline our offer. However, we continue to offer opportunities for Research Assistant and visiting student positions, for which the interview process is ongoing. Interested individuals are encouraged to reach out to us via email for additional information.

I lead the IntelliStream Team: We are a systems research group. From a high-level perspective, our research goal is to optimize and employ distributed and parallel stream processing technology to better support existing areas (e.g., databases, big data analytics) and emerging big data applications (e.g., stateful NFV, fast continual learning). This is vital for improving performance and reducing resource consumption, especially in the network-connected world supported by technologies like 5G, IoT, etc.

Selected Publications

Author notations: ∗ denotes the author is a student advised by me. # denotes the author is a staff advised by me.

SIGMOD
Data Stream Clustering: An In-Depth Empirical Study

Xin Wang^*, Zhengru Wang^*, Zhenyu Wu^#, Shuhao Zhang, Xuanhua Shi, and Li Lu

Proc. ACM Manag. Data Jun 2023

Abs Bib PDF Code

Data Stream Clustering (DSC) plays an important role in mining continuous and unlabeled data streams in real-world applications. Over the last decades, numerous DSC algorithms have been proposed with promising clustering accuracy and efficiency. Despite the significant differences among existing DSC algorithms, they are commonly built around four key design aspects: summarizing data structure, window model, outlier detection mechanism, and offline refinement strategy. However, there is a lack of empirical studies on these key design aspects in the same codebase using real-world workloads with distinct characteristics. As a result, it is difficult for researchers to improve upon the state-of-the-art. In this paper, we conduct such a study of DSC on its four key design aspects. We implemented state-of-the-art variants of all of these design choices in an open-sourced platform from scratch and evaluated them using both real-world and synthetic workloads. Our analysis identifies the fundamental issues and trade-offs of each design choice in terms of both accuracy and efficiency. We even find that combining flexible design choices led to the development of a new algorithm called Benne, which can be tuned to achieve either better accuracy or better efficiency compared to the state-of-the-art.
@article{xin2023dsc, title = {Data Stream Clustering: An In-Depth Empirical Study}, author = {Wang*, Xin and Wang*, Zhengru and Wu#, Zhenyu and Zhang, Shuhao and Shi, Xuanhua and Lu, Li}, year = {2023}, month = jun, journal = {Proc. ACM Manag. Data}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, volume = {1}, number = {2}, doi = {10.1145/3589307}, url = {https://doi.org/10.1145/3589307}, issue_date = {June 2023}, articleno = {162}, numpages = {26}, keywords = {clustering algorithm, data stream, empirical study}, tag = {full paper} }
SIGMOD
MorphStream: Adaptive Scheduling for Scalable Transactional Stream Processing on Multicores

Yancan^# Mao, Jianjun Zhao, Shuhao Zhang, Haikun Liu, and Volker Markl

Proc. ACM Manag. Data May 2023

Abs Bib PDF Code

Transactional stream processing engines (TSPEs) differ significantly in their designs, but all rely on non- adaptive scheduling strategies for processing concurrent state transactions. Subsequently, none exploit multicore parallelism to its full potential due to complex workload dependencies. This paper introduces MorphStream, which adopts a novel approach by decomposing scheduling strategies into three dimensions and then strives to make the right decision along each dimension, based on analyzing the decision trade-offs under varying workload characteristics. Compared to the state-of-the-art, MorphStream achieves up to 3.4 times higher throughput and 69.1% lower processing latency for handling real-world use cases with complex and dynamically changing workload dependencies.
@article{mao2023morphstream, title = {MorphStream: Adaptive Scheduling for Scalable Transactional Stream Processing on Multicores}, author = {Mao, Yancan# and Zhao, Jianjun and Zhang, Shuhao and Liu, Haikun and Markl, Volker}, year = {2023}, month = may, journal = {Proc. ACM Manag. Data}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, volume = {1}, number = {1}, doi = {10.1145/3588913}, url = {https://doi.org/10.1145/3588913}, issue_date = {May 2023}, articleno = {59}, numpages = {26}, keywords = {transaction, stream processing, multicore}, tag = {full paper}, }

ICDE

Scalable Online Interval Join on Modern Multicore Processors in OpenMLDB

Hao Zhang, Xianzhi Zeng^*, Shuhao Zhang, Xinyi Liu, Mian Lu, and Zhao Zheng

In 2023 IEEE 39rd International Conference on Data Engineering (ICDE) May 2023

Bib PDF Code

@inproceedings{OIJ,
  title = {Scalable Online Interval Join on Modern Multicore Processors in OpenMLDB},
  author = {Zhang, Hao and Zeng<sup>*</sup>, Xianzhi and Zhang, Shuhao and Liu, Xinyi and Lu, Mian and Zheng, Zhao},
  year = {2023},
  booktitle = {2023 IEEE 39rd International Conference on Data Engineering (ICDE)},
  volume = {},
  number = {},
  pages = {},
  doi = {},
  tag = {full paper}
}

ICDE

Parallelizing Stream Compression for IoT Applications on Asymmetric Multicores

Xianzhi Zeng^*, and Shuhao Zhang

In 2023 IEEE 39rd International Conference on Data Engineering (ICDE) May 2023

Bib PDF Code

@inproceedings{CStream,
  title = {Parallelizing Stream Compression for IoT Applications on Asymmetric Multicores},
  author = {Zeng<sup>*</sup>, Xianzhi and Zhang, Shuhao},
  year = {2023},
  booktitle = {2023 IEEE 39rd International Conference on Data Engineering (ICDE)},
  volume = {},
  number = {},
  pages = {},
  doi = {},
  tag = {full paper}
}

SIGMOD

Parallelizing Intra-Window Join on Multicores: An Experimental Study

Shuhao Zhang, Yancan Mao, Jiong He, Philipp M. Grulich, Steffen Zeuch, Bingsheng He, Richard T. B. Ma, and Volker Markl

In Proceedings of the 2021 International Conference on Management of Data (SIGMOD) May 2021

Bib PDF Code

@inproceedings{10.1145/1.1,
  title = {Parallelizing Intra-Window Join on Multicores: An Experimental Study},
  author = {Zhang, Shuhao and Mao, Yancan and He, Jiong and Grulich, Philipp M. and Zeuch, Steffen and He, Bingsheng and Ma, Richard T. B. and Markl, Volker},
  year = {2021},
  booktitle = {Proceedings of the 2021 International Conference on Management of Data (SIGMOD)},
  location = {Xi'an, Shaanxi, China},
  publisher = {Association for Computing Machinery},
  address = {New York, NY, USA},
  series = {SIGMOD '21},
  url = {https://dl.acm.org/doi/10.1145/3448016.3452793},
  tag = {full paper}
}

ICDE
Towards Concurrent Stateful Stream Processing on Multicore Processors

Shuhao Zhang, Yingjun Wu, Feng Zhang, and Bingsheng He

In 2020 IEEE 36th International Conference on Data Engineering (ICDE) May 2020

Abs Bib PDF Code Slides

Recent data stream processing systems (DSPSs) can achieve excellent performance when processing large volumes of data under tight latency constraints. However, they sacrifice support for concurrent state access that eases the burden of developing stateful stream applications. Recently, some have proposed managing concurrent state access during stream processing by modeling state accesses as transactions. However, these are realized with locks involving serious contention overhead. The coarse-grained processing paradigm adopted in these proposals magnify contention issues and does not exploit modern multicore architectures to their full potential. This paper introduces TStream, a novel DSPS supporting efficient concurrent state access on multicore processors. Transactional semantics is employed like previous work, but scalability is greatly improved due to two novel designs: 1) dual-mode scheduling, which exposes more parallelism opportunities, 2) dynamic restructuring execution, which aggressively exploits the parallelism opportunities from dual-mode scheduling without centralized lock contentions. To validate our proposal, we evaluate TStream with a benchmark of four applications on a modern multicore machine. Experimental results show that 1) TStream achieves up to 4.8 times higher throughput with similar processing latency compared to the state-of-the-art and 2) unlike prior solutions, TStream is highly tolerant of varying application workloads such as key skewness and multi-partition state accesses.
@inproceedings{9101749, title = {Towards Concurrent Stateful Stream Processing on Multicore Processors}, author = {{Zhang}, Shuhao and {Wu}, Yingjun and {Zhang}, Feng and {He}, Bingsheng}, year = {2020}, booktitle = {2020 IEEE 36th International Conference on Data Engineering (ICDE)}, volume = {}, number = {}, pages = {1537--1548}, doi = {10.1109/ICDE48307.2020.00136}, tag = {full paper} }
SIGMOD
BriskStream: Scaling Data Stream Processing on Shared-Memory Multicore Architectures

Shuhao Zhang, Jiong He, Amelie Chi Zhou, and Bingsheng He

In Proceedings of the 2019 International Conference on Management of Data (SIGMOD) May 2019

Abs Bib PDF Code

We introduce BriskStream, an in-memory data stream processing system (DSPSs) specifically designed for modern shared-memory multicore architectures. BriskStream’s key contribution is an execution plan optimization paradigm, namely RLAS, which takes relative-location (i.e., NUMA distance) of each pair of producer-consumer operators into consideration. We propose a branch and bound based approach with three heuristics to resolve the resulting nontrivial optimization problem. The experimental evaluations demonstrate that BriskStream yields much higher throughput and better scalability than existing DSPSs on multi-core architectures when processing different types of workloads.
@inproceedings{10.1145/3299869.3300067, title = {BriskStream: Scaling Data Stream Processing on Shared-Memory Multicore Architectures}, author = {Zhang, Shuhao and He, Jiong and Zhou, Amelie Chi and He, Bingsheng}, year = {2019}, booktitle = {Proceedings of the 2019 International Conference on Management of Data (SIGMOD)}, location = {Amsterdam, Netherlands}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, series = {SIGMOD '19}, pages = {705–722}, doi = {10.1145/3299869.3300067}, isbn = {9781450356435}, url = {https://doi.org/10.1145/3299869.3300067}, numpages = {18}, keywords = {numa-awareness, operator replication and placement}, tag = {full paper} }

ICDE

Multi-Query Optimization for Complex Event Processing in SAP ESP

Shuhao Zhang, H. T. Vo, D. Dahlmeier, and B. He

In 2017 IEEE 33rd International Conference on Data Engineering (ICDE) May 2017

Bib PDF

@inproceedings{7930061,
  title = {Multi-Query Optimization for Complex Event Processing in SAP ESP},
  author = {{Zhang}, Shuhao and {Vo}, H. T. and {Dahlmeier}, D. and {He}, B.},
  year = {2017},
  booktitle = {2017 IEEE 33rd International Conference on Data Engineering (ICDE)},
  volume = {},
  number = {},
  pages = {1213--1224},
  doi = {10.1109/ICDE.2017.166},
  tag = {full paper},
}

ICDE

Revisiting the Design of Data Stream Processing Systems on Multi-Core Processors

Shuhao Zhang, B. He, D. Dahlmeier, A. C. Zhou, and T. Heinze

In 2017 IEEE 33rd International Conference on Data Engineering (ICDE) May 2017

Bib PDF

@inproceedings{7930015,
  title = {Revisiting the Design of Data Stream Processing Systems on Multi-Core Processors},
  author = {{Zhang}, Shuhao and {He}, B. and {Dahlmeier}, D. and {Zhou}, A. C. and {Heinze}, T.},
  year = {2017},
  booktitle = {2017 IEEE 33rd International Conference on Data Engineering (ICDE)},
  volume = {},
  number = {},
  pages = {659--670},
  doi = {10.1109/ICDE.2017.119},
  tag = {full paper},
}