Selected Publications (full list)

2022

1. TON

   Co-Scheduler: A Coflow-Aware Data-Parallel Job Scheduler in Hybrid Electrical/Optical Datacenter Networks

   Zhuozhao Li, and Haiying Shen

   IEEE/ACM Transactions on Networking, 2022

   Abs Bib URL PDF

   To support higher demand for datacenter networks, networking researchers have proposed hybrid electrical/optical datacenter networks (Hybrid-DCN) that leverages optical circuit switching (OCS) along with traditional electrical packet switching (EPS). However, due to the high reconfiguration delay of OCS, OCS is used only for bulk data transfers between racks to amortize the reconfiguration delay. Existing job schedulers for data-parallel frameworks are not designed for Hybrid-DCN, since they neither place tasks to aggregate data traffic to take advantage of OCS, nor schedule tasks to minimize the Coflow completion time (CCT). In this paper, we describe the mismatch between existing job schedulers and the advanced Hybrid-DCN, introduce the requirements for the new scheduler, and present the implementation of Co-scheduler , a job scheduler for data-parallel frameworks that aims to improve job performance by placing the tasks of jobs to aggregate enough data traffic to better leverage OCS to minimize the CCT in Hybrid-DCN. Specifically, for every job, Co-scheduler computes guidelines on how many racks to place the job’s input data and the job’s tasks. The guidelines are dynamically generated based on the real-time job characteristics or predictable job characteristics from prior runs, with the aim of leveraging OCS whenever possible and efficient and minimizing CCT of jobs. Co-scheduler then schedules the tasks of jobs based on the guidelines. We evaluate the effectiveness of Co-scheduler using trace-driven simulation. The evaluation demonstrates that Co-scheduler can improve makespan, average job completion time, and average CCT of a workload by up to 56%, 61%, and 79%, respectively, compared to the state-of-the-art schedulers.

   @article{li2022coscheduler,
     author = {Li, Zhuozhao and Shen, Haiying},
     journal = {IEEE/ACM Transactions on Networking},
     title = {Co-Scheduler: A Coflow-Aware Data-Parallel Job Scheduler in Hybrid Electrical/Optical Datacenter Networks},
     year = {2022},
     volume = {30},
     number = {4},
     pages = {1599-1612},
     doi = {10.1109/TNET.2022.3143232},
     abbr = {TON},
     bibtex_show = {true},
     url = {https://ieeexplore.ieee.org/abstract/document/9695973},
     pdf = {TON-li-coscheduler-2022.pdf},
     selected = {true}
   }

2. TPDS

   funcX: Federated Function as a Service for Science

   Zhuozhao Li, Ryan Chard, Yadu Babuji, Ben Galewsky, Tyler J. Skluzacek, Kirill Nagaitsev, Anna Woodard, Ben Blaiszik, Josh Bryan, Daniel S. Katz, Ian Foster, and Kyle Chard

   IEEE Transactions on Parallel and Distributed Systems, 2022

   Abs Bib URL PDF

   funcX is a distributed function as a service (FaaS) platform that enables flexible, scalable, and high performance remote function execution. Unlike centralized FaaS systems, funcX decouples the cloud-hosted management functionality from the edge-hosted execution functionality. funcX’s endpoint software can be deployed, by users or administrators, on arbitrary laptops, clouds, clusters, and supercomputers, in effect turning them into function serving systems. funcX’s cloud-hosted service provides a single location for registering, sharing, and managing both functions and endpoints. It allows for transparent, secure, and reliable function execution across the federated ecosystem of endpoints—enabling users to route functions to endpoints based on specific needs. funcX uses containers (e.g., Docker, Singularity, and Shifter) to provide common execution environments across endpoints. funcX implements various container management strategies to execute functions with high performance and efficiency on diverse funcX endpoints. funcX also integrates with an in-memory data store and Globus for managing data that may span endpoints. We motivate the need for funcX, present our prototype design and implementation, and demonstrate, via experiments on two supercomputers, that funcX can scale to more than 130000 concurrent workers. We show that funcX’s container warming-aware routing algorithm can reduce the completion time for 3,000 functions by up to 61% compared to a randomized algorithm and the in-memory data store can speed up data transfers by up to 3x compared to a shared file system.

   @article{li2022funcx,
     author = {Li, Zhuozhao and Chard, Ryan and Babuji, Yadu and Galewsky, Ben and Skluzacek, Tyler J. and Nagaitsev, Kirill and Woodard, Anna and Blaiszik, Ben and Bryan, Josh and Katz, Daniel S. and Foster, Ian and Chard, Kyle},
     journal = {IEEE Transactions on Parallel and Distributed Systems},
     title = {funcX: Federated Function as a Service for Science},
     year = {2022},
     volume = {33},
     number = {12},
     pages = {4948-4963},
     doi = {10.1109/TPDS.2022.3208767},
     abbr = {TPDS},
     bibtex_show = {true},
     url = {https://ieeexplore.ieee.org/abstract/document/9899739},
     pdf = {TPDS-li-funcx-2022.pdf},
     selected = {true}
   }

3. JCIM

   High Throughput Virtual Screening and Validation of a SARS-CoV-2 Main Protease Non-Covalent Inhibitor

   Austin Clyde, Stephanie Galanie, Daniel W. Kneller, Heng Ma, Yadu Babuji, Ben Blaiszik, Alexander Brace, Thomas Brettin, Kyle Chard, Ryan Chard, Leighton Coates, Ian Foster, Darin Hauner, Vilmos Kertesz, Neeraj Kumar, Hyungro Lee, Zhuozhao Li, Andre Merzky, Jurgen G. Schmidt, Li Tan, Mikhail Titov, Anda Trifan, Matteo Turilli, Hubertus Van Dam, Srinivas C. Chennubhotla, Shantenu Jha, Andrey Kovalevsky, Arvind Ramanathan, Martha S. Head, and Rick Stevens

   Journal of Chemical Information and Modeling, 2022

   Abs Bib URL PDF

   Despite the recent availability of vaccines against the acute respiratory syndrome coronavirus 2 (SARS-CoV-2), the search for inhibitory therapeutic agents has assumed importance especially in the context of emerging new viral variants. In this paper, we describe the discovery of a novel non-covalent small-molecule inhibitor, MCULE-5948770040, that binds to and inhibits the SARS-Cov-2 main protease (Mpro) by employing a scalable high throughput virtual screening (HTVS) framework and a targeted compound library of over 6.5 million molecules that could be readily ordered and purchased. Our HTVS framework leverages the U.S. supercomputing infrastructure achieving nearly 91% resource utilization and nearly 126 million docking calculations per hour. Downstream biochemical assays validate this Mpro inhibitor with an inhibition constant (Ki) of 2.9 \textmuM [95% CI 2.2, 4.0]. Further, using room-temperature X-ray crystallography, we show that MCULE-5948770040 binds to a cleft in the primary binding site of Mpro forming stable hydrogen bond and hydrophobic interactions. We then used multiple \textmus-timescale molecular dynamics (MD) simulations, and machine learning (ML) techniques to elucidate how the bound ligand alters the conformational states accessed by Mpro, involving motions both proximal and distal to the binding site. Together, our results demonstrate how MCULE-5948770040 inhibits Mpro and offers a springboard for further therapeutic design.Significance StatementSignificance Statement The ongoing novel coronavirus pandemic (COVID-19) has prompted a global race towards finding effective therapeutics that can target the various viral proteins. Despite many virtual screening campaigns in development, the discovery of validated inhibitors for SARS-CoV-2 protein targets has been limited. We discover a novel inhibitor against the SARS-CoV-2 main protease. Our integrated platform applies downstream biochemical assays, X-ray crystallography, and atomistic simulations to obtain a comprehensive characterization of its inhibitory mechanism. Inhibiting Mpro can lead to significant biomedical advances in targeting SARS-CoV-2 treatment, as it plays a crucial role in viral replication.Competing Interest StatementThe authors have declared no competing interest.

   @article{clyde2021high,
     author = {Clyde, Austin and Galanie, Stephanie and Kneller, Daniel W. and Ma, Heng and Babuji, Yadu and Blaiszik, Ben and Brace, Alexander and Brettin, Thomas and Chard, Kyle and Chard, Ryan and Coates, Leighton and Foster, Ian and Hauner, Darin and Kertesz, Vilmos and Kumar, Neeraj and Lee, Hyungro and Li, Zhuozhao and Merzky, Andre and Schmidt, Jurgen G. and Tan, Li and Titov, Mikhail and Trifan, Anda and Turilli, Matteo and Van Dam, Hubertus and Chennubhotla, Srinivas C. and Jha, Shantenu and Kovalevsky, Andrey and Ramanathan, Arvind and Head, Martha S. and Stevens, Rick},
     title = {High Throughput Virtual Screening and Validation of a SARS-CoV-2 Main Protease Non-Covalent Inhibitor},
     journal = {Journal of Chemical Information and Modeling},
     volume = {62},
     number = {1},
     pages = {116-128},
     year = {2022},
     doi = {10.1021/acs.jcim.1c00851},
     note = {PMID: 34793155},
     url = {https://doi.org/10.1021/acs.jcim.1c00851},
     abbr = {JCIM},
     bibtex_show = {true},
     selected = {true},
     pdf = {JCIM-clyde-screening-2022.pdf}
   }

2021

1. IPDPS

   Lightweight Function Monitors for Fine-Grained Management in Large Scale Python Applications

   Tim Shaffer, Zhuozhao Li, Ben Tovar, Yadu Babuji, TJ Dasso, Zoe Surma, Kyle Chard, Ian Foster, and Douglas Thain

   In 2021 IEEE International Parallel and Distributed Processing Symposium (IPDPS), 2021

   Abs Bib URL PDF

   Python has become a widely used programming language for research, not only for small one-off analyses, but also for complex application pipelines running at supercomputer-scale. Modern parallel programming frameworks for Python present users with a more granular unit of management than traditional Unix processes and batch submissions: the Python function. We review the challenges involved in running native Python functions at scale, and present techniques for dynamically determining a minimal set of dependencies and for assembling a lightweight function monitor (LFM) that captures the software environment and manages resources at the granularity of single functions. We evaluate these techniques in a range of environments, from campus cluster to supercomputer, and show that our advanced dependency management planning and dynamic resource management methods provide superior performance and utilization relative to coarser-grained management approaches, achieving several-fold decrease in execution time for several large Python applications.

   @inproceedings{shaffer2021lightweight,
     author = {Shaffer, Tim and Li, Zhuozhao and Tovar, Ben and Babuji, Yadu and Dasso, TJ and Surma, Zoe and Chard, Kyle and Foster, Ian and Thain, Douglas},
     booktitle = {2021 IEEE International Parallel and Distributed Processing Symposium (IPDPS)},
     title = {Lightweight Function Monitors for Fine-Grained Management in Large Scale Python Applications},
     year = {2021},
     volume = {},
     number = {},
     pages = {786-796},
     keywords = {},
     doi = {10.1109/IPDPS49936.2021.00088},
     url = {https://ieeexplore.ieee.org/abstract/document/9460530},
     pdf = {IPDPS-shaffer-lfm-2021.pdf},
     bibtex_show = {true},
     issn = {1530-2075},
     abbr = {IPDPS},
     selected = {true},
     month = may
   }

2. JPDC

   DLHub: Simplifying publication, discovery, and use of machine learning models in science

   Zhuozhao Li, Ryan Chard, Logan Ward, Kyle Chard, Tyler J. Skluzacek, Yadu Babuji, Anna Woodard, Steven Tuecke, Ben Blaiszik, Michael J. Franklin, and Ian Foster

   Journal of Parallel and Distributed Computing, 2021

   Abs Bib URL PDF

   Machine Learning (ML) has become a critical tool enabling new methods of analysis and driving deeper understanding of phenomena across scientific disciplines. There is a growing need for “learning systems” to support various phases in the ML lifecycle. While others have focused on supporting model development, training, and inference, few have focused on the unique challenges inherent in science, such as the need to publish and share models and to serve them on a range of available computing resources. In this paper, we present the Data and Learning Hub for science (DLHub), a learning system designed to support these use cases. Specifically, DLHub enables publication of models, with descriptive metadata, persistent identifiers, and flexible access control. It packages arbitrary models into portable servable containers, and enables low-latency, distributed serving of these models on heterogeneous compute resources. We show that DLHub supports low-latency model inference comparable to other model serving systems including TensorFlow Serving, SageMaker, and Clipper, and improved performance, by up to 95%, with batching and memoization enabled. We also show that DLHub can scale to concurrently serve models on 500 containers. Finally, we describe five case studies that highlight the use of DLHub for scientific applications.

   @article{li2021dlhub,
     title = {DLHub: Simplifying publication, discovery, and use of machine learning models in science},
     journal = {Journal of Parallel and Distributed Computing},
     volume = {147},
     pages = {64-76},
     year = {2021},
     issn = {0743-7315},
     doi = {https://doi.org/10.1016/j.jpdc.2020.08.006},
     url = {https://www.sciencedirect.com/science/article/pii/S0743731520303464},
     author = {Li, Zhuozhao and Chard, Ryan and Ward, Logan and Chard, Kyle and Skluzacek, Tyler J. and Babuji, Yadu and Woodard, Anna and Tuecke, Steven and Blaiszik, Ben and Franklin, Michael J. and Foster, Ian},
     keywords = {Learning systems, Model serving, Machine learning, DLHub},
     bibtex_show = {true},
     selected = {true},
     abbr = {JPDC},
     pdf = {JPDC-li-dlhub-2021.pdf}
   }

2020

1. HPDC

   FuncX: A Federated Function Serving Fabric for Science

   Ryan Chard, Yadu Babuji, Zhuozhao Li, Tyler Skluzacek, Anna Woodard, Ben Blaiszik, Ian Foster, and Kyle Chard

   In Proceedings of the 29th International Symposium on High-Performance Parallel and Distributed Computing, 2020

   Abs Bib URL PDF

   Exploding data volumes and velocities, new computational methods and platforms, and ubiquitous connectivity demand new approaches to computation in the sciences. These new approaches must enable computation to be mobile, so that, for example, it can occur near data, be triggered by events (e.g., arrival of new data), be offloaded to specialized accelerators, or run remotely where resources are available. They also require new design approaches in which monolithic applications can be decomposed into smaller components, that may in turn be executed separately and on the most suitable resources. To address these needs we present funcX—a distributed function as a service (FaaS) platform that enables flexible, scalable, and high performance remote function execution. funcX’s endpoint software can transform existing clouds, clusters, and supercomputers into function serving systems, while funcX’s cloud-hosted service provides transparent, secure, and reliable function execution across a federated ecosystem of endpoints. We motivate the need for funcX with several scientific case studies, present our prototype design and implementation, show optimizations that deliver throughput in excess of 1 million functions per second, and demonstrate, via experiments on two supercomputers, that funcX can scale to more than more than 130 000 concurrent workers.

   @inproceedings{chard2020funcx,
     author = {Chard, Ryan and Babuji, Yadu and Li, Zhuozhao and Skluzacek, Tyler and Woodard, Anna and Blaiszik, Ben and Foster, Ian and Chard, Kyle},
     title = {FuncX: A Federated Function Serving Fabric for Science},
     year = {2020},
     isbn = {9781450370523},
     publisher = {Association for Computing Machinery},
     address = {New York, NY, USA},
     url = {https://doi.org/10.1145/3369583.3392683},
     doi = {10.1145/3369583.3392683},
     booktitle = {Proceedings of the 29th International Symposium on High-Performance Parallel and Distributed Computing},
     pages = {65–76},
     numpages = {12},
     keywords = {federated function serving, function as a service, funcX},
     location = {Stockholm, Sweden},
     series = {HPDC '20},
     abbr = {HPDC},
     pdf = {HPDC-chard-funcx-2020.pdf},
     selected = {true},
     bibtex_show = {true}
   }

2. arXiv

   Targeting SARS-CoV-2 with AI-and HPC-enabled lead generation: a first data release

   Yadu Babuji, Ben Blaiszik, Tom Brettin, Kyle Chard, Ryan Chard, Austin Clyde, Ian Foster, Zhi Hong, Shantenu Jha, Zhuozhao Li, and others

   arXiv preprint arXiv:2006.02431, 2020

   Abs Bib URL

   Researchers across the globe are seeking to rapidly repurpose existing drugs or discover new drugs to counter the the novel coronavirus disease (COVID-19) caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). One promising approach is to train machine learning (ML) and artificial intelligence (AI) tools to screen large numbers of small molecules. As a contribution to that effort, we are aggregating numerous small molecules from a variety of sources, using high-performance computing (HPC) to computer diverse properties of those molecules, using the computed properties to train ML/AI models, and then using the resulting models for screening. In this first data release, we make available 23 datasets collected from community sources representing over 4.2 B molecules enriched with pre-computed: 1) molecular fingerprints to aid similarity searches, 2) 2D images of molecules to enable exploration and application of image-based deep learning methods, and 3) 2D and 3D molecular descriptors to speed development of machine learning models. This data release encompasses structural information on the 4.2 B molecules and 60 TB of pre-computed data. Future releases will expand the data to include more detailed molecular simulations, computed models, and other products.

   @article{babuji2020targeting,
     title = {Targeting SARS-CoV-2 with AI-and HPC-enabled lead generation: a first data release},
     author = {Babuji, Yadu and Blaiszik, Ben and Brettin, Tom and Chard, Kyle and Chard, Ryan and Clyde, Austin and Foster, Ian and Hong, Zhi and Jha, Shantenu and Li, Zhuozhao and others},
     journal = {arXiv preprint arXiv:2006.02431},
     year = {2020},
     abbr = {arXiv},
     status = {arXiv},
     bibtex_show = {true},
     selected = {true},
     url = {https://arxiv.org/abs/2006.02431}
   }

3. ICPP

   Impact of Memory DoS Attacks on Cloud Applications and Real-Time Detection Schemes

   Zhuozhao Li, Tanmoy Sen, Haiying Shen, and Mooi Choo Chuah

   In 49th International Conference on Parallel Processing - ICPP, 2020

   Abs Bib URL PDF

   Even though memory-based denial-of-service attacks can cause severe performance degradations on co-located virtual machines, a previous detection scheme against such attacks cannot accurately detect the attacks and also generates high detection delay and high performance overhead since it assumes that cache-related statistics of an application follow the same probability distribution at all times, which may not be true for all types of applications. In this paper, we present the experimental results showing the impacts of memory DoS attacks on different types of cloud-based applications. Based on these results, we propose two lightweight, responsive Statistical based Detection Schemes (SDS/B and SDS/P) that can detect such attacks accurately. SDS/B constructs a profile of normal range of cache-related statistics for all applications and use statistical methods to infer an attack when the real-time collected statistics exceed this normal range, while SDS/P exploits the increased periods of access patterns for periodic applications to infer an attack. Our evaluation results show that SDS/B and SDS/P outperform the state-of-the-art detection scheme, e.g., with 65% higher specificity, 40% shorter detection delay, and 7% less performance overhead.

   @inproceedings{li2020impact,
     author = {Li, Zhuozhao and Sen, Tanmoy and Shen, Haiying and Chuah, Mooi Choo},
     title = {Impact of Memory DoS Attacks on Cloud Applications and Real-Time Detection Schemes},
     year = {2020},
     isbn = {9781450388160},
     publisher = {Association for Computing Machinery},
     address = {New York, NY, USA},
     url = {https://doi.org/10.1145/3404397.3404465},
     doi = {10.1145/3404397.3404465},
     booktitle = {49th International Conference on Parallel Processing - ICPP},
     articleno = {67},
     numpages = {11},
     keywords = {detection schemes, memory Denial-of-Service attack, virtual machine},
     location = {Edmonton, AB, Canada},
     series = {ICPP '20},
     abbr = {ICPP},
     pdf = {ICPP-li-mdos-2020.pdf},
     selected = {true},
     bibtex_show = {true}
   }

4. CIKM

   Time-Efficient Geo-Obfuscation to Protect Worker Location Privacy over Road Networks in Spatial Crowdsourcing

   Chenxi Qiu, Anna Squicciarini, Zhuozhao Li, Ce Pang, and Li Yan

   In Proceedings of the 29th ACM International Conference on Information & Knowledge Management, 2020

   Abs Bib URL PDF

   To promote cost-effective task assignment in Spatial Crowdsourcing (SC), workers are required to report their location to servers, which raises serious privacy concerns. As a solution, geo-obfuscation has been widely used to protect the location privacy of SC workers, where workers are allowed to report perturbed location instead of the true location. Yet, most existing geo-obfuscation methods consider workers? mobility on a 2 dimensional (2D) plane, wherein workers can move in arbitrary directions. Unfortunately, 2D-based geo-obfuscation is likely to generate high traveling cost for task assignment over roads, as it cannot accurately estimate the traveling costs distortion caused by location obfuscation. In this paper, we tackle the SC worker location privacy problem over road networks. Considering the network-constrained mobility features of workers, we describe workers? mobility by a weighted directed graph, which considers the dynamic traffic condition and road network topology. Based on the graph model, we design a geo-obfuscation (GO) function for workers to maximize the workers? overall location privacy without compromising the task assignment efficiency. We formulate the problem of deriving the optimal GO function as a linear programming (LP) problem. By using the angular block structure of the LP’s constraint matrix, we apply Dantzig-Wolfe decomposition to improve the time-efficiency of the GO function generation. Our experimental results in the real-trace driven simulation and the real-world experiment demonstrate the effectiveness of our approach in terms of both privacy and task assignment efficiency.

   @inproceedings{qiu2020time,
     author = {Qiu, Chenxi and Squicciarini, Anna and Li, Zhuozhao and Pang, Ce and Yan, Li},
     title = {Time-Efficient Geo-Obfuscation to Protect Worker Location Privacy over Road Networks in Spatial Crowdsourcing},
     year = {2020},
     isbn = {9781450368599},
     publisher = {Association for Computing Machinery},
     address = {New York, NY, USA},
     url = {https://doi.org/10.1145/3340531.3411863},
     doi = {10.1145/3340531.3411863},
     booktitle = {Proceedings of the 29th ACM International Conference on Information & Knowledge Management},
     pages = {1275–1284},
     numpages = {10},
     keywords = {geo-obfuscation, location privacy, spatial crowdsourcing},
     location = {Virtual Event, Ireland},
     series = {CIKM '20},
     abbr = {CIKM},
     bibtex_show = {true},
     selected = {true},
     pdf = {CIKM-qiu-obfuscation-2020.pdf}
   }

2019

1. ICDCS

   Co-scheduler: Accelerating Data-Parallel Jobs in Datacenter Networks with Optical Circuit Switching

   Zhuozhao Li, and Haiying Shen

   In 2019 IEEE 39th International Conference on Distributed Computing Systems (ICDCS), 2019

   Abs Bib URL PDF

   The optical circuit switch (OCS) in recently proposed hybrid electrical/optical datacenter networks (Hybrid-DCN) can only be used to transfer large flows (i.e., flows with a large size of data). Current job schedulers for data-parallel frameworks are not suitable for Hybrid-DCN, since they neither place tasks to aggregate data traffic to take advantage of OCS nor schedule tasks to minimize the Coflow completion time (CCT). In this paper, we propose Co-scheduler, a job scheduler for dataparallel frameworks that aims to improve job performance by attempting to place the tasks of a job to aggregate enough data traffic to take advantage of OCS and minimize the CCT in Hybrid-DCN. Specifically, for each job, Co-scheduler computes a guideline on the number of racks to place the job’s input data and to run the job’s map tasks, so that the job can potentially take full advantage of OCS to transfer its data. When the map tasks of a job complete, Co-scheduler computes all the possible schedules of the reduce tasks of the job. Each possible schedule includes the number of racks to schedule the reduce tasks that enables the job to use OCS to transfer its data, and the number of reduce tasks to place on each of the racks that minimizes CCT of the job. Next, Co-scheduler selects a best schedule among all the possible schedules so that the job completion time is minimized. Finally, Co-scheduler schedules the map tasks and reduce tasks of the job based on the computed guideline and best schedule. The evaluation demonstrates that compared to the state-of-theart schedulers, Co-scheduler achieves performance improvements on makespan, average job completion time, and average CCT by up to 51.2%, 54.6% and 73.6%, respectively.

   @inproceedings{li2019co,
     author = {Li, Zhuozhao and Shen, Haiying},
     booktitle = {2019 IEEE 39th International Conference on Distributed Computing Systems (ICDCS)},
     title = {Co-scheduler: Accelerating Data-Parallel Jobs in Datacenter Networks with Optical Circuit Switching},
     year = {2019},
     volume = {},
     number = {},
     pages = {186-195},
     keywords = {},
     doi = {10.1109/ICDCS.2019.00027},
     issn = {2575-8411},
     month = jul,
     abbr = {ICDCS},
     bibtex_show = {true},
     url = {https://ieeexplore.ieee.org/abstract/document/8885103},
     selected = {true},
     pdf = {ICDCS-li-coscheduler-2019.pdf}
   }

2. HPDC

   Parsl: Pervasive Parallel Programming in Python

   Yadu Babuji, Anna Woodard, Zhuozhao Li, Daniel S. Katz, Ben Clifford, Rohan Kumar, Lukasz Lacinski, Ryan Chard, Justin M. Wozniak, Ian Foster, Michael Wilde, and Kyle Chard

   In Proceedings of the 28th International Symposium on High-Performance Parallel and Distributed Computing, 2019

   Abs Bib URL PDF

   High-level programming languages such as Python are increasingly used to provide intuitive interfaces to libraries written in lower-level languages and for assembling applications from various components. This migration towards orchestration rather than implementation, coupled with the growing need for parallel computing (e.g., due to big data and the end of Moore’s law), necessitates rethinking how parallelism is expressed in programs. Here, we present Parsl, a parallel scripting library that augments Python with simple, scalable, and flexible constructs for encoding parallelism. These constructs allow Parsl to construct a dynamic dependency graph of components that it can then execute efficiently on one or many processors. Parsl is designed for scalability, with an extensible set of executors tailored to different use cases, such as low-latency, high-throughput, or extreme-scale execution. We show, via experiments on the Blue Waters supercomputer, that Parsl executors can allow Python scripts to execute components with as little as 5 ms of overhead, scale to more than 250000 workers across more than 8000 nodes, and process upward of 1200 tasks per second. Other Parsl features simplify the construction and execution of composite programs by supporting elastic provisioning and scaling of infrastructure, fault-tolerant execution, and integrated wide-area data management. We show that these capabilities satisfy the needs of many-task, interactive, online, and machine learning applications in fields such as biology, cosmology, and materials science.

   @inproceedings{babuji2019parsl,
     author = {Babuji, Yadu and Woodard, Anna and Li, Zhuozhao and Katz, Daniel S. and Clifford, Ben and Kumar, Rohan and Lacinski, Lukasz and Chard, Ryan and Wozniak, Justin M. and Foster, Ian and Wilde, Michael and Chard, Kyle},
     title = {Parsl: Pervasive Parallel Programming in Python},
     year = {2019},
     isbn = {9781450366700},
     publisher = {Association for Computing Machinery},
     address = {New York, NY, USA},
     url = {https://doi.org/10.1145/3307681.3325400},
     doi = {10.1145/3307681.3325400},
     booktitle = {Proceedings of the 28th International Symposium on High-Performance Parallel and Distributed Computing},
     pages = {25–36},
     numpages = {12},
     keywords = {parsl, parallel programming, python},
     location = {Phoenix, AZ, USA},
     series = {HPDC '19},
     abbr = {HPDC},
     pdf = {HPDC-babuji-parsl-2019.pdf},
     selected = {true},
     bibtex_show = {true}
   }

3. ICPP

   JobPacker: Job Scheduling for Data-Parallel Frameworks with Hybrid Electrical/Optical Datacenter Networks

   Zhuozhao Li, and Haiying Shen

   In Proceedings of the 48th International Conference on Parallel Processing, 2019

   Abs Bib URL PDF

   In spite of many advantages of hybrid electrical/optical datacenter networks (Hybrid-DCN), current job schedulers for data-parallel frameworks are not suitable for Hybrid-DCN, since the schedulers do not aggregate data traffic to facilitate using optical circuit switch (OCS). In this paper, we propose JobPacker, a job scheduler for data-parallel frameworks in Hybrid-DCN that aims to take full advantage of OCS to improve job performance. JobPacker aggregates the data transfers of a job in order to use OCS to improve data transfer efficiency. It first explores the tradeoff between parallelism and traffic aggregation for each shuffle-heavy recurring job, and then generates an offline schedule including which racks to run each job and the sequence to run the recurring jobs in each rack that yields the best performance. It has a new sorting method to prioritize recurring jobs in offline-scheduling to prevent high resource contention while fully utilizing cluster resources. In real-time scheduler, JobPacker uses the offline schedule to guide the data placement and schedule recurring jobs, and schedules non-recurring jobs to the idle resources not assigned to recurring jobs. Trace-driven simulation and GENI-based emulation show that JobPacker reduces the makespan up to 49% and the median completion time up to 43%, compared to the state-of-the-art schedulers in Hybrid-DCN.

   @inproceedings{li2019jobpacker,
     author = {Li, Zhuozhao and Shen, Haiying},
     title = {JobPacker: Job Scheduling for Data-Parallel Frameworks with Hybrid Electrical/Optical Datacenter Networks},
     year = {2019},
     isbn = {9781450362955},
     publisher = {Association for Computing Machinery},
     address = {New York, NY, USA},
     url = {https://doi.org/10.1145/3337821.3337880},
     doi = {10.1145/3337821.3337880},
     booktitle = {Proceedings of the 48th International Conference on Parallel Processing},
     articleno = {30},
     numpages = {10},
     location = {Kyoto, Japan},
     series = {ICPP 2019},
     abbr = {ICPP},
     bibtex_show = {true},
     selected = {true},
     pdf = {ICPP-li-jobpacker-2019.pdf}
   }

2018

1. ICDCS

   PageRankVM: A PageRank Based Algorithm with Anti-Collocation Constraints for Virtual Machine Placement in Cloud Datacenters

   Zhuozhao Li, Haiying Shen, and Cole Miles

   In 2018 IEEE 38th International Conference on Distributed Computing Systems (ICDCS), 2018

   Abs Bib URL PDF

   There is a dramatic increase in the variety of virtual machines (VMs) and complexity of VM placement problems in clouds. Previous VM placement approaches attempt to accommodate more VMs efficiently on fewer PMs by balancing the resource usages across multiple dimensions. However, these approaches are not sufficiently accurate in measuring the quality of the PMs in terms of fully utilizing PM resource and having the potential to accommodate more VMs. Therefore, it is critical to design a new method that can more accurately measure the probability of a PM of fully utilizing its resources after accommodating a given VM with the consideration of different types of VMs. In addition, anti-collocation constraints must be handled efficiently. We propose a PageRank based VM placement algorithm with anti-collocation constraints (PageRankVM). PageRankVM defines the best PM resource usage profile, which means that the PM has full resource utilization for every resource dimension, and then ranks PM profiles according to their convergence of transferring (by accommodating VMs) to the best profile. PageRankVM then places a given VM to the PM based on the ranks of the resulted PM profiles after accommodating the VM with the consideration of anti-collocation constraints. Compared to previous approaches, PageRankVM effectively measures the ability of different PM profiles to reach the best profiles by accommodating a given VM, and hence differentiates the effectiveness of different VM placement decisions. We conducted extensive trace-driven simulation and GENI testbed experiments and demonstrated that PageRankVM has superior performance compared with other methods in terms of reducing the number of PMs, the energy consumption, the number of VM migrations, and the service level objective (SLO) violations.

   @inproceedings{li2018pagerankvm,
     author = {Li, Zhuozhao and Shen, Haiying and Miles, Cole},
     booktitle = {2018 IEEE 38th International Conference on Distributed Computing Systems (ICDCS)},
     title = {PageRankVM: A PageRank Based Algorithm with Anti-Collocation Constraints for Virtual Machine Placement in Cloud Datacenters},
     year = {2018},
     volume = {},
     number = {},
     pages = {634-644},
     keywords = {},
     doi = {10.1109/ICDCS.2018.00068},
     issn = {2575-8411},
     month = jul,
     abbr = {ICDCS},
     bibtex_show = {true},
     url = {https://ieeexplore.ieee.org/abstract/document/8416331},
     selected = {true},
     pdf = {ICDCS-li-pagerankvm-2018.pdf}
   }

2. UbiComp

   Employing Opportunistic Charging for Electric Taxicabs to Reduce Idle Time

   Li Yan, Haiying Shen, Zhuozhao Li, Ankur Sarker, John A. Stankovic, Chenxi Qiu, Juanjuan Zhao, and Chengzhong Xu

   Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., 2018

   Abs Bib URL PDF

   For electric taxicabs, the idle time spent on cruising for passengers, seeking chargers, and charging is wasteful. Previous works can only save cruising time through better routing, or charger seeking and charging time through proper charger deployment, but not for both. With the advancement of wireless charging techniques, efficient opportunistic charging of electric vehicles at their parked positions becomes possible. This enables a taxicab to get charged while waiting for the next passenger. In this paper, we present an opportunistic wireless charger deployment scheme in a city, which both maximizes the taxicabs’ opportunity of picking up passengers at the chargers and supports the taxicabs’ continuous operability on roads, while minimizing the total deployment cost. We studied a metropolitan-scale taxicab dataset on several factors important for deploying wireless chargers and determining the numbers of the chargers in the regions: the number of passengers, the functionalities of buildings, and the frequency of passenger appearance in a region, and taxicab traffic flows in a city. Then, we formulate a multi-objective optimization problem and find the solution. Our trace-driven experiments demonstrate the superior performance of our scheme over other representative methods in terms of reducing idle time and supporting the operability of the taxicabs.

   @article{yan2018employing,
     author = {Yan, Li and Shen, Haiying and Li, Zhuozhao and Sarker, Ankur and Stankovic, John A. and Qiu, Chenxi and Zhao, Juanjuan and Xu, Chengzhong},
     title = {Employing Opportunistic Charging for Electric Taxicabs to Reduce Idle Time},
     year = {2018},
     issue_date = {March 2018},
     publisher = {Association for Computing Machinery},
     address = {New York, NY, USA},
     volume = {2},
     number = {1},
     url = {https://doi.org/10.1145/3191779},
     doi = {10.1145/3191779},
     journal = {Proc. ACM Interact. Mob. Wearable Ubiquitous Technol.},
     month = mar,
     articleno = {47},
     numpages = {25},
     keywords = {mobile data analysis, kernel density estimation, Vehicle wireless charging, charger deployment},
     abbr = {UbiComp},
     bibtex_show = {true},
     selected = {true},
     pdf = {UbiComp-yan-employing-2018.pdf}
   }

2017

1. TPDS

   Measuring Scale-Up and Scale-Out Hadoop with Remote and Local File Systems and Selecting the Best Platform

   Zhuozhao Li, and Haiying Shen

   IEEE Transactions on Parallel and Distributed Systems, 2017

   Abs Bib URL PDF

   MapReduce is a popular computing model for parallel data processing on large-scale datasets, which can vary from gigabytes to terabytes and petabytes. Though Hadoop MapReduce normally uses Hadoop Distributed File System (HDFS) local file system, it can be configured to use a remote file system. Then, an interesting question is raised: for a given application, which is the best running platform among the different combinations of scale-up and scale-out Hadoop with remote and local file systems. However, there has been no previous research on how different types of applications (e.g., CPU-intensive, data-intensive) with different characteristics (e.g., input data size) can benefit from the different platforms. Thus, in this paper, we conduct a comprehensive performance measurement of different applications on scale-up and scale-out clusters configured with HDFS and a remote file system (i.e., OFS), respectively. We identify and study how different job characteristics (e.g., input data size, the number of file reads/writes, and the amount of computations) affect the performance of different applications on the different platforms. Based on the measurement results, we also propose a performance prediction model to help users select the best platforms that lead to the minimum latency. Our evaluation using a Facebook workload trace demonstrates the effectiveness of our prediction model. This study is expected to provide a guidance for users to choose the best platform to run different applications with different characteristics in the environment that provides both remote and local storage, such as HPC cluster and cloud environment.

   @article{li2017measuring,
     author = {Li, Zhuozhao and Shen, Haiying},
     journal = {IEEE Transactions on Parallel and Distributed Systems},
     title = {Measuring Scale-Up and Scale-Out Hadoop with Remote and Local File Systems and Selecting the Best Platform},
     year = {2017},
     volume = {28},
     number = {11},
     pages = {3201-3214},
     keywords = {},
     doi = {10.1109/TPDS.2017.2712635},
     issn = {1558-2183},
     month = nov,
     abbr = {TPDS},
     bibtex_show = {true},
     url = {https://ieeexplore.ieee.org/abstract/document/7940040},
     selected = {true},
     pdf = {TPDS-li-measurement-2017.pdf}
   }

2. TPDS

   An Exploration of Designing a Hybrid Scale-Up/Out Hadoop Architecture Based on Performance Measurements

   Zhuozhao Li, Haiying Shen, Walter Ligon, and Jeffrey Denton

   IEEE Transactions on Parallel and Distributed Systems, 2017

   Abs Bib URL PDF

   Scale-up machines perform better for jobs with small and median (KB, MB) data sizes, while scale-out machines perform better for jobs with large (GB, TB) data size. Since a workload usually consists of jobs with different data size levels, we propose building a hybrid Hadoop architecture that includes both scale-up and scale-out machines, which however is not trivial. The first challenge is workload data storage. Thousands of small data size jobs in a workload may overload the limited local disks of scale-up machines. Jobs from scale-up and scale-out machines may both request the same set of data, which leads to data transmission between the machines. The second challenge is to automatically schedule jobs to either scale-up or scale-out cluster to achieve the best performance. We conduct a thorough performance measurement of different applications on scale-up and scale-out clusters, configured with Hadoop Distributed File System (HDFS) and a remote file system (i.e., OFS), respectively. We find that using OFS rather than HDFS can solve the data storage challenge. Also, we identify the factors that determine the performance differences on the scale-up and scale-out clusters and their cross points to make the choice. Accordingly, we design and implement the hybrid scale-up/out Hadoop architecture. Our trace-driven experimental results show that our hybrid architecture outperforms both the traditional Hadoop architecture with HDFS and with OFS in terms of job completion time, throughput and job failure rate.

   @article{li2017exploration,
     author = {Li, Zhuozhao and Shen, Haiying and Ligon, Walter and Denton, Jeffrey},
     journal = {IEEE Transactions on Parallel and Distributed Systems},
     title = {An Exploration of Designing a Hybrid Scale-Up/Out Hadoop Architecture Based on Performance Measurements},
     year = {2017},
     volume = {28},
     number = {2},
     pages = {386-400},
     keywords = {},
     doi = {10.1109/TPDS.2016.2573820},
     issn = {1558-2183},
     month = feb,
     abbr = {TPDS},
     bibtex_show = {true},
     url = {https://ieeexplore.ieee.org/abstract/document/7480403},
     selected = {true},
     pdf = {TPDS-li-hybrid-2016.pdf}
   }

2016

1. TPDS

   New Bandwidth Sharing and Pricing Policies to Achieve a Win-Win Situation for Cloud Provider and Tenants

   Haiying Shen, and Zhuozhao Li

   IEEE Transactions on Parallel and Distributed Systems, 2016

   Abs Bib URL PDF

   For predictable application performance or fairness in network sharing in clouds, many bandwidth allocation policies have been proposed. However, with these policies, tenants are not incentivized to use idle bandwidth or prevent link congestion, and may even take advantage of the policies to gain unfair bandwidth allocation. Increasing network utilization while avoiding congestion not only benefits cloud provider but also the tenants by improving application performance. In this paper, we propose a new pricing model that sets different unit prices for reserved bandwidth, the bandwidth on congested links and on uncongested links, and makes the unit price for congested links proportional to their congestion degrees. We use game theory model to analyze tenants’ behaviors in our model and the current pricing models, which shows the effectiveness of our model in providing the incentives. With the pricing model, we propose a network sharing policy to achieve both min-guarantee and proportionality, while prevent tenants from earning unfair bandwidth. We further propose methods for each virtual machine to arrange its traffic to reduce its unsatisfied demand and maximize its utility, while increase network utilization. As a result, our solution creates a win-win situation, where tenants strive to increase their benefits in bandwidth sharing, which also concurrently increases the utilities of cloud provider and other tenants. Our simulation and trace-driven experimental results show the effectiveness of our solutions in creating the win-win situation.

   @article{shen2016new,
     author = {Shen, Haiying and Li, Zhuozhao},
     journal = {IEEE Transactions on Parallel and Distributed Systems},
     title = {New Bandwidth Sharing and Pricing Policies to Achieve a Win-Win Situation for Cloud Provider and Tenants},
     year = {2016},
     volume = {27},
     number = {9},
     pages = {2682-2697},
     keywords = {},
     doi = {10.1109/TPDS.2015.2497701},
     issn = {1558-2183},
     month = sep,
     abbr = {TPDS},
     bibtex_show = {true},
     url = {https://ieeexplore.ieee.org/abstract/document/7317795},
     selected = {true},
     pdf = {TPDS-shen-winwin-2016.pdf}
   }

2015

1. ICPP

   Designing a Hybrid Scale-Up/Out Hadoop Architecture Based on Performance Measurements for High Application Performance

   Zhuozhao Li, and Haiying Shen

   In 2015 44th International Conference on Parallel Processing, 2015

   Abs Bib URL PDF

   Since scale-up machines perform better for jobs with small and median (KB, MB) data sizes while scale-out machines perform better for jobs with large (GB, TB) data size, and a workload usually consists of jobs with different data size levels, we propose building a hybrid Hadoop architecture that includes both scale-up and scale-out machines, which however is not trivial. The first challenge is workload data storage. Thousands of small data size jobs in a workload may overload the limited local disks of scale-up machines. Jobs from scale-up and scale-out machines may both request the same set of data, which leads to data transmission between the machines. The second challenge is to automatically schedule jobs to either scale-up or scale-out cluster to achieve the best performance. We conduct a thorough performance measurement of different applications on scale-up and scale-out clusters, configured with Hadoop Distributed File System (HDFS) and a remote file system (i.e., OFS), respectively. We find that using OFS rather than HDFS can solve the data storage challenge. Also, we identify the factors that determine the performance differences on the scale-up and scale-out clusters and their cross points to make the choice. Accordingly, we design and implement the hybrid scale-up/out Hadoop architecture. Our trace-driven experimental results show that our hybrid architecture outperforms both the traditional Hadoop architecture with HDFS and with OFS in terms of job completion time.

   @inproceedings{li2015designing,
     author = {Li, Zhuozhao and Shen, Haiying},
     booktitle = {2015 44th International Conference on Parallel Processing},
     title = {Designing a Hybrid Scale-Up/Out Hadoop Architecture Based on Performance Measurements for High Application Performance},
     year = {2015},
     volume = {},
     number = {},
     pages = {21-30},
     keywords = {},
     doi = {10.1109/ICPP.2015.11},
     issn = {0190-3918},
     month = sep,
     abbr = {ICPP},
     bibtex_show = {true},
     url = {https://ieeexplore.ieee.org/abstract/document/7349557},
     pdf = {ICPP-li-hybrid-2015.pdf},
     selected = {true}
   }