EUPEX webinar – Scientific Workflows in the Heterogeneous Era
The EUPEX researchers share their results and lessons learnt!
In collaboration with
11am to 12pm CET
We are organising monthly webinars about the outcomes of EUPEX (full programme here). Our fourth webinar will be dedicated to one of the many software tools enhanced and used under EUPEX.
In February, we will address large-scale workflow orchestration, and discover in particular StreamFlow, a container-native workflow manager for hybrid infrastructures.
Abstract: Thanks to their generality, workflow models represent a powerful abstraction for designing complex applications and executing them on large-scale distributed architectures. However, several additional challenges appear when transitioning from cloud/HPC environments to the entire computing continuum. Continuum execution environments are fully distributed and modular, and modules can be heterogeneous and independent of each other. In addition, continuum workflows often rely on multiple intercommunicating agents that form complex micro-services architectures. Different agents deal with different communication and parallelization paradigms: network-based stream processing at the edge and file-based batch processing on HPC facilities. Finally, support for efficient interactive workflows in the continuum remains an open research problem. This webinar delves into these challenges, exploring how current research addresses them with innovatives techniques and tools (i.e., hybrid workflows and StreamFlow, literate workflows and Jupyter Workflow, transparent streaming workflows and CAPIO), and suggesting potential directions for future investigation.
Speaker: Iacopo Colonnelli from University of Torino
Iacopo Colonnelli is a Computer Science assistant professor (RTDA) at the University of Torino, Italy, and a member of the CINI HPC Key Technologies and Tools lab. He received his Ph.D. with honours in Modeling and Data Science at Università di Torino with a thesis on novel workflow models for heterogeneous distributed systems, and his master’s degree in Computer Engineering from Politecnico di Torino with a thesis on a high-performance parallel tracking algorithm for the ALICE experiment at CERN. His research focuses on both statistical and computational aspects of data analysis at large scale and on workflow modeling and management in heterogeneous distributed architectures.
