🔬 Do we really need to recompute everything?

🔬 Do we really need to recompute everything?

In many scenarios, the same workflows are executed repeatedly with only small changes in parameters or datasets. Yet, many systems still recompute everything from scratch. This characteristic motivated a question that led to our recent research.

We are happy to share that our paper “Have We Seen these Data Before? A GRASP-based Execution Strategy for Cloud-based Workflows with Memoization” has been accepted for publication in Concurrency and Computation: Practice and Experience 🎉

💡 Key idea:

Instead of recomputing intermediate results, why not reuse previously computed results?

In this paper, we propose MemoirGRASP, a workflow execution strategy based on the GRASP metaheuristic that uses memoization to detect and reuse previously computed intermediate data across workflow executions.

📊 What we found:

Experiments with synthetic workflows and real-world applications such as Montage and Phenomenal show that reusing intermediate results can significantly reduce redundant computation and improve execution efficiency in multisite cloud environments.

🤝 This work is the result of an international collaboration between researchers from Instituto de Computação UFF (Brazil), University of Göttingen (Germany), Inria / University of Montpellier, CNRS (France).

Grateful to all co-authors (Rodrigo A. P. da Silva, Gaetan Heidsieck, Esther Pacitti, Patrick Valduriez, Yuri Frota, and Daniel de Oliveira) for this collaboration!

#ScientificWorkflows #CloudComputing #DistributedSystems #HighPerformanceComputing #Research