Skip to main content

Network modeling and data movement

Network modeling lets planning distinguish a local dependency from data that must cross a resource boundary. It starts with bytes declared by the workflow, but its result depends on the selected resource assignments and the directed topology chosen for the planning session.

This is an explanation of the model. Use SimGrid modeling to configure bandwidth and latency, or the topology reference for the exact document fields.

From dependency to possible flow​

The control dependency makes the consumer wait for the producer. The matching data dependency gives the planner a logical byte volume. If a selected plan puts both activities on the same resource, no network transfer time is added for that edge. If they are on different resources, the topology is consulted for a route.

The portable workflow importer uses declared bytes in scheduling only when the same pair of activities also has a control dependency. A data declaration alone does not order the activities.

What a topology models​

A NetworkLink is directed. It identifies source and target resources and can carry bandwidth in bits per second, latency in seconds, byte price, whether the reverse direction is available, a sharing group, and a transfer concurrency limit. A bidirectional link makes the same link available in reverse; it does not create a second independently configured route.

PRISM precomputes routes from the frozen topology and includes communication in its candidate evaluation. The SimGrid platform and PRISM choose a route by summing each link's latency plus the time to transmit one byte. They apply the full byte volume afterward, so the chosen route may not be fastest for a large transfer.

PRISM can model known overlapping flows on the chosen route and rejects a cross-resource data transfer without one. HEFT's baseline uses a direct matching link; when none exists, it currently estimates zero transfer time. Supply the links needed by your placements rather than treating that estimate as evidence of a free transfer. Neither model guarantees the better observed run.

Planned route versus executed transfer​

The plan predicts transfer time for each assignment. During execution, AkôFlow may use an existing verified copy, shared storage, or a supported transfer route. The transfer record shows how data became available and, when reported, how many bytes moved and how long it took.

That distinction matters: the model describes a possible network penalty for a placement; the observation says how bytes were actually made available. Shared storage or a verified existing copy may satisfy a dependency without a new network transfer.

Units and a small example​

For a 10 GiB data dependency over a 10 Gbit/s link, the raw serialization time is about 8.6 seconds before latency and sharing. 10 GiB is a byte volume; 10 Gbit/s is a bit rate. The observed time can be higher because of setup, routing, sharing, or provider behavior.

Inspect a completed run's transfer records alongside the assignment and plan prediction. Use the SimGrid guide to construct a topology where these effects are intentional.