Confusion about feature prefetch

[yofufufufu]

Hello everyone, I am trying to figure out DGL dataflow. And I am confused about feature prefetch:
Q1: When does the feature prefetching occur?
Q2: Without feature prefetching(i.e. not set prefetch_node_feats=['feat'], prefetch_labels=['label']), When does the data copy from CPU to GPU occur? I read this blog but I am still confused. Maybe data copy occur when dataloader is “used” in iteration? I am not sure about that.
Thanks!

[BarclayII]

@yofufufufu

Q1: When does the feature prefetching occur?

If prefetching is enabled, a separate thread performs feature fetching.

@yofufufufu

Q2: Without feature prefetching(i.e. not set prefetch_node_feats=['feat'], prefetch_labels=['label']), When does the data copy from CPU to GPU occur? I read this blog but I am still confused. Maybe data copy occur when dataloader is “used” in iteration? I am not sure about that.

Yes.

[yofufufufu]

Thanks for your reply.
So when code like for step, (input_nodes, output_nodes, mfgs) in enumerate(train_dataloader) executed:
if prefetching is enabled, the feature is copied to device since iterator _PrefetchingIter init, and because of the separate thread you mentioned above, model computation and data movement happen in parallel(multi-threads) unless all batch datas have transferred, which means done_event.is_set() is true.
if not, the batch data is copied to device everytime function __ next __ is called every iteration.
Am I right?

[BarclayII]

@yofufufufu

if prefetching is enabled, the feature is copied to device since iterator _PrefetchingIter init, and because of the separate thread you mentioned above, model computation and data movement happen in parallel(multi-threads) unless all batch datas have transferred, which means done_event.is_set() is true.

This is true.

@yofufufufu

if not, the batch data is copied to device everytime function __next__ is called every iteration.
Am I right?

Sorry I misread your previous question. When prefetching is not enabled, the sampled subgraphs are copied to GPU during __next__, and the feature copies are delayed until they are being read (e.g. when actually executing input_features = blocks[0].srcdata['x']).

[yofufufufu]

So there is something about feature lazy loading? I mean:
with prefetching, both subgraph structure info and subgraph all nodes’ feature info are copied to device;
without prefetching, only subgraph structure info is copied to device during __next__, and the feature info will be copied until first time they are “used”.
And I just noticed that the doc says:

prefetch_node_feats : list[str] or dict[ntype, list[str]], optional
The source node data to prefetch for the first MFG, corresponding to the input node features necessary for the first GNN layer.

I think that is because MFG own features from the previous layer, so the first MFG owns all nodes and features in one subgraph, right?

[yofufufufu]

Can anyone help me? I am confused about the “prefetching” concept.

[BarclayII]

@yofufufufu

with prefetching, both subgraph structure info and subgraph all nodes’ feature info are copied to device;
without prefetching, only subgraph structure info is copied to device during __next__, and the feature info will be copied until first time they are “used”.

Yes.

@yofufufufu

I think that is because MFG own features from the previous layer, so the first MFG owns all nodes and features in one subgraph, right?

Yes.
(20 character limit)

[yofufufufu]

Thank you for solving my questions!