Attention inside GraphSAGE

I see that you have an attention aggregator inside graphSAGE. Will it work, because typically attention doesn’t require sampling.

Can I use GAT for unsupervised training like graphSAGE… looking at the demos, it isn’t quiet sure…


Hi Amit,

We do have an initial version of an attention-based aggregator (note currently it is single-headed only). You are correct that the standard GAT does not use attention. When the attention head is used with a sampling mechanism it will learn a functional transform between the features of the nodes and a scalar weight which is then normalized over the sample rather than all edges. This should be though of as just another parameterized aggregation function that can be trained to give useful results.

Calculating attention over a sample can mean that for some samples the normalized attentions are not helpful for the inference task. However, this is true for standard aggregators in GraphSAGE as well. In order to stabilize the results and avoid problematic samples we recommend using GraphSAGE with the Ensemble class, see the Ensembles example in this notebook

Currently the GAT class cannot be used for unsupervised training, as it requires support for link prediction. We are currently working to add this functionality to our library, and will let you know when we do!

Best regards,