I recently started self-learning about GNNs, and I have trouble understanding the difference between the GNN models.

As I understand it, all GNN models apply an aggregation step on each node, and the different models differ in the aggregation function.

  1. Is the aggregation for a single node done by aggregating the entire graph? or only its close neighbors?
  2. What is the difference between GraphSAGE and other GNN models? Do I understand correctly that GraphSAGE aggregates only some of a node's neighbors and not all of them?
  • 1
    $\begingroup$ It's been some time since I worked with them, so I'm not sure I'm correct, but: 1) Graph convolutional networks (GCN) aggregate only neighbor information, while in general GNN can do something different (e.g. as I recall, there are approaches which use random walks). GCN use multiple aggregation layers, so on the $k$-th layer they collect information from $k$-hop neighbors. 2) Pure GCN has a problem: the number of nodes in $k$-hop neighborhood grows exponentially with $k$, making SGD ineffective. Multiple GCN approaches (GraphSAGE too, as I recall) solve this using neighbor subsampling. $\endgroup$
    – user114966
    Apr 30 at 6:05
  • $\begingroup$ Thanks! This was very helpful :) $\endgroup$
    – nir shahar
    Apr 30 at 9:10
  • $\begingroup$ I recommend double-checking what I've written (sorry, it's been 2 years since I worked with them and I don't have time to go over the papers to verify this). I'm confident that multiple GCNs use neighbor subsampling to make SGD feasible, and there are various flavors: uniform sampling, attention-based sampling, sampling but using cached values for other neighbors. There is also a fun approach that doesn't use activation function for hidden layers, therefore reducing the problem to logistic regression while, as they report, providing the same accuracy. $\endgroup$
    – user114966
    Apr 30 at 20:53
  • $\begingroup$ Sure, but from what I have read, it seems to explain everything (and coincides with everything else written) - so I believe you are right. Thanks! :) $\endgroup$
    – nir shahar
    Apr 30 at 21:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.