 # Implementing non-negative defense in GCN graph classification to train a robust model?

I wanted to implement the idea in this paper in my GCN model :

the general idea of paper is this :

As the name suggests, Non-Negative MalConv was constrained during training to have non-negative weight matrices. The point of doing this is to prevent trivial attacks like those created against MalConv. When done properly, the non-negative weights make binary classifiers monotonic; meaning that the addition of new content can only increase the malicious score. This would make evading the model very difficult, because most evasion attacks do require adding content to the file. Fortunately for me, this implementation of Non-Negative MalConv has a subtle but critical flaw.

Right now my model is very similar to this : https://docs.dgl.ai/en/0.4.x/tutorials/basics/4_batch.html

so how can i implement this idea in this code? basically i want my GCN to learn to only decide based on malicious features in nodes, and adding benign features in a node (i don’t have node labels, by benign i mean features that adding a lot of them will cause a model to converge to labeling the graph as benign) will not cause a model to give higher probability for the graph of being benign, and only adding malicious features will cause the graph classification probability to change

for example if each node has 50 features, and in the vanila GCN, the adding to the value of second feature will cause the graph classification to converge to being benign, in this non-negative GCN adding this feature will not affect the model.

any idea how can i implement this in GCN? is it possible?

One possibility is to follow SGC. Basically, it removes all nonlinear activation functions between GCN layers. As a result, you can implement the model as follows:

1. Pre-compute message passing A^{k}X, where A is the adjacency matrix and X is input node features
2. Pass the result of A^{k}X to a logistic regression model.

To implement non-negative weight, you can simply use the non-negative variant of logistic regression.

1 Like

Thank you, so what is the most efficient way of changing the SGConv code of dgl to do this? sorry I’m new to pytorch and I’m not that familiar with the functions pytorch provides to make this easier.

``````class SGConv(nn.Module):

def __init__(self,
in_feats,
out_feats,
k=1,
cached=False,
bias=True,
norm=None):
super(SGConv, self).__init__()
self.fc = nn.Linear(in_feats, out_feats, bias=bias)
self._cached = cached
self._cached_h = None
self._k = k
self.norm = norm
self.reset_parameters()

def reset_parameters(self):
nn.init.xavier_uniform_(self.fc.weight)
if self.fc.bias is not None:
nn.init.zeros_(self.fc.bias)

def forward(self, graph, feat):

graph = graph.local_var()
if self._cached_h is not None:
feat = self._cached_h
else:
# compute normalization
degs = graph.in_degrees().float().clamp(min=1)
norm = th.pow(degs, -0.5)
norm = norm.to(feat.device).unsqueeze(1)
# compute (D^-1 A^k D)^k X
for _ in range(self._k):
feat = feat * norm
graph.ndata['h'] = feat
graph.update_all(fn.copy_u('h', 'm'),
fn.sum('m', 'h'))
feat = graph.ndata.pop('h')
feat = feat * norm

if self.norm is not None:
feat = self.norm(feat)

# cache feature
if self._cached:
self._cached_h = feat
return self.fc(feat)``````

You just need to remove the part related to `self.fc` and pass the output of `SGConv` to a non-negative logistic regression.

1 Like

Sorry for rookie question but how can i implement “non negative” logistic regression in pytorch?

all the tutorial for pytorch’s logistic regression i found implemented something like this :

``````class LogisticRegression(torch.nn.Module):
def __init__(self, input_dim, output_dim):
super(LogisticRegression, self).__init__()
self.linear = torch.nn.Linear(input_dim, output_dim)

def forward(self, x):
outputs = self.linear(x)
return outputs
``````

but i couldn’t find anything regarding a non negative version of logistic regression?

also what do you mean regarding removing the parts related to self.fc? do you mean i should remove any usage of self.fc in the SGconv class like this ? :

``````    class SGConv(nn.Module):

def __init__(self,
in_feats,
out_feats,
k=1,
cached=False,
bias=True,
norm=None):
super(SGConv, self).__init__()
self._cached = cached
self._cached_h = None
self._k = k
self.norm = norm
self.reset_parameters()

def reset_parameters(self):
pass

def forward(self, graph, feat):

graph = graph.local_var()
if self._cached_h is not None:
feat = self._cached_h
else:
# compute normalization
degs = graph.in_degrees().float().clamp(min=1)
norm = th.pow(degs, -0.5)
norm = norm.to(feat.device).unsqueeze(1)
# compute (D^-1 A^k D)^k X
for _ in range(self._k):
feat = feat * norm
graph.ndata['h'] = feat
graph.update_all(fn.copy_u('h', 'm'),
fn.sum('m', 'h'))
feat = graph.ndata.pop('h')
feat = feat * norm

if self.norm is not None:
feat = self.norm(feat)

# cache feature
if self._cached:
self._cached_h = feat
return feat
``````

Also it seems like in keras it is possible to constraint the weights to be positive using

``````tf.keras.constraints.NonNeg()
``````

but i couldn’t find the Equivelent of this in pytorch

Basically, you just need to project the weights after each gradient descent and make them nonzero.

``````model = LogisticRegression(...)
model.linear.weight.data = model.linear.weight.data.clamp(min=0)
``````

The clamping operation above is equivalent to `tf.keras.constraints.NonNeg()`.

For `self.fc`, you need to remove it by `LogisticRegression`.

1 Like