Unsupervised MP Node Embedding for Traffic Signal Control

[bowlmeat]

Sincerely thank you for reading my question and answer it! Thank you very much!

• Background

  • A traffic intersection with east, west, north and south four directions. The maximum queuing length of the four directions in one signal cycle are abstracted as nodes. The nodes are connected with each other, while the two nodes on the same line belong to the same signal group, which is described in the figure using the red lines.
    
  • In this case there exist two type edges and one type node.
    • graph structure: 6 edges, 4 nodes and 1 type node, 2 type edges
    • graph attribute: node features. Just as the queuing lengths belong to respective nodes, it should be treated as node features.
  • The mission:
    node embedding with the message-passing GNN

• My Ideas

  • There are two main questions confused me most:

    • initial input feature: the graph structure is invariable, while the features constantly changes each signal cycle as every cycle exists only one maximum queuing length for each direction.

      But as I learned, the input graph is always only one. Variable is just the size of the graph input, through the technique of mini-batch or sampling. And it seems that I cannot train my RGCN with the changed different node features, since they are different graphs describing different signal cycle but definitely portray the same intersection which I need to get the node embeddings.

      So how can I make the queuing length number value to be a node feature. And how to use the large amount of data to train the RGCN. It will help me a lot!

    • train part: Since the mission is just to embed the node. And the downstream task is as the input of reinforcement learning, it is different from traditional classification or prediction tasks.

      In my opinion, it should adopt the unsupervised training method like the word2vec. But should I use the dot product to express the similarity of nodes? I think it can be used when the embedding feature representing can reflect the structure similarity.

Your advice will be of great help to me! Sincerely thank you！

[BarclayII]

My two cents:

1. Were you trying to control only one traffic intersection, or actually a road network where you need to control multiple junctions at the same time (perhaps cooperatively)? If you are only controlling one junction, then probably you don’t even need a graph neural network, as your graph is a very small (4-node) complete graph where an MLP is usually enough.

2. You said that your mission is “to learn node embedding”, but I doubt that normal node embedding approaches (network embeddings, GNNs for node classification or link prediction, etc.) fits your scenario well, because your graph is too small and I doubt if any node embedding learned on this graph would be meaningful. I would as well learn a global state vector for this entire junction with an LSTM, taking in the queuing lengths at 4 directions as a 4-dimensional sequence, and train your policy with any reinforcement learning approaches (PG, actor-critic, whatever) end-to-end.

[bowlmeat]

Thank you so much!
Actually this GNN idea comes from a paper Transferable traffic signal control: Reinforcement learning with graph centric state representation.

And for the two thoughts:

1. Yes. There exists only one intersection. My goal is to obtain a green time ratio in one cycle which can fit the traffic best as the GNN outcome will be input to the RL.
2. To be honest, I also feel confused when I deal with it although I am new in GNN. But the core idea why GNN is that through GNN we can achieve the transferability of the policy with the background that traffic signal control is special for its “Scenario-based exploration”. So we need the transferable policy with which the model has not been trained but can also perform well.

Thank you again! Your help really means a lot to me!