Towards Neural Scaling Laws for Foundation Models on Temporal Graphs

Temporal Graph Foundation Model Benchmark
Dataset Benchmark Implementation Dataset ML Croissant
Introduction

We work with top brands and startups

TGS provides the implementation of the TGS foundation model benchmarking and includes links to temporal networks suitable for foundation model training. TGS introduces a training process for foundation models using various real-world temporal networks, enabling prediction on previously unseen networks.

Temporal graph learning focuses on predicting future interactions from evolving network data. Our study addresses whether it's possible to predict the evolution of an unseen network within the same domain using observed temporal graphs. We introduce the Temporal Graph Scaling (TGS) dataset, comprising 84 ERC20 token transaction networks collected from 2017 to 2023. To evaluate transferability, we pre-train Temporal Graph Neural Networks (TGNNs) on up to 64 token transaction networks and assess their performance on 20 unseen token types. Our findings reveal that the neural scaling law observed in NLP and Computer Vision also applies to temporal graph learning: pre-training on more networks with more parameters enhances downstream performance. This is the first empirical demonstration of temporal graph transferability. Notably, the largest pre-trained model surpasses fine-tuned TGNNs on unseen test networks, marking a significant step towards building foundation models for temporal graphs. The code and datasets are publicly available.

TGS Benchmark

transaction networks are divided randomly into train and test sets. The train set is used to train foundation models with different sizes; then, the trained models are evaluated on the test set. Test AUC of foundation models trained on 4, 16 and 64 networks and evaluated on unseen test datasets. We compare the performance with persistence forecast, and single models trained and tested on each dataset.

TGS System Overview

TGS has three main steps

  1. (1) Token extraction: extracting the token transaction network from our P2P Ethereum live node.
  2. (2) Discretization: creating weekly snapshots to form discrete time dynamic graphs.
  3. (3) Foundation Model Training: TGS transaction networks are divided randomly into train and test sets. We train the FMs on a collection of training networks. Lastly, FMs are tested on 20 unseen test networks.

Metadata

Datasets provided in this storage are introduced in the paper: Towards Neural Scaling Laws for Foundation Models on Temporal Graphs Each .csv file represents all transactions of the token network that has the same name as the file name (tokenname.csv) Each transaction corresponds to a row in each file.

  1. blockNumber : is the block ID of Ethereum that includes this transaction
  2. timestamp: time that the transaction is made in UNIX timestamp format
  3. tokenAddress : the address that specifies a unique ERC20 token
  4. from: address of sender
  5. to: address of receiver
  6. value: the amount the transaction
  7. fileBlock: we split the whole number of blocks count to 35 buckets and assigned the bucket ID to the transaction to trace the blocks



Raw .csv will be used to divide into generate edgeslist and label, which indicates all node interactions and labels for each snapshot respectively, with help of functions from TGS_Handler defined in TGS.py inside the TGS package (see the code in the Github repository provided along with this data storage) 

                    

                    Data Example:

                    blockNumber,timestamp,tokenAddress,from,to,value,fileBlock
                    16480746,1674612419,0x5a3e6a77ba2f983ec0d371ea3b475f8bc0811ad5,0x0000000000000000000000000000000000000000,0x0000ba9ff5c97f33bd62c216a56b3d02ae6ac4bb,1E+18,26