Tracing Hyperparameter Dependencies for Model Parsing via Learnable Graph Pooling Network

LGPN extracts generation traces and uses graph pooling and unpooling to predict connected model-hyperparameter attributes. — Hyperparameters are treated as related graph nodes rather than independent prediction targets. Figure 2 from the NeurIPS 2024 proceedings paper. Select the figure to enlarge it.

Overview

Generator attributes are not independent: architecture and training choices often have structure. This work uses that structure to improve model parsing from generated images.

How it works

A Generation Trace Capturing Network extracts cues left by the generating model.
The Learnable Graph Pooling Network represents hyperparameter values as nodes and dependencies as edges.
Pooling and unpooling refine those relationships before predicting loss functions and discrete or continuous architecture attributes.

The task covers 37 hyperparameters: 10 loss-function attributes, 18 discrete architecture attributes, and 9 continuous attributes.

A reported result

79.5% F1 for discrete architecture attributes, versus 73.0% for FEN-PN, on RED140, Table 1.

The benchmark contains 140,000 generated images from 140 models. Scores are averaged across the paper’s model-held-out evaluations. Loss-function F1 and continuous-attribute error are separate metrics.

Scope

The predictions concern the attributes defined by the benchmark. They do not recover an arbitrary model’s exact weights or complete implementation, and learned dependencies are not proof of causal relationships.

Source

Proceedings paper: Sections 3-4 and Table 1

Bibliographic reference

Xiao Guo, Vishal Asnani, Sijia Liu, Xiaoming Liu. (2024). Tracing Hyperparameter Dependencies for Model Parsing via Learnable Graph Pooling Network.