aiqbench analysis
This paper introduces DTA-GT, a novel graph transformer that explicitly preserves directional computational semantics within directed acyclic graphs to improve neural architecture search. By integrating direction-aware initialization and spectral encoding, the model achieves superior accuracy and latency prediction on standard NAS benchmarks compared to existing sequence- and GNN-based predictors. The study validates encoder-wide directional consistency as a critical design principle for optimizing neural network representation learning.
DTA-GT: Direction- and Topology-Aware Graph Transformer for Neural Network Representation Learning
arXiv:2610.16822v1 Announce Type: cross Abstract: Architecture attribute predictors reduce the cost of neural architecture search by estimating accuracy or latency from a small set of evaluated candidates. The key difficulty is that neural architectures are operation-labeled DAGs: edge direction defines computational flow, but existing predictors often treat direction as a local structural cue. We propose DTA-GT, a Direction- and Topology-Aware Graph Transformer based on encoder-wide directional consistency, which preserves directed computational semantics across node initialization, global spectral encoding, pairwise interaction, and feature update. DTA-GT realizes this principle with direction- and topology-aware node initialization, Magnetic Laplacian spectral encoding, direction-dual structural attention, and a Direction-Sensitive MoE-FFN. Across accuracy and latency prediction on NAS-Bench-101 and NAS-Bench-201, DTA-GT consistently outperforms representative sequence-, GNN-, Transformer-, and hybrid predictors under limited-budget settings. Ablations and Magnetic spectral controls show that the improvements come from coordinated direction-aware stages and edge-orientation-sensitive spectral information, while mechanistic diagnostics confirm that the modules exhibit the intended directional behavior. These results support encoder-wide directional consistency as an effective design principle for architecture DAG representation.
Abstract:Architecture attribute predictors reduce the cost of neural architecture search by estimating accuracy or latency from a small set of evaluated candidates. The key difficulty is that neural architectures are operation-labeled DAGs: edge direction defines computational flow, but existing predictors often treat direction as a local structural cue. We propose DTA-GT, a Direction- and Topology-Aware Graph Transformer based on encoder-wide directional consistency, which preserves directed computational semantics across node initialization, global spectral encoding, pairwise interaction, and feature update. DTA-GT realizes this principle with direction- and topology-aware node initialization, Magnetic Laplacian spectral encoding, direction-dual structural attention, and a Direction-Sensitive MoE-FFN. Across accuracy and latency prediction on NAS-Bench-101 and NAS-Bench-201, DTA-GT consistently outperforms representative sequence-, GNN-, Transformer-, and hybrid predictors under limited-budget settings. Ablations and Magnetic spectral controls show that the improvements come from coordinated direction-aware stages and edge-orientation-sensitive spectral information, while mechanistic diagnostics confirm that the modules exhibit the intended directional behavior. These results support encoder-wide directional consistency as an effective design principle for architecture DAG representation.
Accepted at NeurIPS 2026. 36 pages, 6 figures
Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Neural and Evolutionary Computing (cs.NE)
arXiv:2610.16822 [cs.LG]
(or arXiv:2610.16822v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2610.16822 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 276 words
Related coverage
Exploring Hierarchical Interest Representation For Meta Ads Deep Funnel Optimization
Meta Engineering · Jul 15, 2026
Physics-Aware Random Walk Fingerprints for Scalable Power Grid Graph Classification
arXiv · Sep 7, 2026
Graph-centric agentic intelligence
Amazon Science · Oct 1, 2026
GLOW: Graph-Language Co-Encoding for Agentic Workflow Performance Prediction
arXiv · Sep 7, 2026