aiqbench analysis
ZoAQ introduces a novel adaptive zeroth-order optimization framework that significantly reduces query costs by reusing past evaluations rather than discarding them. This method achieves substantial efficiency gains in black-box attack scenarios and large-scale model fine-tuning without requiring additional validation queries. The research advances the practicality of gradient-free learning, which is critical for optimizing systems where derivative information is unavailable or expensive to obtain.
ZoAQ: Adaptive Zeroth-Order Querying via Query-Reuse Coupling
arXiv:2609.22115v1 Announce Type: new Abstract: Zeroth-order optimization (ZOO) estimates updates from function evaluations, making perturbation queries a primary cost. Fixed budgets spend the same number of queries at every step, while adaptive controllers may offset their savings by using additional oracle calls to test estimator reliability. We introduce ZoAQ, an adaptive ZOO method built around query reuse. Rather than discarding past evaluations after each step, ZoAQ makes them useful for both the next update and the decision to query further. This enables adaptive query allocation without extra validation queries. Our analysis characterizes when this agreement identifies an update that supports descent and guides the controller to a sufficient query budget. On synthetic objectives, ZoAQ reduces queries by 43-48% relative to fixed baselines using 1.2M queries. In black-box attacks, it reaches 100% success with 320 and 625 average queries on MNIST and CIFAR-10, respectively. Across four OPT fine-tuning settings, ZoAQ saves 43-46% forward evaluations relative to fixed K=4, with accuracy changes within tasks ranging from -0.018 to +0.010.
Abstract:Zeroth-order optimization (ZOO) estimates updates from function evaluations, making perturbation queries a primary cost. Fixed budgets spend the same number of queries at every step, while adaptive controllers may offset their savings by using additional oracle calls to test estimator reliability. We introduce ZoAQ, an adaptive ZOO method built around query reuse. Rather than discarding past evaluations after each step, ZoAQ makes them useful for both the next update and the decision to query further. This enables adaptive query allocation without extra validation queries. Our analysis characterizes when this agreement identifies an update that supports descent and guides the controller to a sufficient query budget. On synthetic objectives, ZoAQ reduces queries by 43-48% relative to fixed baselines using 1.2M queries. In black-box attacks, it reaches 100% success with 320 and 625 average queries on MNIST and CIFAR-10, respectively. Across four OPT fine-tuning settings, ZoAQ saves 43-46% forward evaluations relative to fixed K=4, with accuracy changes within tasks ranging from -0.018 to +0.010.
Machine Learning (cs.LG)
arXiv:2609.22115 [cs.LG]
(or arXiv:2609.22115v1 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2609.22115 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 254 words
Related coverage
On Global Joint Lower Bounds in Zeroth-Order Nonconvex Optimization
arXiv · Oct 15, 2026
VLA-ZO: Fast Zeroth-Order Adaptation for Vision-Language-Action Models
arXiv · Oct 7, 2026
Inference-Native Zeroth-Order Optimization
arXiv · Sep 3, 2026
Building Blocks for Foundation Model Training and Inference on AWS
Hugging Face · May 11, 2026