analiza aiqbench
This research introduces NPBoost, a novel meta-learning framework that integrates gradient-boosted fixed effects with Neural Processes to better capture shared hierarchical patterns across tasks. By jointly training tree ensembles for common structures and neural networks for task-specific residuals, the method effectively handles discontinuities in tabular data where standard NPs struggle. The primary contribution lies in advancing model-based meta-learning architectures through a specific decomposition of response variability rather than general infrastructure or policy shifts.
NPBoost: Neural Processes with Gradient-Boosted Fixed Effects
arXiv:2609.28122v1 Announce Type: cross Abstract: Neural Processes (NPs) are model-based meta-learners that implicitly learn a stochastic process and adapt to a new task from a small context set. Most extensions of NPs focus on improving the neural network architecture. We instead develop an extension motivated by the shared hierarchical interpretation of meta-learning and mixed-effects models. Specifically, we introduce Neural Process Boosting (NPBoost), which decomposes structured response variability into tree-boosted fixed effects shared across tasks and NP random effects that capture stochastic task-to-task variation. We propose to train the two components jointly using a boosting algorithm in which an NP learns residual task-specific structure and a tree ensemble estimates common patterns across tasks. Across synthetic and real-world tabular meta-learning problems, this decomposition improves over a standard NP when the shared structure contains discontinuities or other irregular patterns that boosted trees can represent effectively.
Abstract:Neural Processes (NPs) are model-based meta-learners that implicitly learn a stochastic process and adapt to a new task from a small context set. Most extensions of NPs focus on improving the neural network architecture. We instead develop an extension motivated by the shared hierarchical interpretation of meta-learning and mixed-effects models. Specifically, we introduce Neural Process Boosting (NPBoost), which decomposes structured response variability into tree-boosted fixed effects shared across tasks and NP random effects that capture stochastic task-to-task variation. We propose to train the two components jointly using a boosting algorithm in which an NP learns residual task-specific structure and a tree ensemble estimates common patterns across tasks. Across synthetic and real-world tabular meta-learning problems, this decomposition improves over a standard NP when the shared structure contains discontinuities or other irregular patterns that boosted trees can represent effectively.
Machine Learning (stat.ML); Machine Learning (cs.LG)
arXiv:2609.28122 [stat.ML]
(or arXiv:2609.28122v1 [stat.ML] for this version)
https://doi.org/10.48550/arXiv.2609.28122 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 234 cuvinte
Subiecte conexe
Links #5: 2026/07
LessWrong · 31 iul. 2026
Handling Imbalanced Classification: What Works Better Than SMOTE
Analytics Vidhya · 12 iul. 2026
[AINews] Reality Checks on AI News (Yegge shuts down Gas Town, Databricks’ +60% Astra cost)
Latent Space · 17 sept. 2026
I trained GPT-2 with full-batch gradient descent so you don't have to
LessWrong · 14 oct. 2026