Sari la conținut
aiqbench

Română · beta. Traducerea a fost făcută automat și este încă în curs de revizuire. Citește pagina în engleză

perspectiva aiqbenchMachine learning

analiza aiqbench

This research investigates the recall capabilities of a fixed-size recurrent memory architecture, demonstrating high accuracy in retrieving specific key-value associations within constrained matrix states. The study highlights critical dependencies on training configurations and token context length while noting current limitations in comparing memory efficiency against standard Transformer baselines due to their failure to achieve comparable performance under similar conditions.
Sursă care acoperăarXiv
Ce este aceastaLectura unui model asupra articolului și a acoperirii conexe, scrisă pentru aiqbench. Nu sunt cuvintele publicației.

Constant-Memory Recall: Learned Associations in a Fixed Matrix State

arXiv2 min de cititArticolul și perspectiva sunt în engleză.

arXiv:2610.00232v1 Announce Type: new Abstract: Fixed-size recurrent memory limits storage growth during inference, but successful recall depends on the task and training. We study a small DeltaNet variant with fixed token-specific key biases, trained to remember 32 new key-value pairings per sequence. With 32 KiB of recurrent matrix state, it achieves 99.95% mean accuracy across three training seeds when choosing among the sequence's values. Recall remains near perfect when filler extends the pre-query context to 1,798 tokens without adding pairings. Zeroing the first memory block removes this recall. An exploratory 48-pair test remains near chance after one quarter of the primary training budget and does not locate a capacity limit. Parameter-matched vector and Transformer baselines remain near chance, including the Transformer after additional training searches. This unresolved baseline failure prevents a memory-efficiency comparison.

Abstract:Fixed-size recurrent memory limits storage growth during inference, but successful recall depends on the task and training. We study a small DeltaNet variant with fixed token-specific key biases, trained to remember 32 new key-value pairings per sequence. With 32 KiB of recurrent matrix state, it achieves 99.95% mean accuracy across three training seeds when choosing among the sequence's values. Recall remains near perfect when filler extends the pre-query context to 1,798 tokens without adding pairings. Zeroing the first memory block removes this recall. An exploratory 48-pair test remains near chance after one quarter of the primary training budget and does not locate a capacity limit. Parameter-matched vector and Transformer baselines remain near chance, including the Transformer after additional training searches. This unresolved baseline failure prevents a memory-efficiency comparison.

Machine Learning (cs.LG)

arXiv:2610.00232 [cs.LG]

(or arXiv:2610.00232v1 [cs.LG] for this version)

https://doi.org/10.48550/arXiv.2610.00232 Focus to learn more arXiv-issued DOI via DataCite (pending registration)

HISTORICAL_EXCERPTS.md

PUBLICATION_CHECK.json

experiment-runs/2026-07-09_h2h_calib3/results/h2h_calib_ablation_task1_calib_primary_K32_auxrev2.json

experiment-runs/2026-07-09_h2h_calib3/results/h2h_calib_ablation_task1_calib_stress_locate_only_K48_auxrev2.json

experiment-runs/2026-07-09_h2h_calib3/results/h2h_calib_contender_task1_calib_primary_K32_auxrev2.json

experiment-runs/2026-07-09_h2h_calib3/results/h2h_calib_contender_task1_calib_stress_locate_only_K48_auxrev2.json

experiment-runs/2026-07-09_h2h_tap_localization/results/tap_localization_ablation.json

experiment-runs/2026-07-09_h2h_tap_localization/results/tap_localization_contender.json

experiment-runs/2026-07-10_h2h_mstar/MSTAR_VERDICT.json

experiment-runs/2026-07-10_h2h_mstar/fanout/contender_horizon_refs.json

experiment-runs/2026-07-10_h2h_sweep_harvest/h2h_ablation_task1_sweep_s0.json

experiment-runs/2026-07-10_h2h_sweep_harvest/h2h_contender_task1_sweep_s0.json

experiment-runs/2026-07-10_h2h_sweep_harvest/h2h_transformer_task1_sweep_s0.json

experiment-runs/2026-07-10_h2h_sweep_harvest/sweep_remetric/h2h_ablation_task1_sweep_s0_round4.json

experiment-runs/2026-07-10_h2h_sweep_harvest/sweep_remetric/h2h_ablation_task1_sweep_s1_round4.json

experiment-runs/2026-07-10_h2h_sweep_harvest/sweep_remetric/h2h_ablation_task1_sweep_s2_round4.json

experiment-runs/2026-07-10_h2h_sweep_harvest/sweep_remetric/h2h_contender_task1_sweep_s0_round4.json

experiment-runs/2026-07-10_h2h_sweep_harvest/sweep_remetric/h2h_contender_task1_sweep_s1_round4.json

experiment-runs/2026-07-10_h2h_sweep_harvest/sweep_remetric/h2h_contender_task1_sweep_s2_round4.json

experiment-runs/2026-07-10_h2h_sweep_harvest/sweep_remetric/h2h_transformer_task1_sweep_s0_round4.json

experiment-runs/2026-07-10_h2h_sweep_harvest/sweep_remetric/h2h_transformer_task1_sweep_s1_round4.json

experiment-runs/2026-07-10_h2h_sweep_harvest/sweep_remetric/h2h_transformer_task1_sweep_s2_round4.json

experiment-runs/2026-07-10_h2h_task2diag/results/TASK2DIAG_VERDICT.json

transformer-search/PROVENANCE_AND_CHECK.json

transformer-search/experiment-runs/2026-07-10_h2h_sweep_harvest/h2h_transformer_task1_sweep_s0.json

transformer-search/experiment-runs/2026-07-10_h2h_sweep_harvest/h2h_transformer_task1_sweep_s1.json

transformer-search/experiment-runs/2026-07-10_h2h_sweep_harvest/h2h_transformer_task1_sweep_s2.json

transformer-search/experiment-runs/2026-07-10_h2h_sweep_harvest/sweep_remetric/h2h_transformer_task1_sweep_s0_round4.json

transformer-search/experiment-runs/2026-07-10_h2h_sweep_harvest/sweep_remetric/h2h_transformer_task1_sweep_s1_round4.json

transformer-search/experiment-runs/2026-07-10_h2h_sweep_harvest/sweep_remetric/h2h_transformer_task1_sweep_s2_round4.json

transformer-search/experiment-runs/2026-07-11_h2h_fix5_lrgrid/results/h2h_fix5_transformer_task1_lr1e-03_s0.json

transformer-search/experiment-runs/2026-07-11_h2h_fix5_lrgrid/results/h2h_fix5_transformer_task1_lr1e-03_s1.json

transformer-search/experiment-runs/2026-07-11_h2h_fix5_lrgrid/results/h2h_fix5_transformer_task1_lr1e-03_s2.json

transformer-search/experiment-runs/2026-07-11_h2h_fix5_lrgrid/results/remetric/h2h_fix5_transformer_task1_lr1e-03_s0_round4.json

transformer-search/experiment-runs/2026-07-11_h2h_fix5_lrgrid/results/remetric/h2h_fix5_transformer_task1_lr1e-03_s1_round4.json

transformer-search/experiment-runs/2026-07-11_h2h_fix5_lrgrid/results/remetric/h2h_fix5_transformer_task1_lr1e-03_s2_round4.json

transformer-search/experiment-runs/2026-09-03_recall_strengthen/STRENGTHEN_VERDICT.json

transformer-search/experiment-runs/2026-09-03_recall_strengthen/h2h_strengthen_C0_lr1e-03_st60000_s0.json

transformer-search/experiment-runs/2026-09-03_recall_strengthen/h2h_strengthen_C0_lr1e-03_st60000_s1.json

transformer-search/experiment-runs/2026-09-03_recall_strengthen/h2h_strengthen_C0_lr1e-03_st60000_s2.json

transformer-search/experiment-runs/2026-09-03_recall_strengthen/h2h_strengthen_C0_lr3e-04_st60000_s0.json

transformer-search/experiment-runs/2026-09-03_recall_strengthen/h2h_strengthen_C0_lr3e-04_st60000_s1.json

transformer-search/experiment-runs/2026-09-03_recall_strengthen/h2h_strengthen_C0_lr3e-04_st60000_s2.json

transformer-search/experiment-runs/2026-09-03_recall_strengthen/h2h_strengthen_C1_lr1e-03_st20000_s0.json

transformer-search/experiment-runs/2026-09-03_recall_strengthen/h2h_strengthen_C1_lr1e-03_st20000_s1.json

transformer-search/experiment-runs/2026-09-03_recall_strengthen/h2h_strengthen_C1_lr1e-03_st20000_s2.json

transformer-search/experiment-runs/2026-09-03_recall_strengthen/h2h_strengthen_C1_lr1e-03_st60000_s0.json

transformer-search/experiment-runs/2026-09-03_recall_strengthen/h2h_strengthen_C1_lr1e-03_st60000_s1.json

transformer-search/experiment-runs/2026-09-03_recall_strengthen/h2h_strengthen_C1_lr1e-03_st60000_s2.json

transformer-search/experiment-runs/2026-09-03_recall_strengthen/h2h_strengthen_C1_lr3e-04_st20000_s0.json

transformer-search/experiment-runs/2026-09-03_recall_strengthen/h2h_strengthen_C1_lr3e-04_st20000_s1.json

transformer-search/experiment-runs/2026-09-03_recall_strengthen/h2h_strengthen_C1_lr3e-04_st20000_s2.json

transformer-search/experiment-runs/2026-09-03_recall_strengthen/h2h_strengthen_C1_lr3e-04_st60000_s0.json

transformer-search/experiment-runs/2026-09-03_recall_strengthen/h2h_strengthen_C1_lr3e-04_st60000_s1.json

transformer-search/experiment-runs/2026-09-03_recall_strengthen/h2h_strengthen_C1_lr3e-04_st60000_s2.json

transformer-search/experiment-runs/2026-09-03_recall_strengthen/h2h_strengthen_C2_lr1e-03_st20000_s0.json

transformer-search/experiment-runs/2026-09-03_recall_strengthen/h2h_strengthen_C2_lr1e-03_st20000_s1.json

transformer-search/experiment-runs/2026-09-03_recall_strengthen/h2h_strengthen_C2_lr1e-03_st20000_s2.json

transformer-search/experiment-runs/2026-09-03_recall_strengthen/h2h_strengthen_C2_lr1e-03_st60000_s0.json

transformer-search/experiment-runs/2026-09-03_recall_strengthen/h2h_strengthen_C2_lr1e-03_st60000_s1.json

transformer-search/experiment-runs/2026-09-03_recall_strengthen/h2h_strengthen_C2_lr1e-03_st60000_s2.json

transformer-search/experiment-runs/2026-09-03_recall_strengthen/h2h_strengthen_C2_lr3e-04_st20000_s0.json

transformer-search/experiment-runs/2026-09-03_recall_strengthen/h2h_strengthen_C2_lr3e-04_st20000_s1.json

transformer-search/experiment-runs/2026-09-03_recall_strengthen/h2h_strengthen_C2_lr3e-04_st20000_s2.json

transformer-search/experiment-runs/2026-09-03_recall_strengthen/remetric/h2h_strengthen_C0_lr1e-03_st60000_s0_round4.json

transformer-search/experiment-runs/2026-09-03_recall_strengthen/remetric/h2h_strengthen_C0_lr1e-03_st60000_s1_round4.json

transformer-search/experiment-runs/2026-09-03_recall_strengthen/remetric/h2h_strengthen_C0_lr1e-03_st60000_s2_round4.json

transformer-search/experiment-runs/2026-09-03_recall_strengthen/remetric/h2h_strengthen_C0_lr3e-04_st60000_s0_round4.json

transformer-search/experiment-runs/2026-09-03_recall_strengthen/remetric/h2h_strengthen_C0_lr3e-04_st60000_s1_round4.json

transformer-search/experiment-runs/2026-09-03_recall_strengthen/remetric/h2h_strengthen_C0_lr3e-04_st60000_s2_round4.json

transformer-search/experiment-runs/2026-09-03_recall_strengthen/remetric/h2h_strengthen_C1_lr1e-03_st20000_s0_round4.json

transformer-search/experiment-runs/2026-09-03_recall_strengthen/remetric/h2h_strengthen_C1_lr1e-03_st20000_s1_round4.json

transformer-search/experiment-runs/2026-09-03_recall_strengthen/remetric/h2h_strengthen_C1_lr1e-03_st20000_s2_round4.json

transformer-search/experiment-runs/2026-09-03_recall_strengthen/remetric/h2h_strengthen_C1_lr1e-03_st60000_s0_round4.json

transformer-search/experiment-runs/2026-09-03_recall_strengthen/remetric/h2h_strengthen_C1_lr1e-03_st60000_s1_round4.json

transformer-search/experiment-runs/2026-09-03_recall_strengthen/remetric/h2h_strengthen_C1_lr1e-03_st60000_s2_round4.json

transformer-search/experiment-runs/2026-09-03_recall_strengthen/remetric/h2h_strengthen_C1_lr3e-04_st20000_s0_round4.json

transformer-search/experiment-runs/2026-09-03_recall_strengthen/remetric/h2h_strengthen_C1_lr3e-04_st20000_s1_round4.json

transformer-search/experiment-runs/2026-09-03_recall_strengthen/remetric/h2h_strengthen_C1_lr3e-04_st20000_s2_round4.json

transformer-search/experiment-runs/2026-09-03_recall_strengthen/remetric/h2h_strengthen_C1_lr3e-04_st60000_s0_round4.json

transformer-search/experiment-runs/2026-09-03_recall_strengthen/remetric/h2h_strengthen_C1_lr3e-04_st60000_s1_round4.json

transformer-search/experiment-runs/2026-09-03_recall_strengthen/remetric/h2h_strengthen_C1_lr3e-04_st60000_s2_round4.json

transformer-search/experiment-runs/2026-09-03_recall_strengthen/remetric/h2h_strengthen_C2_lr1e-03_st20000_s0_round4.json

transformer-search/experiment-runs/2026-09-03_recall_strengthen/remetric/h2h_strengthen_C2_lr1e-03_st20000_s1_round4.json

transformer-search/experiment-runs/2026-09-03_recall_strengthen/remetric/h2h_strengthen_C2_lr1e-03_st20000_s2_round4.json

transformer-search/experiment-runs/2026-09-03_recall_strengthen/remetric/h2h_strengthen_C2_lr1e-03_st60000_s0_round4.json

transformer-search/experiment-runs/2026-09-03_recall_strengthen/remetric/h2h_strengthen_C2_lr1e-03_st60000_s1_round4.json

transformer-search/experiment-runs/2026-09-03_recall_strengthen/remetric/h2h_strengthen_C2_lr1e-03_st60000_s2_round4.json

transformer-search/experiment-runs/2026-09-03_recall_strengthen/remetric/h2h_strengthen_C2_lr3e-04_st20000_s0_round4.json

transformer-search/experiment-runs/2026-09-03_recall_strengthen/remetric/h2h_strengthen_C2_lr3e-04_st20000_s1_round4.json

transformer-search/experiment-runs/2026-09-03_recall_strengthen/remetric/h2h_strengthen_C2_lr3e-04_st20000_s2_round4.json

(90 additional files not shown) You must enabled JavaScript to view entire file list.

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

Text integral sub licența deschisă a publicației · 327 cuvinte

Citește pe arXiv ↗Întreabă despre acest articol

Subiecte conexe

Cercetare AI

J++ Lens: Jacobian Filtering Enables More Faithful Workspace Lenses

LessWrong · 8 oct. 2026

Machine learning

Can we change an AI model’s architecture without retraining it from scratch?

Amazon Science · 12 oct. 2026

Produse AI

DI Series: Serving Kimi-K3 MXFP4 Wide-EP Disaggregated on AMD Instinct MI300X and MI325X

AMD · 8 oct. 2026

Koopman Theory and Metaethics

LessWrong · 19 sept. 2026