Sari la conținut

Română · beta. Traducerea a fost făcută automat și este încă în curs de revizuire. Citește pagina în engleză

perspectiva aiqbenchCercetare AI

analiza aiqbench

This research demonstrates that for long-document retrieval, the structural efficiency of a deterministic segment tree can outperform language model-generated summaries when evaluated at matched computational costs. By eliminating LLM calls during indexing and using the tree solely as a navigation scaffold, NavTree achieves superior recall in multi-hop QA tasks compared to both flat retrievers and standard hierarchical methods like RAPTOR. This finding shifts the focus from content generation to architectural optimization for scaling retrieval-augmented generation systems.
Sursă care acoperăarXiv
Ce este aceastaLectura unui model asupra articolului și a acoperirii conexe, scrisă pentru aiqbench. Nu sunt cuvintele publicației.

Tree Navigation Without LLM Summaries: A Matched-Cost Study of Hierarchical Retrieval for Long-Document QA

arXiv5 min de cititArticolul și perspectiva sunt în engleză.

arXiv:2610.06902v1 Announce Type: new Abstract: Retrieval-augmented generation grounds language models in external context, but for long documents flat top-$k$ retrieval can cluster on a single region and miss complementary evidence. RAPTOR-style summary trees address this by recursively clustering chunks and using a language model to summarize each cluster at indexing time, then ranking summary nodes alongside raw chunks at query time. We show the main benefit of summary trees in long-document QA can come from navigation rather than the generated summary content. We introduce NavTree, a leaves-only retriever that builds a deterministic balanced segment tree over chunks (zero language-model calls at indexing) and uses the tree purely as a navigation scaffold: a hybrid lexical-and-dense frontier walk, anchored on top retrieved leaves, descends from the root and emits only leaf chunks to the reader. On a matched-cost evaluation against flat retrievers and an extractive re-implementation of RAPTOR, NavTree is the strongest matched-cost hierarchical retriever in our evaluated grid and ties the strongest flat baseline. On long-document multi-hop QA, it is the only hierarchical method that significantly beats BM25 on a class-vs-class basis, corroborated by a reader-free retrieval-recall check. A matched-reader replication of the published abstractive RAPTOR variant, given strong cluster summaries, still loses to NavTree at every multi-chunk budget, at zero indexing cost. The ranking carries across stronger and open-weight readers, a stronger encoder, and a full factorial that isolates leaves-only emission as the structural lever.

Abstract:Retrieval-augmented generation grounds language models in external context, but for long documents flat top-$k$ retrieval can cluster on a single region and miss complementary evidence. RAPTOR-style summary trees address this by recursively clustering chunks and using a language model to summarize each cluster at indexing time, then ranking summary nodes alongside raw chunks at query time. We show the main benefit of summary trees in long-document QA can come from navigation rather than the generated summary content. We introduce NavTree, a leaves-only retriever that builds a deterministic balanced segment tree over chunks (zero language-model calls at indexing) and uses the tree purely as a navigation scaffold: a hybrid lexical-and-dense frontier walk, anchored on top retrieved leaves, descends from the root and emits only leaf chunks to the reader. On a matched-cost evaluation against flat retrievers and an extractive re-implementation of RAPTOR, NavTree is the strongest matched-cost hierarchical retriever in our evaluated grid and ties the strongest flat baseline. On long-document multi-hop QA, it is the only hierarchical method that significantly beats BM25 on a class-vs-class basis, corroborated by a reader-free retrieval-recall check. A matched-reader replication of the published abstractive RAPTOR variant, given strong cluster summaries, still loses to NavTree at every multi-chunk budget, at zero indexing cost. The ranking carries across stronger and open-weight readers, a stronger encoder, and a full factorial that isolates leaves-only emission as the structural lever.

Computation and Language (cs.CL); Information Retrieval (cs.IR)

arXiv:2610.06902 [cs.CL]

(or arXiv:2610.06902v1 [cs.CL] for this version)

https://doi.org/10.48550/arXiv.2610.06902 Focus to learn more arXiv-issued DOI via DataCite

REPRODUCIBILITY.md

code/asrmb/__init__.py

code/asrmb/baselines/__init__.py

code/asrmb/baselines/bm25.py

code/asrmb/baselines/dense_flat.py

code/asrmb/baselines/hybrid_flat.py

code/asrmb/baselines/raptor_baseline.py

code/asrmb/benchmarks/__init__.py

code/asrmb/benchmarks/base.py

code/asrmb/benchmarks/multihop.py

code/asrmb/benchmarks/narrativeqa.py

code/asrmb/benchmarks/qasper.py

code/asrmb/benchmarks/quality.py

code/asrmb/core/__init__.py

code/asrmb/core/inference.py

code/asrmb/core/packing.py

code/asrmb/core/segtree.py

code/asrmb/index/__init__.py

code/asrmb/index/faiss_utils.py

code/asrmb/retrieval/__init__.py

code/asrmb/retrieval/ce_rerank.py

code/asrmb/retrieval/navigated_fill.py

code/requirements.txt

code/scripts/bench_index_cost.py

code/scripts/bench_query_latency.py

code/scripts/build_anon_zip.sh

code/scripts/build_paper_figures.py

code/scripts/build_paper_tables.py

code/scripts/compute_confidence_intervals.py

code/scripts/diagnostics/qasper_tail_token_stats.py

code/scripts/exp_abstractive_raptor.py

code/scripts/exp_alpha_kleaf_sweep.sh

code/scripts/exp_factorial_w7_quality2000.sh

code/scripts/exp_hybrid_flat_multihop_fill.sh

code/scripts/exp_multi_seed.sh

code/scripts/exp_multihop.sh

code/scripts/exp_queue_remaining.sh

code/scripts/exp_summary_vs_leaf.py

code/scripts/exp_summary_vs_leaf_extractive.py

code/scripts/exp_winning_cell_ablation.sh

code/scripts/merge_hybrid_flat_multihop.py

code/scripts/paper_status.sh

code/scripts/phase1_build_contexts.py

code/scripts/phase1_longdoc.py

code/scripts/phase2_longdoc.py

code/scripts/phase2_run_reader.py

code/scripts/plot_results.py

code/scripts/post_qwen_qasper_finalize.sh

code/scripts/render_appendix_table.py

code/scripts/render_factorial_w7.py

code/scripts/render_table_multihop_recall.py

code/scripts/render_table_openweight.py

code/scripts/render_table_qasper_recall.py

code/scripts/rescore_nqa_summary_leaf.py

code/scripts/resume_arr_experiments.sh

code/scripts/run_a1_ollama.sh

code/scripts/run_a1_openweight.sh

code/scripts/run_a3_bge_large.sh

code/scripts/run_ablation_qasper256.sh

code/scripts/run_benchmarks.py

code/scripts/run_canonical_baseline.sh

code/scripts/run_multihop_recall.py

code/scripts/run_niah.py

code/scripts/run_niah_multi.py

code/scripts/run_overnight_orchestrator.sh

code/scripts/run_paired_tests.py

code/scripts/run_paired_tests_openweight.py

code/scripts/run_qasper_recall.py

code/scripts/sync_asrmb_paper_tables.py

code/scripts/sync_quality_asrmb_tables_from_canon.py

responsible_nlp_checklist.md

results/abstractive_raptor_gpt4o/quality_checkpoint.json

results/abstractive_raptor_gpt4o/quality_predictions.json

results/abstractive_raptor_gpt4o/quality_results.json

results/abstractive_raptor_gpt4o/summary.json

results/bge_large_run/qasper/qasper_checkpoint.json

results/bge_large_run/qasper/qasper_results.csv

results/bge_large_run/qasper/qasper_results.json

results/bge_large_run/quality/quality_checkpoint.json

results/bge_large_run/quality/quality_results.csv

results/bge_large_run/quality/quality_results.json

results/canonical_v1/narrativeqa/narrativeqa_checkpoint.json

results/canonical_v1/narrativeqa/narrativeqa_results.csv

results/canonical_v1/narrativeqa/narrativeqa_results.json

results/canonical_v1/qasper/qasper_checkpoint.json

results/canonical_v1/qasper/qasper_results.csv

results/canonical_v1/qasper/qasper_results.json

results/canonical_v1/quality/quality_checkpoint.json

results/canonical_v1/quality/quality_results.csv

results/canonical_v1/quality/quality_results.json

results/factorial_w7_quality2000/A0_B0_C0_D0/quality_checkpoint.json

results/factorial_w7_quality2000/A0_B0_C0_D0/quality_results.csv

results/factorial_w7_quality2000/A0_B0_C0_D0/quality_results.json

results/factorial_w7_quality2000/A0_B0_C0_D1/quality_checkpoint.json

results/factorial_w7_quality2000/A0_B0_C0_D1/quality_results.csv

results/factorial_w7_quality2000/A0_B0_C0_D1/quality_results.json

results/factorial_w7_quality2000/A0_B0_C1_D0/quality_checkpoint.json

results/factorial_w7_quality2000/A0_B0_C1_D0/quality_results.csv

results/factorial_w7_quality2000/A0_B0_C1_D0/quality_results.json

results/factorial_w7_quality2000/A0_B0_C1_D1/quality_checkpoint.json

results/factorial_w7_quality2000/A0_B0_C1_D1/quality_results.csv

results/factorial_w7_quality2000/A0_B0_C1_D1/quality_results.json

results/factorial_w7_quality2000/A0_B1_C0_D0/quality_checkpoint.json

results/factorial_w7_quality2000/A0_B1_C0_D0/quality_results.csv

results/factorial_w7_quality2000/A0_B1_C0_D0/quality_results.json

results/factorial_w7_quality2000/A0_B1_C0_D1/quality_checkpoint.json

results/factorial_w7_quality2000/A0_B1_C0_D1/quality_results.csv

results/factorial_w7_quality2000/A0_B1_C0_D1/quality_results.json

results/factorial_w7_quality2000/A0_B1_C1_D0/quality_checkpoint.json

results/factorial_w7_quality2000/A0_B1_C1_D0/quality_results.csv

results/factorial_w7_quality2000/A0_B1_C1_D0/quality_results.json

results/factorial_w7_quality2000/A0_B1_C1_D1/quality_checkpoint.json

results/factorial_w7_quality2000/A0_B1_C1_D1/quality_results.csv

results/factorial_w7_quality2000/A0_B1_C1_D1/quality_results.json

results/factorial_w7_quality2000/A1_B0_C0_D0/quality_checkpoint.json

results/factorial_w7_quality2000/A1_B0_C0_D0/quality_results.csv

results/factorial_w7_quality2000/A1_B0_C0_D0/quality_results.json

results/factorial_w7_quality2000/A1_B0_C0_D1/quality_checkpoint.json

results/factorial_w7_quality2000/A1_B0_C0_D1/quality_results.csv

results/factorial_w7_quality2000/A1_B0_C0_D1/quality_results.json

results/factorial_w7_quality2000/A1_B0_C1_D0/quality_checkpoint.json

results/factorial_w7_quality2000/A1_B0_C1_D0/quality_results.csv

results/factorial_w7_quality2000/A1_B0_C1_D0/quality_results.json

results/factorial_w7_quality2000/A1_B0_C1_D1/quality_checkpoint.json

results/factorial_w7_quality2000/A1_B0_C1_D1/quality_results.csv

results/factorial_w7_quality2000/A1_B0_C1_D1/quality_results.json

results/factorial_w7_quality2000/A1_B1_C0_D0/quality_checkpoint.json

results/factorial_w7_quality2000/A1_B1_C0_D0/quality_results.csv

results/factorial_w7_quality2000/A1_B1_C0_D0/quality_results.json

results/factorial_w7_quality2000/A1_B1_C0_D1/quality_checkpoint.json

results/factorial_w7_quality2000/A1_B1_C0_D1/quality_results.csv

results/factorial_w7_quality2000/A1_B1_C0_D1/quality_results.json

results/factorial_w7_quality2000/A1_B1_C1_D0/quality_checkpoint.json

results/factorial_w7_quality2000/A1_B1_C1_D0/quality_results.csv

results/factorial_w7_quality2000/A1_B1_C1_D0/quality_results.json

results/factorial_w7_quality2000/A1_B1_C1_D1/quality_checkpoint.json

results/factorial_w7_quality2000/A1_B1_C1_D1/quality_results.csv

results/factorial_w7_quality2000/A1_B1_C1_D1/quality_results.json

results/factorial_w7_quality2000/_summary.json

results/final_archive/README.md

results/final_archive/longdoc_ci.json

results/final_archive/longdoc_results.json

results/final_archive/longdoc_results.json.checkpoint

results/final_archive/narrativeqa_checkpoint.json

results/final_archive/narrativeqa_results.json

results/final_archive/qasper_checkpoint.json

results/final_archive/qasper_ci.json

results/final_archive/qasper_results.json

results/final_archive/qasper_unifiedqa_results.json

results/final_archive/quality_ci.json

results/final_archive/quality_navigated_checkpoint.json

results/final_archive/quality_results.json

results/final_archive/quality_unifiedqa_results.json

results/final_archive/quality_unifiedqa_results.json.checkpoint

results/frozen_opt/AUDIT.md

results/frozen_opt/ablation_qasper256/_meta/manifest.json

results/frozen_opt/ablation_qasper256/row1_classic_steiner/qasper_checkpoint.json

results/frozen_opt/ablation_qasper256/row1_classic_steiner/qasper_ci.json

results/frozen_opt/ablation_qasper256/row1_classic_steiner/qasper_results.csv

results/frozen_opt/ablation_qasper256/row1_classic_steiner/qasper_results.json

results/frozen_opt/ablation_qasper256/row2_kleaf48/qasper_checkpoint.json

results/frozen_opt/ablation_qasper256/row2_kleaf48/qasper_ci.json

results/frozen_opt/ablation_qasper256/row2_kleaf48/qasper_results.csv

results/frozen_opt/ablation_qasper256/row2_kleaf48/qasper_results.json

results/frozen_opt/ablation_qasper256/row3_hybrid_alpha035/qasper_checkpoint.json

results/frozen_opt/ablation_qasper256/row3_hybrid_alpha035/qasper_ci.json

results/frozen_opt/ablation_qasper256/row3_hybrid_alpha035/qasper_results.csv

results/frozen_opt/ablation_qasper256/row3_hybrid_alpha035/qasper_results.json

results/frozen_opt/ablation_qasper256/row4_alpha050/qasper_checkpoint.json

results/frozen_opt/ablation_qasper256/row4_alpha050/qasper_ci.json

results/frozen_opt/ablation_qasper256/row4_alpha050/qasper_results.csv

results/frozen_opt/ablation_qasper256/row4_alpha050/qasper_results.json

results/frozen_opt/ablation_qasper256/row5_tight_tail/qasper_checkpoint.json

results/frozen_opt/ablation_qasper256/row5_tight_tail/qasper_ci.json

results/frozen_opt/ablation_qasper256/row5_tight_tail/qasper_results.csv

results/frozen_opt/ablation_qasper256/row5_tight_tail/qasper_results.json

results/frozen_opt/ablation_qasper256/summary.md

results/frozen_opt/ablation_quality2000/row1_classic_steiner/quality_checkpoint.json

results/frozen_opt/ablation_quality2000/row1_classic_steiner/quality_results.csv

results/frozen_opt/ablation_quality2000/row1_classic_steiner/quality_results.json

results/frozen_opt/ablation_quality2000/row2_kleaf48/quality_checkpoint.json

results/frozen_opt/ablation_quality2000/row2_kleaf48/quality_results.csv

results/frozen_opt/ablation_quality2000/row2_kleaf48/quality_results.json

results/frozen_opt/ablation_quality2000/row3_hybrid_alpha035/quality_checkpoint.json

results/frozen_opt/ablation_quality2000/row3_hybrid_alpha035/quality_results.csv

results/frozen_opt/ablation_quality2000/row3_hybrid_alpha035/quality_results.json

results/frozen_opt/ablation_quality2000/row4_alpha050/quality_checkpoint.json

results/frozen_opt/ablation_quality2000/row4_alpha050/quality_results.csv

results/frozen_opt/ablation_quality2000/row4_alpha050/quality_results.json

results/frozen_opt/ablation_quality2000/row5_full/quality_checkpoint.json

results/frozen_opt/ablation_quality2000/row5_full/quality_results.csv

results/frozen_opt/ablation_quality2000/row5_full/quality_results.json

results/frozen_opt/abstractive_raptor/narrativeqa_results.json

results/frozen_opt/abstractive_raptor/qasper_results.json

results/frozen_opt/abstractive_raptor/quality_results.json

results/frozen_opt/abstractive_raptor/summary.json

results/frozen_opt/alpha_kleaf_sweep/alpha0.00_kleaf10/quality_checkpoint.json

results/frozen_opt/alpha_kleaf_sweep/alpha0.00_kleaf10/quality_results.csv

results/frozen_opt/alpha_kleaf_sweep/alpha0.00_kleaf10/quality_results.json

results/frozen_opt/alpha_kleaf_sweep/alpha0.25_kleaf10/quality_checkpoint.json

results/frozen_opt/alpha_kleaf_sweep/alpha0.25_kleaf10/quality_results.csv

results/frozen_opt/alpha_kleaf_sweep/alpha0.25_kleaf10/quality_results.json

results/frozen_opt/alpha_kleaf_sweep/alpha0.35_kleaf10/quality_checkpoint.json

results/frozen_opt/alpha_kleaf_sweep/alpha0.35_kleaf10/quality_results.csv

results/frozen_opt/alpha_kleaf_sweep/alpha0.35_kleaf10/quality_results.json

results/frozen_opt/alpha_kleaf_sweep/alpha0.35_kleaf24/quality_checkpoint.json

results/frozen_opt/alpha_kleaf_sweep/alpha0.35_kleaf24/quality_results.csv

results/frozen_opt/alpha_kleaf_sweep/alpha0.35_kleaf24/quality_results.json

results/frozen_opt/alpha_kleaf_sweep/alpha0.35_kleaf48/quality_checkpoint.json

results/frozen_opt/alpha_kleaf_sweep/alpha0.35_kleaf48/quality_results.csv

results/frozen_opt/alpha_kleaf_sweep/alpha0.35_kleaf48/quality_results.json

results/frozen_opt/alpha_kleaf_sweep/alpha0.35_kleaf5/quality_checkpoint.json

results/frozen_opt/alpha_kleaf_sweep/alpha0.35_kleaf5/quality_results.csv

results/frozen_opt/alpha_kleaf_sweep/alpha0.35_kleaf5/quality_results.json

results/frozen_opt/alpha_kleaf_sweep/alpha0.50_kleaf10/quality_checkpoint.json

results/frozen_opt/alpha_kleaf_sweep/alpha0.50_kleaf10/quality_results.csv

results/frozen_opt/alpha_kleaf_sweep/alpha0.50_kleaf10/quality_results.json

results/frozen_opt/alpha_kleaf_sweep/alpha0.65_kleaf10/quality_checkpoint.json

results/frozen_opt/alpha_kleaf_sweep/alpha0.65_kleaf10/quality_results.csv

results/frozen_opt/alpha_kleaf_sweep/alpha0.65_kleaf10/quality_results.json

results/frozen_opt/alpha_kleaf_sweep/alpha1.00_kleaf10/quality_checkpoint.json

results/frozen_opt/alpha_kleaf_sweep/alpha1.00_kleaf10/quality_results.csv

results/frozen_opt/alpha_kleaf_sweep/alpha1.00_kleaf10/quality_results.json

results/frozen_opt/canonical_v1/_meta/code_snapshot/CHECKSUMS.txt

results/frozen_opt/canonical_v1/_meta/code_snapshot/SNAPSHOT.txt

results/frozen_opt/canonical_v1/_meta/code_snapshot/bm25.py

results/frozen_opt/canonical_v1/_meta/code_snapshot/ce_rerank.py

results/frozen_opt/canonical_v1/_meta/code_snapshot/faiss_utils.py

results/frozen_opt/canonical_v1/_meta/code_snapshot/inference.py

results/frozen_opt/canonical_v1/_meta/code_snapshot/navigated_fill.py

results/frozen_opt/canonical_v1/_meta/code_snapshot/packing.py

results/frozen_opt/canonical_v1/_meta/code_snapshot/raptor_baseline.py

results/frozen_opt/canonical_v1/_meta/code_snapshot/run_benchmarks.py

results/frozen_opt/canonical_v1/_meta/code_snapshot/segtree.py

results/frozen_opt/canonical_v1/_meta/manifest.json

results/frozen_opt/canonical_v1/narrativeqa/narrativeqa_ci.json

results/frozen_opt/canonical_v1/narrativeqa/narrativeqa_ci_paired.json

results/frozen_opt/canonical_v1/narrativeqa/narrativeqa_predictions.json

results/frozen_opt/canonical_v1/narrativeqa/narrativeqa_predictions_raw.json

results/frozen_opt/canonical_v1/narrativeqa/narrativeqa_results.csv

results/frozen_opt/canonical_v1/narrativeqa/narrativeqa_results.json

results/frozen_opt/canonical_v1/qasper/qasper_ci.json

results/frozen_opt/canonical_v1/qasper/qasper_ci_paired.json

results/frozen_opt/canonical_v1/qasper/qasper_predictions.json

results/frozen_opt/canonical_v1/qasper/qasper_predictions_raw.json

results/frozen_opt/canonical_v1/qasper/qasper_results.csv

results/frozen_opt/canonical_v1/qasper/qasper_results.json

results/frozen_opt/canonical_v1/quality/quality_ci.json

results/frozen_opt/canonical_v1/quality/quality_ci_paired.json

results/frozen_opt/canonical_v1/quality/quality_predictions.json

results/frozen_opt/canonical_v1/quality/quality_predictions_raw.json

results/frozen_opt/canonical_v1/quality/quality_results.csv

results/frozen_opt/canonical_v1/quality/quality_results.json

results/frozen_opt/index_cost_bench/_meta/manifest.json

results/frozen_opt/index_cost_bench/per_doc.json

results/frozen_opt/index_cost_bench/summary.json

results/frozen_opt/index_cost_bench/summary.md

results/frozen_opt/multi_seed_quality1024/seed1/quality_checkpoint.json

results/frozen_opt/multi_seed_quality1024/seed1/quality_results.csv

results/frozen_opt/multi_seed_quality1024/seed1/quality_results.json

results/frozen_opt/multi_seed_quality1024/seed2/quality_results.json

results/frozen_opt/multi_seed_quality1024/seed2_rerun/quality_checkpoint.json

results/frozen_opt/multi_seed_quality1024/seed2_rerun/quality_results.csv

results/frozen_opt/multi_seed_quality1024/seed2_rerun/quality_results.json

results/frozen_opt/multi_seed_quality1024/seed3/quality_results.json

results/frozen_opt/multi_seed_quality1024/seed3_rerun/quality_checkpoint.json

results/frozen_opt/multi_seed_quality1024/seed3_rerun/quality_results.csv

results/frozen_opt/multi_seed_quality1024/seed3_rerun/quality_results.json

results/frozen_opt/multi_seed_quality1024/seed4/quality_results.json

results/frozen_opt/multi_seed_quality1024/seed4_rerun/quality_checkpoint.json

results/frozen_opt/multi_seed_quality1024/seed4_rerun/quality_results.csv

results/frozen_opt/multi_seed_quality1024/seed4_rerun/quality_results.json

results/frozen_opt/multi_seed_quality1024/seed5/quality_results.json

results/frozen_opt/multi_seed_quality1024/seed5_rerun/quality_checkpoint.json

results/frozen_opt/multi_seed_quality1024/seed5_rerun/quality_results.csv

results/frozen_opt/multi_seed_quality1024/seed5_rerun/quality_results.json

results/frozen_opt/nqa_alpha050/_meta/code_snapshot/CHECKSUMS.txt

results/frozen_opt/nqa_alpha050/_meta/code_snapshot/SNAPSHOT.txt

results/frozen_opt/nqa_alpha050/_meta/code_snapshot/bm25.py

results/frozen_opt/nqa_alpha050/_meta/code_snapshot/ce_rerank.py

results/frozen_opt/nqa_alpha050/_meta/code_snapshot/faiss_utils.py

results/frozen_opt/nqa_alpha050/_meta/code_snapshot/inference.py

results/frozen_opt/nqa_alpha050/_meta/code_snapshot/navigated_fill.py

results/frozen_opt/nqa_alpha050/_meta/code_snapshot/packing.py

results/frozen_opt/nqa_alpha050/_meta/code_snapshot/raptor_baseline.py

results/frozen_opt/nqa_alpha050/_meta/code_snapshot/run_benchmarks.py

results/frozen_opt/nqa_alpha050/_meta/code_snapshot/segtree.py

results/frozen_opt/nqa_alpha050/_meta/manifest.json

results/frozen_opt/nqa_alpha050/narrativeqa_cells.json

results/frozen_opt/nqa_alpha050/narrativeqa_ci.json

results/frozen_opt/nqa_alpha050/narrativeqa_predictions.json

results/frozen_opt/nqa_alpha050/narrativeqa_predictions_raw.json

results/frozen_opt/nqa_alpha050/narrativeqa_results.csv

results/frozen_opt/nqa_alpha050/narrativeqa_results.json

results/frozen_opt/nqa_short_prompt/_meta/code_snapshot/CHECKSUMS.txt

results/frozen_opt/nqa_short_prompt/_meta/code_snapshot/SNAPSHOT.txt

results/frozen_opt/nqa_short_prompt/_meta/code_snapshot/bm25.py

results/frozen_opt/nqa_short_prompt/_meta/code_snapshot/ce_rerank.py

results/frozen_opt/nqa_short_prompt/_meta/code_snapshot/faiss_utils.py

results/frozen_opt/nqa_short_prompt/_meta/code_snapshot/inference.py

results/frozen_opt/nqa_short_prompt/_meta/code_snapshot/navigated_fill.py

results/frozen_opt/nqa_short_prompt/_meta/code_snapshot/packing.py

results/frozen_opt/nqa_short_prompt/_meta/code_snapshot/raptor_baseline.py

results/frozen_opt/nqa_short_prompt/_meta/code_snapshot/run_benchmarks.py

results/frozen_opt/nqa_short_prompt/_meta/code_snapshot/segtree.py

results/frozen_opt/nqa_short_prompt/_meta/manifest.json

results/frozen_opt/nqa_short_prompt/narrativeqa_cells.json

results/frozen_opt/nqa_short_prompt/narrativeqa_ci.json

results/frozen_opt/nqa_short_prompt/narrativeqa_predictions.json

results/frozen_opt/nqa_short_prompt/narrativeqa_predictions_raw.json

results/frozen_opt/nqa_short_prompt/narrativeqa_results.csv

results/frozen_opt/nqa_short_prompt/narrativeqa_results.json

results/frozen_opt/qasper_ce/_meta/code_snapshot/CHECKSUMS.txt

results/frozen_opt/qasper_ce/_meta/code_snapshot/SNAPSHOT.txt

results/frozen_opt/qasper_ce/_meta/code_snapshot/bm25.py

results/frozen_opt/qasper_ce/_meta/code_snapshot/ce_rerank.py

results/frozen_opt/qasper_ce/_meta/code_snapshot/faiss_utils.py

results/frozen_opt/qasper_ce/_meta/code_snapshot/inference.py

results/frozen_opt/qasper_ce/_meta/code_snapshot/navigated_fill.py

results/frozen_opt/qasper_ce/_meta/code_snapshot/packing.py

results/frozen_opt/qasper_ce/_meta/code_snapshot/raptor_baseline.py

results/frozen_opt/qasper_ce/_meta/code_snapshot/run_benchmarks.py

results/frozen_opt/qasper_ce/_meta/code_snapshot/segtree.py

results/frozen_opt/qasper_ce/_meta/manifest.json

results/frozen_opt/qasper_ce/qasper_cells.json

results/frozen_opt/qasper_ce/qasper_ci.json

results/frozen_opt/qasper_ce/qasper_predictions.json

results/frozen_opt/qasper_ce/qasper_predictions_raw.json

results/frozen_opt/qasper_ce/qasper_results.csv

results/frozen_opt/qasper_ce/qasper_results.json

results/frozen_opt/quality_gpt4o_sota/_meta/code_snapshot/CHECKSUMS.txt

results/frozen_opt/quality_gpt4o_sota/_meta/code_snapshot/SNAPSHOT.txt

results/frozen_opt/quality_gpt4o_sota/_meta/code_snapshot/bm25.py

results/frozen_opt/quality_gpt4o_sota/_meta/code_snapshot/ce_rerank.py

results/frozen_opt/quality_gpt4o_sota/_meta/code_snapshot/faiss_utils.py

results/frozen_opt/quality_gpt4o_sota/_meta/code_snapshot/inference.py

results/frozen_opt/quality_gpt4o_sota/_meta/code_snapshot/navigated_fill.py

results/frozen_opt/quality_gpt4o_sota/_meta/code_snapshot/packing.py

results/frozen_opt/quality_gpt4o_sota/_meta/code_snapshot/raptor_baseline.py

results/frozen_opt/quality_gpt4o_sota/_meta/code_snapshot/run_benchmarks.py

results/frozen_opt/quality_gpt4o_sota/_meta/code_snapshot/segtree.py

results/frozen_opt/quality_gpt4o_sota/_meta/manifest.json

results/frozen_opt/quality_gpt4o_sota/quality_cells.json

results/frozen_opt/quality_gpt4o_sota/quality_ci.json

results/frozen_opt/quality_gpt4o_sota/quality_predictions.json

results/frozen_opt/quality_gpt4o_sota/quality_predictions_raw.json

results/frozen_opt/quality_gpt4o_sota/quality_results.csv

results/frozen_opt/quality_gpt4o_sota/quality_results.json

results/frozen_opt/quality_t3/_meta/code_snapshot/CHECKSUMS.txt

results/frozen_opt/quality_t3/_meta/code_snapshot/SNAPSHOT.txt

results/frozen_opt/quality_t3/_meta/code_snapshot/bm25.py

results/frozen_opt/quality_t3/_meta/code_snapshot/ce_rerank.py

results/frozen_opt/quality_t3/_meta/code_snapshot/faiss_utils.py

results/frozen_opt/quality_t3/_meta/code_snapshot/inference.py

results/frozen_opt/quality_t3/_meta/code_snapshot/navigated_fill.py

results/frozen_opt/quality_t3/_meta/code_snapshot/packing.py

results/frozen_opt/quality_t3/_meta/code_snapshot/raptor_baseline.py

results/frozen_opt/quality_t3/_meta/code_snapshot/run_benchmarks.py

results/frozen_opt/quality_t3/_meta/code_snapshot/segtree.py

results/frozen_opt/quality_t3/_meta/manifest.json

results/frozen_opt/quality_t3/quality_cells.json

results/frozen_opt/quality_t3/quality_ci.json

results/frozen_opt/quality_t3/quality_predictions.json

results/frozen_opt/quality_t3/quality_predictions_raw.json

results/frozen_opt/quality_t3/quality_results.csv

results/frozen_opt/quality_t3/quality_results.json

results/frozen_opt/summary_vs_leaf/narrativeqa.json

results/frozen_opt/summary_vs_leaf/narrativeqa_rescored.json

results/frozen_opt/summary_vs_leaf/qasper.json

results/frozen_opt/summary_vs_leaf/results.json

results/hybrid_flat_run/multihop/2wikimultihopqa/2wikimultihopqa_checkpoint.json

results/hybrid_flat_run/multihop/2wikimultihopqa/2wikimultihopqa_results.csv

results/hybrid_flat_run/multihop/2wikimultihopqa/2wikimultihopqa_results.json

results/hybrid_flat_run/multihop/2wikimultihopqa_checkpoint.json

results/hybrid_flat_run/multihop/2wikimultihopqa_results.csv

results/hybrid_flat_run/multihop/2wikimultihopqa_results.json

results/hybrid_flat_run/multihop/_pre_g_backup/2wikimultihopqa_checkpoint.json

results/hybrid_flat_run/multihop/_pre_g_backup/hotpotqa_checkpoint.json

results/hybrid_flat_run/multihop/_pre_g_backup/musique_checkpoint.json

results/hybrid_flat_run/multihop/hotpotqa/hotpotqa_checkpoint.json

results/hybrid_flat_run/multihop/hotpotqa/hotpotqa_results.csv

results/hybrid_flat_run/multihop/hotpotqa/hotpotqa_results.json

results/hybrid_flat_run/multihop/hotpotqa_checkpoint.json

results/hybrid_flat_run/multihop/hotpotqa_results.csv

results/hybrid_flat_run/multihop/hotpotqa_results.json

results/hybrid_flat_run/multihop/musique/musique_checkpoint.json

results/hybrid_flat_run/multihop/musique/musique_results.csv

results/hybrid_flat_run/multihop/musique/musique_results.json

results/hybrid_flat_run/multihop/musique_checkpoint.json

results/hybrid_flat_run/multihop/musique_results.json

results/hybrid_flat_run/multihop/narrativeqa_full/narrativeqa_full_checkpoint.json

results/hybrid_flat_run/multihop/narrativeqa_full/narrativeqa_full_results.csv

results/hybrid_flat_run/multihop/narrativeqa_full/narrativeqa_full_results.json

results/hybrid_flat_run/multihop/narrativeqa_full_checkpoint.json

results/hybrid_flat_run/multihop/narrativeqa_full_results.json

results/hybrid_flat_run/qasper/qasper_checkpoint.json

results/hybrid_flat_run/qasper/qasper_results.csv

results/hybrid_flat_run/qasper/qasper_results.json

results/hybrid_flat_run/quality/quality_checkpoint.json

results/hybrid_flat_run/quality/quality_results.csv

results/hybrid_flat_run/quality/quality_results.json

results/multihop/2wikimultihopqa/2wikimultihopqa_checkpoint.json

results/multihop/2wikimultihopqa/2wikimultihopqa_results.csv

results/multihop/2wikimultihopqa/2wikimultihopqa_results.json

results/multihop/hotpotqa/hotpotqa_checkpoint.json

results/multihop/hotpotqa/hotpotqa_results.csv

results/multihop/hotpotqa/hotpotqa_results.json

results/multihop/musique/musique_checkpoint.json

results/multihop/musique/musique_results.csv

results/multihop/musique/musique_results.json

results/multihop/narrativeqa_full/narrativeqa_full_checkpoint.json

results/multihop/narrativeqa_full/narrativeqa_full_results.csv

results/multihop/narrativeqa_full/narrativeqa_full_results.json

results/multihop_recall/2wikimultihopqa_recall.json

results/multihop_recall/hotpotqa_recall.json

results/multihop_recall/multihop_recall.json

results/multihop_recall/musique_recall.json

results/navtree_ce_qasper/qasper_checkpoint.json

results/navtree_ce_qasper/qasper_results.csv

results/navtree_ce_qasper/qasper_results.json

results/openweight_run/llama8b_local/qasper/qasper_checkpoint.json

results/openweight_run/llama8b_local/qasper/qasper_results.csv

results/openweight_run/llama8b_local/qasper/qasper_results.json

results/openweight_run/llama8b_local/quality/quality_checkpoint.json

results/openweight_run/llama8b_local/quality/quality_results.csv

results/openweight_run/llama8b_local/quality/quality_results.json

results/openweight_run/qwen7b_local/qasper/qasper_checkpoint.json

results/openweight_run/qwen7b_local/qasper/qasper_results.csv

results/openweight_run/qwen7b_local/qasper/qasper_results.json

results/openweight_run/qwen7b_local/quality/quality_checkpoint.json

results/openweight_run/qwen7b_local/quality/quality_results.csv

results/openweight_run/qwen7b_local/quality/quality_results.json

results/paired_tests/2wikimultihopqa_paired.json

results/paired_tests/hotpotqa_paired.json

results/paired_tests/musique_paired.json

results/paired_tests/qasper_paired.json

results/paired_tests/quality_paired.json

results/paired_tests/summary.json

results/paired_tests_openweight/qasper_paired.json

results/paired_tests_openweight/quality_paired.json

results/paired_tests_openweight/summary_llama8b_local.json

results/paired_tests_openweight/summary_qwen7b_local.json

results/qasper_recall/qasper_recall.csv

results/qasper_recall/qasper_recall.json

(441 additional files not shown) You must enabled JavaScript to view entire file list.

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

Text integral sub licența deschisă a publicației · 784 cuvinte

Subiecte conexe