Can we Retrieve Everything All at Once?
ARM: An Alignment-Oriented LLM-based Retrieval Method

Peter Baile Chen1, Yi Zhang2, Michael Cafarella1, Dan Roth3
1MIT, 2AWS AI, 3University of Pennsylvania
Paper Code

Real-world open-domain questions can be complex, especially when answering them requires integrating information from multiple sources. Effectively identifying the necessary information involves aligning it with the available data and its organization.
To address the alignment problem, we introduce the alignment-oriented retrieval method — ARM, designed to better align questions with the organization of the data collection. Instead of solely matching query utterance, ARM explores relationships among data objects, enabling a retrieve-all-at-once solution for complex queries.

An optimal retrieval system performs both information and structure alignment

  • Information alignment: In the example above, both Document A and Table D address concepts in the user query and should therefore be retrieved. However, since A and D do not contain overlapping information, they cannot be connected together to form a complete answer to the query.
  • Structure alignment: Tables B and C, although not directly referenced in the user question, are necessary to bridge the connection between A and D, and therefore should also be retrieved.

What are existing solutions?

  • Standard RAG with query decomposition: While these methods excel at aligning information, they fall short in capturing structural alignment, as the bridging objects needed may not be explicitly referenced in the user query.
  • React-style agentic systems: These methods iteratively explore semantically relevant information but do not explicitly take into account the available data objects and how they are organized. Additionally, they often lack convergence guarantees, which results in significant inefficiency.

What is our solution?

  • We propose alignment-oriented method that enables an efficient and comprehensive retrieval for complex questions by aligning the question and its decomposition with existing data objects and their organization.
  • Specifically, we evaluate not only the semantic similarity between the query and each object but also the compatibility between objects themselves. Using these scores, we generate multiple alignment drafts that account for both informational and structural alignment, and ultimately select only the objects from these drafts that the model is most confident in.
  • Although this paper focuses on tables and documents, our framework can be readily extended to other modalities (e.g., images), provided that a compatibility score can be computed.

How does ARM perform?

  • We evaluated ARM on three complex retrieval tasks involving both passages and tables, and found that it substantially outperforms traditional RAG systems—with or without query decomposition—as well as ReAct-style agent-based frameworks. ARM delivers superior retrieval and downstream performance while also significantly improving efficiency and reducing monetary costs.

BibTeX

@article{chen2025can,
  title={Can we Retrieve Everything All at Once? ARM: An Alignment-Oriented LLM-based Retrieval Method},
  author={Chen, Peter Baile and Zhang, Yi and Cafarella, Michael and Roth, Dan},
  journal={arXiv preprint arXiv:2501.18539},
  year={2025}
}