aiqbench analysis
This research advances self-supervised learning for patent documents by leveraging internal structural relationships between sections like claims and abstracts to create effective positive pairs. By eliminating reliance on external labels or citations, the proposed method improves retrieval performance across diverse benchmarks without requiring fine-tuning for specific tasks. The approach demonstrates that exploiting document architecture is a viable strategy for building robust representations in specialized legal-technical domains.
Patent Representation Learning via Self-supervision
arXiv:2511.10657v3 Announce Type: replace-cross Abstract: We study self-supervised patent representation learning with contrastive objectives. A standard baseline constructs positives by encoding the same text twice under independent dropout masks, but applying this recipe to long, structured patent documents requires careful calibration. We show that dropout-only training can be substantially strengthened by tuning temperature and dropout rate, yet its best configuration is evaluation-dependent and does not transfer uniformly from title--abstract retrieval to claim-to-disclosure retrieval. We propose mixed dropout--section positives, a patent-specific view construction strategy in which each positive pair links the title--abstract view of a patent either to a dropout re-encoding of itself or to another section of the same patent, such as claims, summary, background, drawings, or description. This uses patent-internal structure as a training-time signal without IPC labels, citations, or relevance annotations. We evaluate on graded EPO search-report retrieval, DAPFAM, a recently proposed family-level patent retrieval benchmark, and IPC subclass classification. Section-based positives improve over calibrated dropout-only and generic title--abstract augmentation baselines, are competitive with citation-informed patent encoders and a general-purpose embedding model, and perform strongly on the out-of-domain split of DAPFAM. Additional cross-section alignment diagnostics show that section-pair training improves compatibility among abstracts, claims, and descriptions of the same invention. These results indicate that patent sections provide effective self-supervised positive views for learning dense patent representations.
Abstract:We study self-supervised patent representation learning with contrastive objectives. A standard baseline constructs positives by encoding the same text twice under independent dropout masks, but applying this recipe to long, structured patent documents requires careful calibration. We show that dropout-only training can be substantially strengthened by tuning temperature and dropout rate, yet its best configuration is evaluation-dependent and does not transfer uniformly from title--abstract retrieval to claim-to-disclosure retrieval. We propose mixed dropout--section positives, a patent-specific view construction strategy in which each positive pair links the title--abstract view of a patent either to a dropout re-encoding of itself or to another section of the same patent, such as claims, summary, background, drawings, or description. This uses patent-internal structure as a training-time signal without IPC labels, citations, or relevance annotations. We evaluate on graded EPO search-report retrieval, DAPFAM, a recently proposed family-level patent retrieval benchmark, and IPC subclass classification. Section-based positives improve over calibrated dropout-only and generic title--abstract augmentation baselines, are competitive with citation-informed patent encoders and a general-purpose embedding model, and perform strongly on the out-of-domain split of DAPFAM. Additional cross-section alignment diagnostics show that section-pair training improves compatibility among abstracts, claims, and descriptions of the same invention. These results indicate that patent sections provide effective self-supervised positive views for learning dense patent representations.
Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
arXiv:2511.10657 [cs.CL]
(or arXiv:2511.10657v3 [cs.CL] for this version)
ICTIR 2026 - International ACM SIGIR Conference on Innovative Concepts and Theories in Information Retrieval, Jul 2026, Melbourne, Australia. pp.436-445
Full text under the publisher's open licence · 251 words
Related coverage
Pointer-Augmented Autoregressive Generation of Patent Claims with Joint Topology and Content Decoding
arXiv · Jul 28, 2026
Amazon is investing in the Lean Focused Research Organization
Amazon Science · Jul 26, 2026
From Pixels, Without Pre-training: Joint Generative and Self-Supervised Representation Learning in One Model
arXiv · Oct 10, 2026
Momentum-Guided Semantic Forecasting (MoFore) for Self-Supervised Video Representation Learning
arXiv · Jun 16, 2026