New benchmark · The Hebrew University of Jerusalem
SABLE
A leakage-controlled structural antibody benchmark for reproducible deep-learning evaluation
· 4 claims checked against 1 source
What it is
SABLE (Structural Antibody Benchmark for deep-Learning Evaluation) is a versioned dataset and evaluation suite for antibody structure prediction and antibody-antigen interaction models, released by researchers at the Hebrew University of Jerusalem. It combines a fixed collection of experimental training structures with a held-out test set built specifically to prevent train-test leakage, a problem that has made many antibody deep learning papers difficult to compare directly.
How it works
Candidate test structures were selected after AlphaFold3's temporal cutoff, then filtered against the training set using antibody and antigen sequence similarity filters. This catches near-duplicate leakage that a date cutoff alone would miss. Each test entry also records its nearest training-set neighbour by sequence similarity, letting users quantify remaining relatedness and stratify performance by similarity rather than assume independence between train and test.
What is new
SABLE pairs 16,511 experimental training entries with 327 manually reviewed test structures. It extends coverage with 3,274 high-confidence, patent-derived AlphaFold3 models spanning 476 antigens. Structures ship with CDR annotations, redundancy labels and model-confidence fields in a versioned release format, alongside a Python/PyTorch API and standardised metrics for reproducible access and evaluation without additional structure processing.
Why it matters
Antibody design and structure-prediction papers often report incompatible numbers because their test sets overlap training data to varying, undisclosed degrees. SABLE gives the field a shared, leakage-audited yardstick, letting different methods be compared against the same controlled test set and letting results be stratified by how similar a test case is to anything the model may have already seen.
Availability
SABLE is described in a preprint posted on bioRxiv. The benchmark is distributed as versioned releases with accompanying processing and evaluation code.
The evidence
Each statement above was written with the sentence from the source that supports it, and a program (not a model) checked that every quote and every figure appears in the pages it read. Here they are.
- 01
SABLE combines 16,511 experimental training entries with 327 manually reviewed test entries and 3,274 high-confidence AlphaFold3 models spanning 476 antigens
“16,511 experimental training entries with 327 manually reviewed test entries and 3,274 high-confidence, patent-derived AlphaFold3 models spanning 476 antigens” S1
- 02
Candidate test structures were selected after the AlphaFold3 temporal cutoff and filtered against the training set using sequence similarity filters
“Candidate test structures were selected after the AlphaFold3 temporal cutoff and filtered against the training set using antibody and antigen sequence similarity filters” S1
- 03
Each test entry records its nearest training set neighbour to allow stratification by similarity
“Each test entry records its nearest training set neighbour, allowing users to quantify remaining relatedness and stratify performance by similarity” S1
- 04
SABLE provides a Python/PyTorch API and standardised benchmark metrics for reproducible evaluation without extra structure processing
“A Python/PyTorch API and standardised benchmark metrics provide reproducible database access and evaluation code without requiring additional antibody-structure processing” S1
