Biomedical AI Specialist

Five kinds of data, five kinds of trouble

"Biomedical AI" is not one field. It is at least five, held together by the fact that mistakes cost patients. Each modality has its own structure, its own way of producing labels, and its own way of fooling a model into looking better than it is. Before you learn any model, learn the data.

Sequences

DNA is a string over four letters; a human genome is about three billion of them. Proteins are strings over twenty amino acids, typically a few hundred long, shipped as FASTA files from databases like UniProt. The difficulty is not the alphabet. It is that most sequence variation is neutral, that function labels are sparse and often inherited by similarity from an already-annotated relative (so a model that learns "similar sequence, same label" is partly learning the annotation pipeline), and that evolution makes samples non-independent. Two proteins that are 90% identical are not two data points. If one is in the training set and the other in the test set, you have measured memorisation. Serious benchmarks cluster sequences by identity and split by cluster.

Structures

The Protein Data Bank holds a few hundred thousand experimentally solved 3D structures, most from X-ray crystallography and increasingly from cryo-EM. A structure is a snapshot: one conformation, at one resolution, often with flexible regions missing because they did not resolve. The PDB is also biased toward proteins that crystallise well, which is not a random sample of biology. The PDB-101 guide in this module's materials explains what resolution, B-factors and missing residues mean; those fields decide whether a structure is a trustworthy label.

Images

CT volumes record tissue density in Hounsfield units, a physical scale that is comparable across scanners. MRI intensities are not; they depend on the sequence, the machine and the site. Pathology slides are gigapixel images annotated at the level of a whole slide, not a pixel. Expert annotation is expensive, and experts disagree with each other more than newcomers expect. Images also carry a hospital's fingerprint — scanner model, positioning protocol, burned-in text — which a model will happily learn instead of the disease. Module 4 returns to this.

Health records

Electronic health records (EHRs) combine coded tables (ICD-10 diagnoses, LOINC lab tests, RxNorm medications), free-text notes and time stamps. The codes exist for billing and were assigned after the fact; they are not ground truth about what the patient had. Sampling is irregular and informative: a lab test exists because someone was worried enough to order it, so "missing" carries meaning. Records are fragmented across institutions and are the most privacy-sensitive data you will touch. The MIMIC-IV documentation shows how a de-identified research database is structured; read the table descriptions, not just the schema.

Omics

RNA sequencing, proteomics and methylation arrays measure thousands of features per sample, while a study may have fifty samples. This "p much larger than n" regime guarantees that something will correlate with the outcome by chance. Worse, batch effects — which lab processed the samples, on which day — are frequently stronger than the biological signal, and they are often confounded with the label because cases and controls were processed separately.

The two failures that cut across everything

Label noise. Many large image datasets derive labels automatically from radiology reports with rule-based text extraction. Reports hedge ("cannot exclude"), and the extractor does not. Outcome labels taken from billing codes inherit coding practice. A model that reaches the ceiling of a noisy label has learned the noise.

Leakage. The same patient on both sides of a split; the same protein family; features recorded after the outcome was already known; a site that appears only in the training set with a different disease prevalence. Leakage is the single most common reason a published number does not survive contact with a new hospital.

A worked procedure: splitting a chest X-ray dataset honestly

1. Group every image by patient identifier. All of a patient's images go to one side.
2. Hash the pixel data and remove exact and near duplicates across sides.
3. Record the source site of each image. Hold out at least one entire site as an external test set.
4. Check that label prevalence and demographic mix in the held-out site are reported, not assumed equal.
5. Freeze the split in a file with a checksum before training anything. Never re-split after looking at results.

The same five steps, with "patient" replaced by "sequence cluster", "scaffold" or "batch", apply to every modality above.

What is still unsettled

How much published progress survives external validation is an open question; the review by Ching and colleagues is honest that deep learning's wins in biology are real but patchy. Whether sequence foundation models generalise to poorly sampled branches of the tree of life is actively debated. And there is no agreed standard for how much label noise is acceptable before a benchmark stops meaning anything.

What to remember

• Learn the data-generating process before the model: who produced the label, when, and why.
• Samples are rarely independent in biology; split by patient, family, scaffold or batch, not by row.
• Missingness in health records is a signal, not an absence.
• Leakage and label noise inflate every modality's numbers; design the split before you train.