Xenium data analysis: segmentation, QC and the pitfalls that create false biology
How to analyze 10x Xenium data: transcript QC, re-segmentation with Baysor or Proseg, panel-aware annotation, neighborhoods and niches.
Key takeaways
- Segmentation errors assign neighbors' transcripts to the wrong cell, creating false co-expression and false ligand-receptor pairs.
- Measure segmentation quality directly: co-expression of mutually exclusive markers is a simple, powerful metric.
- Filter low-quality transcripts and report background from negative-control probes and codewords.
- Annotate with panel-aware references; a panel cannot detect cell types it has no markers for.
Xenium measures individual transcripts in place, at subcellular resolution, for a targeted panel of genes. The data is precise and attractive, and it contains a trap. Every transcript must be assigned to a cell, and when that assignment is wrong, the analysis inherits biology that does not exist.
Step 1: Understand the output bundle
The Xenium output includes transcript coordinates with quality scores, cell and nucleus boundaries, a cell-by-gene matrix, and morphology images (DAPI and, with multimodal segmentation, boundary and interior stains). Start from the transcripts, not only the matrix, so segmentation and quality can be re-evaluated.
Step 2: Transcript-level QC
- Quality scores: filter transcripts below the Q-score threshold (Xenium's default is Q20) and check the score distribution across the slide.
- Negative controls: negative-control probes and unused codewords estimate the false-detection rate. Report it per region.
- Detection efficiency: compare per-gene detection with expected expression from scRNA-seq. Very low detection suggests probe or tissue problems.
- Regional artifacts: check for damaged areas, edges, detachment and out-of-focus regions.
Step 3: Evaluate segmentation
The default segmentation is built from nuclei, either by expanding nuclear boundaries by a fixed distance or by using multimodal membrane and interior stains. Nuclear expansion is a reasonable default for sparse tissue. In dense tissue (lymphoid organs, tumors, brain gray matter), expanded boundaries overlap neighbors, and transcripts are assigned to the wrong cells.
We quantify this directly:
- Choose pairs of mutually exclusive markers from the panel, such as CD3E with LYZ, or EPCAM with PTPRC.
- Compute the fraction of cells co-expressing both above background.
- Map where that happens. Leakage concentrates at cell-cell contacts and in dense regions.
What our experts avoid: reporting autocrine signaling, "dual-identity" cells or contact-dependent expression without measuring segmentation leakage first. In one case study, 14% of T cells co-expressed a myeloid marker before re-segmentation and under 3% after. The autocrine signal was an artifact.
Step 4: Re-segment when needed
- Baysor and Proseg infer boundaries from the spatial distribution of transcripts, using nuclei as priors.
- Cellpose with membrane or boundary stains improves boundaries when good membrane signal exists.
- Xenium Ranger resegment lets you change the expansion distance or switch segmentation inputs.
We compare segmentations with the exclusive-marker metric, cell-size distributions and visual checks, then choose the one that minimizes leakage without fragmenting cells.
Step 5: Annotation on a targeted panel
A 300 to 500 gene panel is designed around specific biology. Clustering on it tends to over-split the cell types the panel was designed for and merge everything else.
- Use reference-based annotation from matched or atlas scRNA-seq, restricted to panel genes.
- Validate labels with marker expression and with morphology.
- Report what the panel cannot resolve. A cell type missing from the results may be missing from the panel.
- With Xenium Prime 5K, unsupervised clustering becomes more reliable, though reference mapping still helps.
Step 6: Spatial organization
- Cellular neighborhoods and niches: cluster cells by the composition of their spatial neighbors.
- Neighborhood enrichment and co-localization: test whether cell types sit together more than chance, using permutation within each sample.
- Distance analyses: expression or abundance as a function of distance to structures.
- Subcellular localization: nuclear versus cytoplasmic distribution of transcripts, available because transcripts are individually located.
Step 7: Compare conditions
As with any spatial data, independent donors or animals are the replicates. Summarize cell-type abundance, niche composition and pseudobulk expression per sample, then test across samples. Many cells from one section do not make a cohort.
Panel design tips
If you are designing a custom panel, choose genes that separate your key cell states in scRNA-seq data. Include robust lineage markers for every expected cell type, so absence means absence. Avoid extremely highly expressed genes that crowd optical detection. We help design panels from your single-cell data.
How OmicsQuest can help
Our Xenium analysis service measures segmentation quality first, re-segments when needed, and only then annotates and compares. Choosing a platform? See Visium HD vs Xenium vs CosMx.
Frequently asked questions
How do I know if my Xenium segmentation is good?
Check co-expression of mutually exclusive markers, for example a T-cell marker with a myeloid or epithelial marker in the same cell. High rates, especially in dense regions, indicate transcripts leaking between neighbors. Also inspect transcript positions over the morphology image in Xenium Explorer, and check that cell sizes are plausible.
What is Proseg?
Proseg is a probabilistic cell segmentation method for imaging-based spatial transcriptomics. It infers cell boundaries from transcript positions, starting from a nuclear segmentation, and often reduces transcript misassignment compared with simple nuclear expansion. Baysor is a related, widely used alternative.
Work with OmicsQuest. Related services:
References
- Janesick A, Shelansky R, Gottscho AD, et al. High resolution mapping of the tumor microenvironment using integrated single-cell, spatial and in situ analysis. Nat Commun. 2023;14:8353.
- Petukhov V, Xu RJ, Soldatov RA, et al. Cell segmentation in imaging-based spatial transcriptomics. Nat Biotechnol. 2022;40:345-354.
- Stringer C, Wang T, Michaelos M, Pachitariu M. Cellpose: a generalist algorithm for cellular segmentation. Nat Methods. 2021;18:100-106.
- Palla G, Spitzer H, Klein M, et al. Squidpy: a scalable framework for spatial omics analysis. Nat Methods. 2022;19:171-178.
- Moses L, Pachter L. Museum of spatial transcriptomics. Nat Methods. 2022;19:534-546.