From G1 to M: a comparative study of methods for identifying cell cycle phases
Accurate identification of cell cycle phases in single-cell RNA-sequencing (scRNA-seq) data is crucial for biomedical research. Many methods have been developed to tackle this challenge, employing diverse approaches to predict cell cycle phases. In this review article, we delve into the standard processes in identifying cell cycle phases within scRNA-seq data and present several representative methods for comparison. To rigorously assess the accuracy of these methods, we propose an error function and employ multiple benchmarking datasets encompassing human and mouse data. Our evaluation results reveal a key finding: the fit between the reference data and the dataset being analyzed profoundly impacts the effectiveness of cell cycle phase identification methods. Therefore, researchers must carefully consider the compatibility between the reference data and their dataset to achieve optimal results. Furthermore, we explore the potential benefits of incorporating benchmarking data with multiple known cell cycle phases into the analysis. Merging such data with the target dataset shows promise in enhancing prediction accuracy. By shedding light on the accuracy and performance of cell cycle phase prediction methods across diverse datasets, this review aims to motivate and guide future methodological advancements. Our findings offer valuable insights for researchers seeking to improve their understanding of cellular dynamics through scRNA-seq analysis, ultimately fostering the development of more robust and widely applicable cell cycle identification methods.
The cell cycle is one of the strongest sources of variation in a single-cell RNA-seq experiment, which makes it two problems at once. Sometimes it is the signal — you want to know which cells are dividing and where they are in the cycle. More often it is a confounder that has to be identified before it can be regressed out, and removing it badly distorts everything downstream.
About a dozen tools exist for this, and they disagree. We wanted to know how much, and why.
Making the comparison fair
Two obstacles come before any benchmark.
The methods do not agree on what the phases are. Seurat returns G1, S and G2M. Revelio returns G1S, S, G2, G2M and MG1. The benchmark datasets label differently again — one gives G1/S/G2, another G1/S/G2M. Scoring these against each other as plain categories is not well defined.
And the cell cycle is a circle, so accuracy is not categorical. Calling a G2 cell "G2M" is a small error; calling it "G1" is a large one. A confusion matrix treats both as simply wrong.
We handled both by placing the phases on a circle as angles and measuring the error as between the true and predicted angle. Phases that neighbour each other cost little; opposite points on the cycle cost the most. That gives one number comparable across methods with different phase vocabularies.
We ran four representative methods — Tricycle, Seurat, Revelio and CS1CC — against four datasets with experimentally established phase labels: human embryonic stem cells sorted by FUCCI, mouse ES cells by Quartz-Seq, mouse ES cells sorted on Hoechst staining, and human RPE-FUCCI cells. Mouse genes were mapped to human homologs with biomaRt where a method's marker set required it.
No method wins
The headline result is that no method is best across all four datasets, and the spread is not small.
CS1CC had the lowest error on human ES cells (0.27, against 0.485 for Seurat) — and the highest on mouse ES cells (0.596, against 0.268 for Tricycle). Tricycle was the reverse. Averaged over all four datasets the four methods land between 0.433 and 0.468, which is close enough to be meaningless as a ranking.
The averages hide the finding. What matters is which dataset.
Why the reference matters more than the method
Tricycle works by transfer learning: it projects new data into a reference embedding built from mouse cortical neurosphere data. It was consistently strong on the mouse datasets and noticeably weaker on human. The method did not change between those runs. The distance between its reference and the data being analyzed did.
This is the practical conclusion of the paper. For a transfer-learning method, performance is set by how well the reference matches your data. For a marker-gene method, it is set by how well the marker list matches your data. Either way, the question worth asking before you pick a tool is not which scored best in a benchmark — it is which one's assumptions fit the experiment in front of you.
The two families also fail differently. Seurat and Revelio assign every cell to whichever predetermined phase scores highest, so a dataset containing only S cells still gets spread across all phases. CS1CC clusters first and lets the number of clusters vary, which handles unusual phase compositions better and carries its own risk: a misassigned cluster takes every cell in it along.
Padding the data helps
If a method's accuracy depends on how well the data fits its assumptions, you can improve the fit from the data side.
We tested this by blending cells from the benchmark datasets, with known phases, into the target data at 20%, 40%, 60% and 80%. Error dropped for every method as the proportion rose, and most for the clustering-based approaches, which now had well-populated phases to cluster against.
That is a usable technique: merging in a small labelled reference alongside your own cells improves phase prediction without changing the tool.
The stress test also exposed the failure mode. Given only S-phase cells and no padding, CS1CC assigned the entire sample to MG1 and G2M — with nothing to contrast against, the clustering has no way to locate itself on the cycle. CS1CC's strength on G1 across all the datasets is the same effect running the other way: G1 simply has more well-characterised marker genes than the other phases.
What is still missing
G0 remains largely out of reach. Quiescent cells are not paused mid-cycle, and the marker-gene and pseudotime machinery built for the dividing phases does not transfer cleanly, so this review covers G1 through M.
The broader gap is a reference embedding, or a marker list, that holds across tissues and species. Until one exists, choosing a cell cycle method means matching it to the dataset, and the safest practice is to check more than one.
- 2024 / Human Genetics and Genomics Advances
Subset-based method for cross-tissue transcriptome-wide association studies improves power and interpretability
- 2024 / Oncogene
ERK hyperactivation serves as a unified mechanism of escape in intrinsic and acquired CDK4/6 inhibitor resistance in acral lentiginous melanoma