# Chapter 15 Doublet detection

## 15.1 Overview

In single-cell RNA sequencing (scRNA-seq) experiments, doublets are artifactual libraries generated from two cells. They typically arise due to errors in cell sorting or capture, especially in droplet-based protocols (Zheng et al. 2017) involving thousands of cells. Doublets are obviously undesirable when the aim is to characterize populations at the single-cell level. In particular, doublets can be mistaken for intermediate populations or transitory states that do not actually exist. Thus, it is desirable to identify and remove doublet libraries so that they do not compromise interpretation of the results.

Several experimental strategies are available for doublet removal. One approach exploits natural genetic variation when pooling cells from multiple donor individuals (Kang et al. 2018). Doublets can be identified as libraries with allele combinations that do not exist in any single donor. Another approach is to mark a subset of cells (e.g., all cells from one sample) with an antibody conjugated to a different oligonucleotide (Stoeckius, Zheng, et al. 2017). Upon pooling, libraries that are observed to have different oligonucleotides are considered to be doublets and removed. These approaches can be highly effective but rely on experimental information that may not be available.

A more general approach is to infer doublets from the expression profiles alone (Dahlin et al. 2018). In this workflow, we will describe two purely computational approaches for detecting doublets from scRNA-seq data. The main difference between these two methods is whether or not they need cluster information beforehand. We will demonstrate the use of these methods on 10X Genomics data from a droplet-based scRNA-seq study of the mouse mammary gland (Bach et al. 2017).

#--- loading ---#
library(scRNAseq)
sce.mam <- BachMammaryData(samples="G_1")

#--- gene-annotation ---#
library(scater)
rownames(sce.mam) <- uniquifyFeatureNames(
rowData(sce.mam)$Ensembl, rowData(sce.mam)$Symbol)

library(AnnotationHub)
ens.mm.v97 <- AnnotationHub()[["AH73905"]]
rowData(sce.mam)$SEQNAME <- mapIds(ens.mm.v97, keys=rowData(sce.mam)$Ensembl,
keytype="GENEID", column="SEQNAME")

#--- quality-control ---#
is.mito <- rowData(sce.mam)$SEQNAME == "MT" stats <- perCellQCMetrics(sce.mam, subsets=list(Mito=which(is.mito))) qc <- quickPerCellQC(stats, percent_subsets="subsets_Mito_percent") sce.mam <- sce.mam[,!qc$discard]

#--- normalization ---#
library(scran)
set.seed(101000110)
clusters <- quickCluster(sce.mam)
sce.mam <- computeSumFactors(sce.mam, clusters=clusters)
sce.mam <- logNormCounts(sce.mam)

#--- variance-modelling ---#
set.seed(00010101)
dec.mam <- modelGeneVarByPoisson(sce.mam)
top.mam <- getTopHVGs(dec.mam, prop=0.1)

#--- dimensionality-reduction ---#
library(BiocSingular)
set.seed(101010011)
sce.mam <- denoisePCA(sce.mam, technical=dec.mam, subset.row=top.mam)
sce.mam <- runTSNE(sce.mam, dimred="PCA")

#--- clustering ---#
snn.gr <- buildSNNGraph(sce.mam, use.dimred="PCA", k=25)
colLabels(sce.mam) <- factor(igraph::cluster_walktrap(snn.gr)$membership) sce.mam ## class: SingleCellExperiment ## dim: 27998 2772 ## metadata(0): ## assays(2): counts logcounts ## rownames(27998): Xkr4 Gm1992 ... Vmn2r122 CAAA01147332.1 ## rowData names(3): Ensembl Symbol SEQNAME ## colnames: NULL ## colData names(5): Barcode Sample Condition sizeFactor label ## reducedDimNames(2): PCA TSNE ## altExpNames(0): ## 15.2 Doublet detection with clusters The doubletCluster() function identifies clusters with expression profiles lying between two other clusters (Bach et al. 2017). We consider every possible triplet of clusters consisting of a query cluster and two putative “source” clusters. Under the null hypothesis that the query consists of doublets from the two sources, we then compute a number of statistics: • The number of genes (N) that are differentially expressed in the same direction in the query cluster compared to both of the source clusters. Such genes would be unique markers for the query cluster and provide evidence against the null hypothesis. Clusters with few unique genes are more likely to be composed of doublets. • The ratio of the median library size in each source to the median library size in the query (lib.size fields). Doublet libraries are generated from a larger initial pool of RNA compared to libraries for single cells, and thus the former should have larger library sizes. Library size ratios much greater than unity are inconsistent with a doublet identity for the query. • The proportion of cells in the query cluster should also be reasonable - typically less than 5% of all cells, depending on how many cells were loaded onto the 10X Genomics device. For each query cluster, the best pair of sources is identified based solely on the lowest N. (This means that any lib.size* above unity is not definitive evidence against a doublet identity for a query cluster, as there may be a pair of sources with slightly higher N but both lib.size* values below unity.) If more detail is necessary, the all.pairs field contains statistics on all possible source pairs for a given query cluster. # Like 'findMarkers', this function will automatically # retrieve cluster assignments from 'colLabels'. library(scran) dbl.out <- doubletCluster(sce.mam) dbl.out ## DataFrame with 10 rows and 9 columns ## source1 source2 N best p.value lib.size1 ## <character> <character> <integer> <character> <numeric> <numeric> ## 6 2 1 13 Pcbp2 1.28336e-03 0.811531 ## 2 10 3 109 Pigr 4.34790e-21 0.619865 ## 4 6 5 111 Cotl1 1.09709e-08 1.540751 ## 5 10 7 139 Gde1 9.30195e-12 1.125474 ## 10 8 5 191 Krt18 5.54539e-20 0.888432 ## 7 8 5 270 AF251705 3.29661e-24 0.856192 ## 9 8 5 295 Fabp4 2.21523e-32 0.655624 ## 8 10 9 388 Col1a1 6.82664e-32 1.125578 ## 1 8 6 513 Acta2 1.07294e-24 0.865449 ## 3 6 5 530 Sapcd2 6.08574e-16 0.872951 ## lib.size2 prop all.pairs ## <numeric> <numeric> <DataFrameList> ## 6 0.516399 0.03030303 2:1:13:...,4:1:14:...,3:1:28:...,... ## 2 1.411579 0.28823954 10:3:109:...,5:3:194:...,8:3:205:...,... ## 4 0.688651 0.16305916 6:5:111:...,10:6:146:...,6:2:209:...,... ## 5 1.167854 0.00865801 10:7:139:...,9:7:153:...,10:9:168:...,... ## 10 0.888514 0.00865801 8:5:191:...,9:5:224:...,6:5:229:...,... ## 7 0.856271 0.01875902 8:5:270:...,9:5:337:...,10:5:362:...,... ## 9 0.655685 0.01154401 8:5:295:...,10:8:323:...,5:1:338:...,... ## 8 1.525264 0.01406926 10:9:388:...,9:7:400:...,10:7:436:...,... ## 1 1.936489 0.19841270 8:6:513:...,9:6:609:...,6:5:854:...,... ## 3 0.390173 0.25829726 6:5:530:...,5:2:593:...,5:4:700:...,... Clusters are then ranked by N in the output of doubletClusters(), allowing us to prioritize high-risk clusters that require careful investigation. If a more concrete threshold is necessary, we can identify clusters that have unusually low N using an outlier-based approach. library(scater) chosen.doublet <- rownames(dbl.out)[isOutlier(dbl.out$N,
type="lower", log=TRUE)]
chosen.doublet
## [1] "6"

Examination of the output of doubletCluster() indicates that cluster 6 has the fewest unique genes and library sizes that are comparable to or greater than its sources. We see that every gene detected in this cluster is also expressed in either of the two proposed source clusters (Figure 15.1).

markers <- findMarkers(sce.mam, direction="up")
dbl.markers <- markers[[chosen.doublet]]

library(scater)
chosen <- rownames(dbl.markers)[dbl.markers$Top <= 10] plotHeatmap(sce.mam, order_columns_by="label", features=chosen, center=TRUE, symmetric=TRUE, zlim=c(-5, 5)) Closer examination of some known markers suggests that the offending cluster consists of doublets of basal cells (Acta2) and alveolar cells (Csn2) (Figure 15.2). Indeed, no cell type is known to strongly express both of these genes at the same time, which supports the hypothesis that this cluster consists solely of doublets rather than being an entirely novel cell type. plotExpression(sce.mam, features=c("Acta2", "Csn2"), x="label", colour_by="label") The strength of doubletCluster() lies in its simplicity and ease of interpretation. Suspect clusters can be quickly flagged based on the metrics returned by the function. However, it is obviously dependent on the quality of the clustering. Clusters that are too coarse will fail to separate doublets from other cells, while clusters that are too fine will complicate interpretation. The method is also somewhat biased towards clusters with fewer cells, where the reduction in power is more likely to result in a low N. (Fortunately, this is a desirable effect as doublets should be rare in a properly performed scRNA-seq experiment.) ## 15.3 Doublet detection by simulation The other doublet detection strategy involves in silico simulation of doublets from the single-cell expression profiles (Dahlin et al. 2018). This is performed using the doubletCells() function from scran, which will: 1. Simulate thousands of doublets by adding together two randomly chosen single-cell profiles. 2. For each original cell, compute the density of simulated doublets in the surrounding neighborhood. 3. For each original cell, compute the density of other observed cells in the neighborhood. 4. Return the ratio between the two densities as a “doublet score” for each cell. This approach assumes that the simulated doublets are good approximations for real doublets. The use of random selection accounts for the relative abundances of different subpopulations, which affect the likelihood of their involvement in doublets; and the calculation of a ratio avoids high scores for non-doublet cells in highly abundant subpopulations. We see the function in action below. To speed up the density calculations, doubletCells() will perform a PCA on the log-expression matrix, and we perform some (optional) parametrization to ensure that the computed PCs are consistent with that from our previous analysis on this dataset. library(BiocSingular) set.seed(100) # Setting up the parameters for consistency with denoisePCA(); # this can be changed depending on your feature selection scheme. dbl.dens <- doubletCells(sce.mam, subset.row=top.mam, d=ncol(reducedDim(sce.mam))) summary(dbl.dens) ## Min. 1st Qu. Median Mean 3rd Qu. Max. ## 0.00 7.63 21.04 395.42 49.30 39572.52 The highest doublet scores are concentrated in a single cluster of cells in the center of Figure 15.3. sce.mam$DoubletScore <- log10(dbl.dens+1)
plotTSNE(sce.mam, colour_by="DoubletScore")

From the clustering information, we see that the affected cells belong to the same cluster that was identified using doubletCluster() (Figure 15.4), which is reassuring.

plotColData(sce.mam, x="label", y="DoubletScore", colour_by="label")

The advantage of doubletCells() is that it does not depend on clusters, reducing the sensitivity of the results to clustering quality. The downside is that it requires some strong assumptions about how doublets form, such as the combining proportions and the sampling from pure subpopulations. In particular, doubletCells() treats the library size of each cell as an accurate proxy for its total RNA content. If this is not true, the simulation will not combine expression profiles from different cells in the correct proportions. This means that the simulated doublets will be systematically shifted away from the real doublets, resulting in doublet scores that are too low.

Simply removing cells with high doublet scores will not be sufficient to eliminate real doublets from the data set. In some cases, only a subset of the cells in the putative doublet cluster actually have high scores, and removing these would still leave enough cells in that cluster to mislead downstream analyses. In fact, even defining a threshold on the doublet score is difficult as the interpretation of the score is relative. There is no general definition for a fixed threshold above which libraries are to be considered doublets.

We recommend interpreting the doubletCells() scores in the context of cluster annotation. All cells from a cluster with a large average doublet score should be considered suspect, and close neighbors of problematic clusters should also be treated with caution. In contrast, a cluster containing a small proportion of high-scoring cells is probably safe provided that any interesting results are not being driven by those cells (e.g., checking that DE in an interesting gene is not driven solely by cells with high doublet scores). While clustering is still required, this approach is more robust than doubletClusters() to the quality of the clustering as the scores are computed on a per-cell basis.

(As an aside, the issue of unknown combining proportions can be solved completely if spike-in information is available, e.g., in plate-based protocols. This will provide an accurate estimate of the total RNA content of each cell. To this end, spike-in-based size factors from Section 7.4 can be supplied to the doubletCells() function via the size.factors.content= argument. This will use the spike-in size factors to scale the contribution of each cell to a doublet library.)

## 15.4 Doublet detection in multiplexed experiments

### 15.4.1 Background

For multiplexed samples (Kang et al. 2018; Stoeckius et al. 2018), we can identify doublet cells based on the cells that have multiple labels. The idea here is that cells from the same sample are labelled in a unique manner, either implicitly with genotype information or experimentally with hashing tag oligos (HTOs). Cells from all samples are then mixed together and the multiplexed pool is subjected to scRNA-seq, avoiding batch effects and simplifying the logistics of processing a large number of samples. Importantly, most per-cell libraries are expected to contain one label that can be used to assign that cell to its sample of origin. Cell libraries containing two labels are thus likely to be doublets of cells from different samples.

To demonstrate, we will use some data from the original cell hashing study (Stoeckius et al. 2018). Each sample’s cells were stained with an antibody against a ubiquitous surface protein, where the antibody was conjugated to a sample-specific HTO. Sequencing of the HTO-derived cDNA library ultimately yields a count matrix where each row corresponds to a HTO and each column corresponds to a cell barcode. (We use BiocFileCache to avoid repeated downloads of the same file.)

library(BiocFileCache)

fname <- "GSM2895283_Hashtag-HTO-count.csv.gz"
untar(hash.tar, files=fname, exdir=tempdir())
hto.counts <- as.matrix(hto.counts[1:8,])

dim(hto.counts)
## [1]     8 65000
hto.counts[,1:3]
##                     GGCGACTAGAGGACGG CATCAAGGTCTTGTCC AAACCTGAGTGATCGG
## BatchA-AGGACCATCCAA               30                4               12
## BatchB-ACATGTTACCGT               16               39               15
## BatchC-AGCTTACTATCC               26                0               19
## BatchD-TCGATAATGCGA             2698               22                2
## BatchE-GAGGCTGAGCTA                8               24               32
## BatchF-GTGTGACGTATT               15               47               12
## BatchG-ACTGTCTAACGG               55               21               76
## BatchH-TATCACATCGGT               32               17                3

### 15.4.2 Identifying inter-sample doublets

Before we proceed to doublet detection, we simplify the problem by first identifying the barcodes that contain cells. This is most conventionally done using the gene expression matrix for the same set of barcodes, as shown in Section 6.5. While we could extract the gene expression matrix from hash.tar, we will instead demonstrate an alternative approach that uses the HTO count matrix directly in the emptyDrops() function. This requires some adjustment of the lower= argument to obtain an appropriate value depending on the sequencing depth of the HTOs; otherwise, the barcode-rank plots are quite similar to what one might expect from gene expression data (Figure 15.5).

library(DropletUtils)

set.seed(101)
hash.calls <- emptyDrops(hto.counts, lower=200)
is.cell <- which(hash.calls$FDR <= 0.001) length(is.cell) ## [1] 21655 par(mfrow=c(1,2)) r <- rank(-hash.calls$Total)
plot(r, hash.calls$Total, log="xy", xlab="Rank", ylab="Total HTO count", main="") hist(log10(hash.calls$Total[is.cell]), xlab="Log[10] HTO count", main="")

We then run hashedDrops() on the subset of cell barcode libraries that actually contain cells. This returns the likely sample of origin for each barcode library based on its most abundant HTO, using abundances adjusted for ambient contamination in the ambient= argument. (The adjustment process itself involves a fair number of assumptions that we will not discuss here; see ?hashedDrops for more details.) For quality control, it returns the log-fold change between the first and second-most abundant HTOs in each barcode libary (Figure 15.6), allowing us to quantify the certainty of each assignment. Confidently assigned singlets are marked using the Confident field in the output.

hash.stats <- hashedDrops(hto.counts[,is.cell],
ambient=metadata(hash.calls)$ambient) hist(hash.stats$LogFC, xlab="Log fold-change from best to second HTO", main="")
# Raw assignments:
table(hash.stats$Best) ## ## 1 2 3 4 5 6 7 8 ## 2694 3272 2733 2782 2489 2377 2508 2800 # Confident assignments based on (i) a large log-fold change # and (ii) not being a doublet, see below. table(hash.stats$Best[hash.stats$Confident]) ## ## 1 2 3 4 5 6 7 8 ## 2344 2775 2440 2274 2089 1988 2101 2457 Of greater interest here is how we can use the hashing information to detect doublets. This is achieved by reporting the log-fold change between the count for the second HTO and the estimated contribution from ambient contamination. A large log-fold change indicates that the second HTO still has an above-expected abundance, consistent with a doublet containing HTOs from two samples. We use outlier detection to explicitly identify putative doublets as those barcode libraries that have large log-fold changes; this is visualized in Figure 15.7, which shows a clear separation between the putative singlets and doublets. summary(hash.stats$Doublet)
##    Mode   FALSE    TRUE
## logical   18620    3035
colors <- rep("grey", nrow(hash.stats))
colors[hash.stats$Doublet] <- "red" colors[hash.stats$Confident] <- "black"

plot(hash.stats$LogFC, hash.stats$LogFC2,
xlab="Log fold-change from best to second HTO",
ylab="Log fold-change of second HTO over ambient",
col=colors)

### 15.4.3 Guilt by association for unmarked doublets

One obvious limitation of this approach is that doublets of cells marked with the same HTO are not detected. In a simple multiplexing experiment involving $$N$$ samples with similar numbers of cells, we would expect around $$1/N$$ of all doublets to involve cells from the same sample. For typical values of $$N$$ of 5 to 12, this may still be enough to cause the formation of misleading doublet clusters even after the majority of known doublets are removed. To avoid this, we recover the remaining intra-sample doublets based on their similarity with known doublets in gene expression space (hence, “guilt by association”). We illustrate by loading the gene expression data for this study:

gname <- "GSM2895282_Hashtag-RNA.umi.txt.gz"
untar(hash.tar, files=gname, exdir=tempdir())

# Reading it in as a sparse matrix in a SingleCellExperiment.
library(scater)
sce.hash <- SingleCellExperiment(list(counts=gene.counts))

# Subsetting to all barcodes detected as cells. Requires an intersection,
# because hto.counts and gene.counts are not the same dimensions!
common <- intersect(colnames(sce.hash), rownames(hash.stats))
sce.hash <- sce.hash[,common]
colData(sce.hash) <- hash.stats[common,]

sce.hash
## class: SingleCellExperiment
## dim: 40899 20718
## assays(1): counts
## rownames(40899): 0610007N19Rik 0610007P14Rik ... snoU2-30 snoU83B
## rowData names(0):
## colnames(20718): ACTGCTCAGGTGTTAA ATGAGGGAGATGTTAG ... AACTGGTTCTTGGGTA
##   CTCGGAGTCTAACTCT
## colData names(7): Total Best ... Doublet Confident
## reducedDimNames(0):
## altExpNames(0):

For each cell, we calculate the proportion of its nearest neighbors that are known doublets. Intra-sample doublets should have high proportions under the assumption that their gene expression profiles are similar to inter-sample doublets involving the same combination of cell states/types. Unlike in Section 15.3, the use of experimentally derived doublet calls avoids any assumptions about the relative quantity of total RNA or the probability of doublet formation across different cell types.

# Performing a quick-and-dirty analysis to get some PCs to use
# for nearest neighbor detection inside doubletRecovery().
library(scran)
sce.hash <- logNormCounts(sce.hash)
dec.hash <- modelGeneVar(sce.hash)
top.hash <- getTopHVGs(dec.hash, n=1000)
set.seed(1011110)
sce.hash <- runPCA(sce.hash, subset_row=top.hash, ncomponents=20)

# Recovering the intra-sample doublets:
hashed.doublets <- doubletRecovery(sce.hash, use.dimred="PCA",
doublets=sce.hash$Doublet, samples=table(sce.hash$Best))
hashed.doublets
## DataFrame with 20718 rows and 3 columns
##       proportion     known predicted
##        <numeric> <logical> <logical>
## 1           0.12      TRUE     FALSE
## 2           0.02     FALSE     FALSE
## 3           0.14     FALSE     FALSE
## 4           0.08     FALSE     FALSE
## 5           0.18     FALSE     FALSE
## ...          ...       ...       ...
## 20714       0.10     FALSE     FALSE
## 20715       0.02     FALSE     FALSE
## 20716       0.02     FALSE     FALSE
## 20717       0.00     FALSE     FALSE
## 20718       0.10     FALSE     FALSE

The doubletRecovery() function also returns explicit intra-sample doublet predictions based on the doublet neighbor proportions. Given the distribution of cells across multiplexed samples in samples=, we estimate the fraction of doublets that would not be observed from the HTO counts. This is converted into an absolute number based on the number of observed doublets; the top set of libraries with the highest proportions are then marked as intra-sample doublets (Figure 15.8)

set.seed(1000101001)
sce.hash <- runTSNE(sce.hash, dimred="PCA")
sce.hash$proportion <- hashed.doublets$proportion
sce.hash$predicted <- hashed.doublets$predicted

gridExtra::grid.arrange(
plotTSNE(sce.hash, colour_by="proportion") + ggtitle("Doublet proportions"),
plotTSNE(sce.hash, colour_by="Doublet") + ggtitle("Known doublets"),
ggcells(sce.hash) +
geom_point(aes(x=TSNE.1, y=TSNE.2), color="grey") +
geom_point(aes(x=TSNE.1, y=TSNE.2), color="red",
data=function(x) x[x$predicted,]) + ggtitle("Predicted intra-sample doublets"), ncol=2 ) As an aside, it is worth noting that even known doublets may not necessarily have high doublet neighbor proportions. This is typically observed for doublets involving cells of the same type or state, which are effectively intermixed in gene expression space with the corresponding singlets. The latter are much more abundant in most (well-controlled) experiments, which results in low proportions for the doublets involved (Figure 15.9). This effect can generally be ignored given the mostly harmless nature of these doublets. state <- ifelse(hashed.doublets$predicted, "predicted",
ifelse(hashed.doublets\$known, "known", "singlet"))
ggplot(as.data.frame(hashed.doublets)) +
geom_violin(aes(x=state, y=proportion)) 

Doublet detection procedures should only be applied to libraries generated in the same experimental batch. It is obviously impossible for doublets to form between two cells that were captured separately. Thus, some understanding of the experimental design is required prior to the use of the above functions. This avoids unnecessary concerns about the validity of batch-specific clusters that cannot possibly consist of doublets.

It is also difficult to interpret doublet predictions in data containing cellular trajectories. By definition, cells in the middle of a trajectory are always intermediate between other cells and are liable to be incorrectly detected as doublets. Some protection is provided by the non-linear nature of many real trajectories, which reduces the risk of simulated doublets coinciding with real cells in doubletCells(). One can also put more weight on the relative library sizes in doubletCluster() instead of relying solely on N, under the assumption that sudden spikes in RNA content are unlikely in a continuous biological process.

The best solution to the doublet problem is experimental - that is, to avoid generating them in the first place. This should be a consideration when designing scRNA-seq experiments, where the desire to obtain large numbers of cells at minimum cost should be weighed against the general deterioration in data quality and reliability when doublets become more frequent.

## Session Info

R version 4.0.0 Patched (2020-05-01 r78341)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 18.04.4 LTS

Matrix products: default
BLAS:   /home/luna/Software/R/R-4-0-branch/lib/libRblas.so
LAPACK: /home/luna/Software/R/R-4-0-branch/lib/libRlapack.so

locale:
[1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C
[3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8
[5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8
[7] LC_PAPER=en_US.UTF-8       LC_NAME=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C

attached base packages:
[1] stats4    parallel  stats     graphics  grDevices utils     datasets
[8] methods   base

other attached packages:
[1] DropletUtils_1.8.0          BiocFileCache_1.12.0
[3] dbplyr_1.4.3                BiocSingular_1.4.0
[5] scater_1.16.0               ggplot2_3.3.0
[7] scran_1.16.0                SingleCellExperiment_1.10.1
[9] SummarizedExperiment_1.18.1 DelayedArray_0.14.0
[11] matrixStats_0.56.0          Biobase_2.48.0
[13] GenomicRanges_1.40.0        GenomeInfoDb_1.24.0
[15] IRanges_2.22.1              S4Vectors_0.26.0
[17] BiocGenerics_0.34.0         OSCAUtils_0.0.2
[19] BiocStyle_2.16.0

loaded via a namespace (and not attached):
[1] bitops_1.0-6              bit64_0.9-7
[3] RColorBrewer_1.1-2        httr_1.4.1
[5] tools_4.0.0               R6_2.4.1
[7] irlba_2.3.3               HDF5Array_1.16.0
[9] vipor_0.4.5               DBI_1.1.0
[11] colorspace_1.4-1          withr_2.2.0
[13] tidyselect_1.1.0          gridExtra_2.3
[15] processx_3.4.2            bit_1.1-15.2
[17] curl_4.3                  compiler_4.0.0
[19] BiocNeighbors_1.6.0       labeling_0.3
[21] bookdown_0.19             scales_1.1.1
[23] callr_3.4.3               rappdirs_0.3.1
[25] stringr_1.4.0             digest_0.6.25
[27] R.utils_2.9.2             rmarkdown_2.1
[29] XVector_0.28.0            pkgconfig_2.0.3
[31] htmltools_0.4.0           highr_0.8
[33] limma_3.44.1              rlang_0.4.6
[35] RSQLite_2.2.0             DelayedMatrixStats_1.10.0
[37] farver_2.0.3              BiocParallel_1.22.0
[39] R.oo_1.23.0               dplyr_0.8.5
[41] RCurl_1.98-1.2            magrittr_1.5
[43] GenomeInfoDbData_1.2.3    Matrix_1.2-18
[45] Rhdf5lib_1.10.0           Rcpp_1.0.4.6
[47] ggbeeswarm_0.6.0          munsell_0.5.0
[49] viridis_0.5.1             R.methodsS3_1.8.0
[51] lifecycle_0.2.0           stringi_1.4.6
[53] yaml_2.2.1                edgeR_3.30.0
[55] zlibbioc_1.34.0           Rtsne_0.15
[57] rhdf5_2.32.0              grid_4.0.0
[59] blob_1.2.1                dqrng_0.2.1
[61] crayon_1.3.4              lattice_0.20-41
[63] cowplot_1.0.0             locfit_1.5-9.4
[65] knitr_1.28                ps_1.3.3
[67] pillar_1.4.4              igraph_1.2.5
[69] codetools_0.2-16          glue_1.4.1
[71] evaluate_0.14             BiocManager_1.30.10
[73] vctrs_0.3.0               gtable_0.3.0
[75] purrr_0.3.4               assertthat_0.2.1
[77] xfun_0.13                 rsvd_1.0.3
[79] viridisLite_0.3.0         pheatmap_1.0.12
[81] tibble_3.0.1              beeswarm_0.2.3
[83] memoise_1.1.0             statmod_1.4.34
[85] ellipsis_0.3.1           

### Bibliography

Bach, K., S. Pensa, M. Grzelak, J. Hadfield, D. J. Adams, J. C. Marioni, and W. T. Khaled. 2017. “Differentiation dynamics of mammary epithelial cells revealed by single-cell RNA sequencing.” Nat Commun 8 (1): 2128.

Dahlin, J. S., F. K. Hamey, B. Pijuan-Sala, M. Shepherd, W. W. Y. Lau, S. Nestorowa, C. Weinreb, et al. 2018. “A single-cell hematopoietic landscape resolves 8 lineage trajectories and defects in Kit mutant mice.” Blood 131 (21): e1–e11.

Kang, H. M., M. Subramaniam, S. Targ, M. Nguyen, L. Maliskova, E. McCarthy, E. Wan, et al. 2018. “Multiplexed droplet single-cell RNA-sequencing using natural genetic variation.” Nat. Biotechnol. 36 (1): 89–94.

Stoeckius, M., S. Zheng, B. Houck-Loomis, S. Hao, B. Z. Yeung, W. M. Mauck, P. Smibert, and R. Satija. 2018. “Cell Hashing with barcoded antibodies enables multiplexing and doublet detection for single cell genomics.” Genome Biol. 19 (1): 224.

Stoeckius, M., S. Zheng, B. Houck-Loomis, S. Hao, B. Yeung, P. Smibert, and R. Satija. 2017. “Cell ‘Hashing’ with Barcoded Antibodies Enables Multiplexing and Doublet Detection for Single Cell Genomics.” bioRxiv. Cold Spring Harbor Laboratory. doi:10.1101/237693.

Zheng, G. X., J. M. Terry, P. Belgrader, P. Ryvkin, Z. W. Bent, R. Wilson, S. B. Ziraldo, et al. 2017. “Massively parallel digital transcriptional profiling of single cells.” Nat Commun 8 (January): 14049.