iDEG algorithm
For negative binomial RNA-Seq data:
-
Step 0 Normalize the data (for unequal library sizes only).
-
Step 1. Group genes into windows based on their gene expression levels as in (\ref{eq:index-gene-window}).
-
Step 2. Compute \(\hat{\bar{\mu}}_{w}\) and \(\hat{\bar{\sigma}}^{2}_{w}\) for each window \(w\), and obtain a “raw” estimate of \(\delta_{g}\).
-
Step 3. Obtain a “refined” estimate of \(\delta_{g}\) by fitting a smoothing spline.
-
(Step 3’. Alternatively, if a constant dispersion is more appropriate, fit a linear regression model (\ref{eq:ols-disp}) to estimate the dispersion value \(\hat{\delta}_{0}\).)
-
Step 4. Apply the VST \(h_{nb}\) (\ref{eq:NBvst}) to each gene expression count.
-
Step 5. Compute the standardized summary statistics \(Z_{g}\) for each gene.
-
Step 6. Estimate the local false discovery rate locfdr for each gene.