The simulated data are generated by MetaSim(Richter 2008). The genomes are fetched from Mational Center for Biotechnology Information (NCBI) database ( and the NCBI taxonomy is downloaded from . We generate 3 datasets with various number of species and abundance.

Basic information about the 3 datasets
Dataset Number of species Number of reads Abundance
Dataset 1 4 50000  26x
Dataset 2 9 50000  12x
Dataset 3 19 200000  25x

We compare the performance of DirichletCluster with MarkovBin and MetaCluster, which are the best among all the unsupervised binning tools for NGS short reads. For all the 3 methods, we set the input type to paired-end. MarkovBin needs the number of species as input so we set the true value according to each dataset.

The performances of these binning methods are evaluated on precision, sensitivity and number of discovered species as (Wang 2012) does. We also apply adjusted Rand Index as the suggestion from (citation not found: Nguyen_2013) .

As we can see, MetaCluster always gets very high sensitivity but the overall performance is limited by number of discovered species. Abundance means a lot to it since MetaCluster gets better result in Dataset 1 and 3 (higher abundance, ~25x) than Dataset 2 (lower abundance, ~12x). For MarkovBin, however, the result becomes unacceptable with the number of species increasing.

DirichletCluster discovers most species in all the three datasets. Lower abundance does have some effect but precision and sensitivity are both acceptable in Dataset 2. Sensitivity of DirichletCluster can be compared to MetaCluster when the abundance is low or the number of species is small. When number of species is large, though the sensitivity is not as good as MetaCluster, the overall performance is much better.

Result of dataset 1
DirichletCluster MarkovBin MetaCluster
Precision 0.9795 0.7424 0.6473
Sensitivity 0.9795 0.7286 0.9808
NDS 4/4 3/4 2/4
ARI 0.9437 0.5332 0.4298
Result of dataset 2
DirichletCluster MarkovBin MetaCluster
Precision 0.7721 0.5702 0.5061
Sensitivity 0.8474 0.5186 0.8488
NDS 7/9 5/9 3/9
ARI 0.6286 0.2926 0.3786
Result of dataset 3
DirichletCluster MarkovBin MetaCluster
Precision 0.8877 0.4317 0.5945
Sensitivity 0.8914 0.4148 0.9923
NDS 17/19 2/19 8/19
ARI 0.7988 0.2094 0.4598