Results evaluation

Evaluation of the results is performed following the schema reported here.


Participants

Team name Institution(s) Contact member Dataset(s) analyzed
GSA Dept. of Computer Science - Univ. Helsinki Niko Välimäki 1
BoBiocomp CIRI Health Science and technologies - Univ. Bologna; Dept. of Biology - Univ. Bologna Rita Casadio 4
GBT INSERM UMR_S 910 - Aix Marseille University David Salgado 2


Evaluation metrics

Let R be the reference genome and let C be the genome representing contamination. A read sampled from mutated copy of R is called R read. A read sampled from C is called C read.

For mapping accuracy TP means a correctly mapped A read, FP means an incorrectly mapped A read or a mapped C read, TN means a C read that was not mapped, and FN means an R read that was not mapped.

For variant calling accuracy TP means predicted variant that is true variant, FP means a predicted variant that is not true variant, and FN means a true variant that was not predicted.


Human datasets

Here the reference C is human genome chromosome and contamination C is a mouse genome chromosome.


Artificial Chromosome 20, frequent variations

Mapping accuracy
Team TP FP TN FN
GBT 9734053 233964 9950604 81379
BoBiocomp 9734957 240847 9997905 26291
Variation calling accuracy
Team TP FP FN
GBT 66190 323 29714
BoBiocomp 86589 2288 9267


Artificial Chromosome 2

Mapping accuracy
Team TP FP TN FN
GBT 39167946 2012773 38819272 9
BoBiocomp 38977087 883908 39995667 143338


Artificial Chromosome 20, long deletions

Variation calling accuracy
Team TP FP FN
GSA 30847 1118 211144


Bacterial and Yeast datasets

For variant calling accuracy, here we distinguish between FP variants that were co-located with some sequencing errors in some reads covering them from the remaining FP variants that were not. FP-error counts the FP co-located with an error, while FP-simple counts the remaining FP.


Bacterial dataset

Mapping accuracy
Team TP FP FN
BoBiocomp 615732 365 87741
Variation calling accuracy
Team TP FP-simple FP-error Total FP FN
BoBiocomp 1229 25 38 63 1192


Yeast dataset

Mapping accuracy
Team TP FP FN
BoBiocomp 5538681 3668 240622
Variation calling accuracy
Team TP FP-simple FP-error Total FP FN
BoBiocomp 11093 270 266 536 8919