Mutation T@ster

Comparison with similar tools

We compared MutationTaster with similar prediction tools and obtained the following results:

programme	total	tp	tn	fp	fn	nd	with prediction	true positives	true negatives	false positives	false negatives	accuracy (this tool)	accuracy (common)	accuracy (all cases)
MutationTaster	2000	859	855	145	141		100.0%	85.6%	85.8%	14.4%	14.2%	85.7%	86.1%	85.7%
SNAP [1]	2000	789	403	362	185	261	87.0%	68.5%	68.5%	31.5%	31.5%	68.5%	68.3%	59.6%
Panther* [2]	2000	510	196	503	181	610	69.5%	50.3%	52.0%	49.7%	48.0%	50.8%	53.4%	35.3%
Pmut [3]	2000	581	720	270	418	11	99.5%	68.3%	63.3%	31.7%	36.7%	65.4%	62.0%	65.0%
PolyPhen [4]	2000	728	789	206	272	5	99.8%	77.9%	74.4%	22.1%	25.6%	76.0%	75.8%	75.8%
PolyPhen-2** [5] HumVar	2000	773	666	211	134	216	89.2%	78.6%	83.2%	21.4%	16.8%	80.7%	81.9%	72.0%
PolyPhen-2 [5] HumDiv	2000	776	655	222	131	216	89.2%	77.8%	83.3%	22.2%	16.7%	80.2%	81.3%	71.5%

Number of cases all programs could predict: 1088

For predictions, the MutationTaster simple amino acid exchange model (simple_aae) was applied. tp: true positive; tn: true negative; fp: false positive, fn: false negative; nd: not determined nd means that the tool could not generate a prediction; for Pmut, PolyPhen (proteins > 30.000 amino acids can not be queried) and SNAP this often happened when large proteins were queried). Panther comes up with so many nd cases due to frequently occurring alignment problems.
Accuracy (this tool): accuracy based on all cases a program (this tool) could analyze; accuracy = (tp + tn) / nd
Accuracy (common): accuracy based on the common cases which all programs could analyze; accuracy = (tp + tn) / (tp + tn + fp + fn + nd)
Accuracy (all cases) accuracy based on all cases (i.e. all 2000 submitted cases), cases without a prediction are counted as wrong predictions; accuracy = (tp + tn) / 2000
* Panther does not offer an explicit prediction whether or not a mutation is probably disease-causing but only gives a probability (a value between 0 and 1) that a mutation has a deleterious effect. For our statistics, we needed a clear cut-off in order to allocate predictions to TP, FP, TN and FN. A cut-off of P_deleterious of 0.5 was suggested to us by the Panther team. However, the poor correct prediction rate might rather reflect a non-optimal cut-off than a poor predictive performance.
**PolyPhen-2 is the improved successor of PolyPhen.

Click here to see our test set and the results.

Literature:

[1]	Bromberg Y, Rost B: SNAP: predict effect of non-synonymous polymorphisms on function. Nucleic Acids Research, 2007, Vol. 35, No. 11 3823-3835.
[2]	Thomas PD, Campbell MJ, Kejariwal A, Mi H, Karlak B, Daverman R, Diemer K, Muruganujan A, Narechania A: PANTHER: a library of protein families and subfamilies indexed by function. Genome Res. 2003 Sep;13(9):2129-41
[3]	Ferrer-Costa C, Gelpi JL, Zamakola L, Parraga I, de la Cruz X, Orozco M: PMUT: a web-based tool for the annotation of pathological mutations on proteins.Bioinformatics. 2005 Jul 15;21(14):3176-8. Epub 2005 May 6.
[4]	Ramensky V, Bork P, Sunyaev S: Human non-synonymous SNPs: server and survey. Nucleic Acids Res. 2002 Sep 1;30(17):3894-900.
[5]	Adzhubei IA, Schmidt S, Peshkin L, Ramensky VE, Gerasimova A, Bork P, Kondrashov AS, Sunyaev SR: A method and server for predicting damaging missense mutations. Nat Methods. 2010 Apr;7(4):248-9.