How reliable is bioinformatics software? A challenge in determining the reliability and accuracy of many tools is that in the search for new knowledge, a ‘gold standard’ to test against does often not exist. Metamorphic testing can be used to overcome this issue by comparing the output of bioinformatics programs when input data is deliberately altered, as shown in recent work (Giannoulatou 2014, Kamali 2015). Approaches inspired by metamorphic testing can reveal biases by discovering unexpected changes in program output .
To encourage the use of metamorphic testing in bioinformatics, Mighty Morphin FASTA Files (MMFF) is an open source Python script and an accompanying web interface for generating biological test data in the common FASTA sequence file format.
It is intended to complement other forms of testing on programs that use the FASTA file format. Various metamorphic relationships (MRs) can be applied to a seed of test data to produce additional FASTA files for each MR. Changes to the file format, sequence header, sequence content (nucleotides or amino acids), and sequence order can each be applied, and the results compared to unmodified data.
MMFF can also produce other classes of test data such as boundary and coverage testing. It is a simple framework for quickly generating test data that can then be run through existing programs and pipelines or during the development of new programs.
Mighty Morphin FASTA Files (MMFF) can be used to improve and validate bioinformatics software. Silly name, serious purpose.
Eleni Giannoulatou, Shin-Ho Park, David T Humphreys, Joshua WK Ho. Verification and validation of bioinformatics software without a gold standard: a case study of BWA and Bowtie. BMC Bioinformatics 15, S15 Springer Nature, 2014. Link
Amir Hossein Kamali, Eleni Giannoulatou, Tsong Yueh Chen, Michael A. Charleston, Alistair L. McEwan, Joshua W. K. Ho. How to test bioinformatics software?. Biophys Rev 7, 343–352 Springer Science \(\mathplus\) Business Media, 2015. Link