Following the initial down-selection, this material was imported into the NVivo™ qualitative data analysis software package to carry out a systematic assessment of patterns occurring in the literature, enabling the identification of commonly occurring validation themes. This consisted of several stages:
  1. Co-occurrence: First, a combined text search query was conducted for synonyms of the key terms ‘valid’ and ‘method’ occuring in the same paragraph within each individual literature source. This process enabled all of the paragraphs discussing validity to be identified for both ABM and SD modelling techniques from the previously selected sources.
  2. Cross-tabulation: The results of this combined text search query were then processed using NVivo's autocode function to generate a 'Node Matrix' based on the paragraphs where co-occurrence had been identified. This produced a hyperlinked reference table of the most commonly occurring terms within the paragraphs of interest across the complete set of papers (in this case terms relating to the validity of methods employed). This step provided a systematic means of ranking both the relevance of each individual literature source to the discussion of model validity, as well as specific terms within the cross-tabulated results, by assessing the relative frequency and clustering of stemmed terms (based on Pearson’s correlation coefficient \cite{RN952}) in these highlighted paragraphs. This is illustrated in the example weighted keyword list and dendrogram provided in Fig. \ref{479717} and Fig. \ref{385644} respectively.
  3. Identification of validation themes from clustered terms: Using the dendrograms generated from the cross-tabulation of literature sources, core themes associated with model validity were identified from the clustered terms relating to ABM and SD techniques. This was based on identifying connecting natural language patterns between key words within each cluster that fitted the context of the overall analysis (i.e. relevant to the validation of simulation methods), as shown in Fig. \ref{385644}.