Introduction Otoscopy is a key clinical skill which following the introduction of the Medical Licensing Assessment all newly qualified doctors in the United Kingdom will be required to be able to perform independently. At present there is no consensus on a standardised method for objectively assessing otoscopy skills. Methods Prospective mixed methods study comparing face, content and construct validity of three different platforms for otoscopy skills assessment, using a traditional otoscope with manikin, digital otoscope (Tympahealth) with manikin, and traditional otoscope with a low-cost model ear (SimEar). Skills were assessed using a standardised mark scheme, and five expert assessors and twelve participants who rotated through three Objective Structured Clinical Examination (OSCE) stations representing each model. Assessors numerically ranked validity of each model, and participated in a semi-structured interview of opinion. Results Each platform differed in face, construct and content validity scores, with no one platform consistently outperforming others. Three main themes were identified during thematic analysis of expert assessor interviews: ability to assess what is seen, anatomical reality, and ease of use. The low-cost model showed greatest potential, where modification to include a silicone ear could lead to high validity with marginal increase in cost. Conclusion Several modalities for assessing otoscopy skills exist, each with advantages and disadvantages. Modifications to a low-cost model, for use with either a traditional or digital otoscope, could prove to be the best model.