Commonly, the efficacy of new FMECA methods is conducted through qualitative comparisons between rankings; for a small number of failure modes, this approach is suitable but can become unpractical or lead to misleading results for more extensive problems. This fact motivated us to introduce an alternative approach to compare different FMECA methods based on agreement metrics that allow the statistical comparison between rankings generated by independent raters. Despite its relevance, the application of agreement coefficients is limited in the FMECA context. The proposed approach considers the agreement assessment between different methodologies used in FMECA analysis (Risk Priority Isosurfaces RPI, VIKOR, ITWH, and Type-II Fuzzy Inference System) when applied to a study case regarding blood transfusion widely used in the literature for benchmarking and consisting of eleven failure modes. We selected the RPI methods as a reference to compare the other forenamed methods. Results show that our agreement coefficient-based comparison approach proves effective for the statistical comparison of different FMECA methods instead of the rankings qualitative comparison.