Inter-observer agreement in assessment of clinically overt extra nodal
extension (cN3b)
Key Points
- Clinical N3b staging is defined as unequivocal and unambiguous signs
of extra nodal extension (ENE), such as skin involvement, muscle
invasion or nerve dysfunction, supported by radiological evidence.
- There is a paucity of studies determining interobserver reliability of
clinical detection of overt ENE.
- A prospective comparative cohort study assessing 12 Head and Neck
surgeons’ inter-observer agreement in detecting the components of
overt ENE.
- We reveal high surgeon agreement in the ability to detect lymph nodes,
the presence of skin involvement and in surrounding nerve involvement.
- Variability of surgeon assessment was seen in the assessment of overt
muscular involvement.
Data Availability Statement
The data that support the findings of this study are available from the
corresponding author upon reasonable request.
Introduction
Regional lymph node metastasis is known to be one of the most powerful
prognostic indicators in head and neck cancer. Furthermore, the presence
of extra nodal extension (ENE) in these metastatic nodes is of profound
prognostic significance. 1,2,3,4,5.
In 2018 the American Joint Committee on Cancer (AJCC) released the
eighth edition of the AJCC Cancer Staging Manual with several major
changes to Head & Neck staging6. These changes
included the introduction of a new N3b stage for ‘clinically overt’
evidence of ENE in nodal characterization of high-risk Human Papilloma
Virus (HPV) -ve and non nasopharyngeal carcinomas6,7.
The AJCC has defined clinical N3b staging as clinically overt signs of
gross ENE, such as skin involvement, muscle invasion or dysfunction to
surrounding nerves, supported by radiological
evidence6,7. This differs significantly from
pathological N3b on the other hand is defined as extension of metastatic
carcinoma through the fibrous capsule of a lymph node into the
surrounding connective tissue, regardless of the presence of stromal
reaction and is further subcategorized as macroscopic and microscopic
ENE6,7.
In the literature, the accuracy of both pathological and radiological
assessment of ENE has been previously assessed. It is well documented
that early or microscopic ENE can be reliably identified on pathologic
examination and given a quantitative value in
millimetres8. The ability to diagnose ENE in radiology
has also been extensively studied in both MRI and CT imaging with varied
results8,9,10.
However, there is no study to date assessing accuracy of clinical
examination in reliably detecting ENE and moreover no studies assessing
whether clinicians agree on cases ‘clinically overt’ ENE. The terms
overt, unequivocal and unambiguous rightly encourage a very high bar for
inclusion in this staging category. The stringency of the wording is
designed to prevent stage migration and the requirement of supportive
radiology further reinforces this. One might suspect that this
clarification would provide for unanimous agreement in making this
clinical diagnosis, but several questions must be answered to
empirically support this assumption. Can surgeons accurately and
consistently identify overt ENE on their clinical examination alone? Is
there variability between surgeon’s assessments of the same patient and
how does this affect the clinical staging in context of the new cN3b
AJCC 8th edition staging system? This will ultimately improve our
understanding of the AJCC HNSCC staging and give insight into whether
cN3b staging is made in a consistent fashion. This remains of critical
importance because without clinical suspicion, radiological evidence
alone, however clear, does not in itself permit the cN3b diagnosis to be
made.7 We aimed to investigate the accuracy and
inter-observer agreement of Head and Neck surgeons in their clinical
examination & assessment of clinically overt ENE and compare surgeons
assessment to radiological findings consistent with clinically overt
ENE.
Materials and Methods
This was a prospective cohort study conducted with the participation of
12 Head and Neck Surgeons at a weekly Head & Neck Cancer
Multidisciplinary Meeting at a high-volume tertiary referral hospital.
Institutional approval from a Human Research Ethics Committee was
obtained prior to commencing. We included all patients with new
diagnosis of Head and Neck SCC (either cutaneous or mucosal) with
palpable cervical lymph nodes of any size who had accompanying cross
sectional radiology (CT or MRI) (N=64). Palpability of lymph nodes was
determined by the senior author (SV). Participating clinicians were
consultant Head and Neck Surgical Consultants across the ENT, Plastic
Surgery and Oral and Maxillofacial Surgery departments. Prior to the
study all surgeons attended a briefing explaining the study, with
further clarification of the cN3b stage and the need for overt and
unambiguous clinical findings for diagnosis. Data were collected over a
five-month period. Head & Neck Surgeons were blinded to all
radiological findings prior to clinical assessment and asked to a)
assess the presence of palpable cervical lymph nodes and b) identify
specific physical examination findings of overt ECE in partivcular
unambiguous skin involvement, muscle invasion and/or dysfunction to
surrounding nerves. Surgeons completed a study proforma in a dichotomous
(‘yes’ or ‘no’) fashion, along with an illustration of the location and
number of lymph nodes for that patient (Fig 1). Radiological assessment
of each patient was performed by a Head and Neck Radiologist blinded to
the clinical findings. Again, the participating specialist documented
their findings in a dichotomous fashion (‘yes’ or ‘no’) using the
proforma (Fig 1). Statistical Analysis was performed calculating
Inter-rater agreement using the overall percentage agreement and Fleiss’
Kappa statistic, which allows for more than two raters to be evaluated.
Coefficients were calculated for the presence of palpable lymph nodes,
skin involvement, muscle invasion and dysfunction to surrounding nerves.
We used STATA version 15.1 (Stata Corporation, College Station, Texas,
USA) for the analysis. Guidelines for Reporting Reliability and
Agreement Studies (GRRAS) was utilised.
Results
A total of 82 palpable lymph nodes in 64 patients were assessed for
clinically overt ENE. On average, three clinicians clinically assessed
each patient with a total of 246 lymph node assessments.
A total of 246 lymph node assessments over a five-month period revealed
substantial agreement (93%) with regards to the ability of surgeons to
detect palpable lymph nodes in the patient sample. Statistical analysis
with identification of the clinical signs of overt ENE revealed that
there was perfect overall percentage agreement and inter-observer
agreement for invasion of skin (100% and Fleiss’ Kappa=1, respectively)
and almost perfect overall agreement on interobserver agreement on
dysfunction of surrounding nerves (99% and Fleiss’ Kappa=0.91,
respectively). Lower overall percentage agreement and inter-observer
agreement were demonstrated for assessment of overt muscular
infiltration with 81% and Fleiss’ Kappa=0.53, respectively.
Comparison of clinical and radiological assessment was also performed.
Where all surgeons agreed there was clinically overt ENE (n=13), there
was concordant radiological evidence of features of overt ENE in all
cases. Conversely in patients where the surgeons all agreed that
clinically overt ENE was not present (n=33) all cases demonstrated no
radiological evidence of ENE. All cases where some surgeon disagreement
existed as to the presence of overt ENE were based on the presence or
absence of obvious muscle invasion or tethering to surrounding
structures. (n=18). Each of these patients were assessed by three
surgeons giving a majority of agreement or disagreement in each case. Of
the ten patients in whom most surgeons agreed on the presence of
muscular invasion a total six (6/10) demonstrated clear radiological
evidence of muscular or deep tissue invasion. Of the eight patients in
whom most surgeons felt that obvious muscular invasion did not exist,
only one (1/8) showed radiological evidence of muscular invasion.
Discussion
This novel prospective comparative cohort study is the first of its kind
looking at inter-observer agreement between Head and Neck surgeons in
the assessment and identification of the clinical components of stage
cN3b disease. Consistency in the staging of patients is important for
several reasons. Staging allows for accurate prognostication which is
valuable both to the patient and the treating clinician. When
undertaking research, comparisons between patients are only valid when
staging has been performed accurately and uniformly. Additionally,
staging can at times aid in decision making with regards to best
clinical management of the patients. The new N3b staging in head and
neck cancer is a highly valuable addition for patient prognostication.
The stringency for making the diagnosis is clear from both the
unambiguous nature of the clinical findings required and confirmation
with radiology. Our findings demonstrate a high level of consistency in
making this diagnosis. This was particularly the case for cutaneous or
nerve involvement by a lymph node. Evidence of overt muscular invasion
showed high level of agreement but was not unanimous. When all surgeons
agreed in their clinical assessment, the radiological findings concurred
in every case. This was the case in 46 out of 64 patients. Based on our
findings several patients might have been staged differently by
different surgeons. These are all, by definition, patients with
radiological evidence of ENE (without this evidence the cN3b diagnosis
cannot be made). In several cases where there was disagreement, surgeons
reported during tumour board discussion that they had a suspicion of
muscular invasion but that it was not sufficiently unambiguous to meet
the criteria. In the same circumstance’s others felt that there was
sufficient tethering or fixation to label it unambiguous. Experience of
the surgeon was not assessed.
As a multidisciplinary team it is often possible to get multiple
surgeons to examine one patient. If during our study the ‘majority
decision’ was to be used in making the cN3b diagnosis only one patient
was at risk of mis-staging (1/64).
Conclusion
A total of 246 lymph node assessments were performed in 64 patients,
between 12 Head and Neck surgeons. Our study has revealed high surgeon
agreement in the ability to detect lymph nodes, the presence of skin
involvement and in surrounding nerve involvement. Some variability of
surgeon assessment was seen in the assessment of overt muscular
involvement. Multiple surgeon assessment of each patient with acceptance
of a majority decision on clinical findings increased concordance with
radiological findings.
References
1. Myers JN, Greenberg JS, Mo V, et al. Extracapsular spread. A
significant predictor of treatment failure in patients with squamous
cell carcinoma of the tongue. Cancer. 2001;92:3030-3036.
2. Wreesmann VB, Katabi N, Palmer FL, et al. Influence of extracapsular
nodal spread extent on prognosis of oral squamous cell carcinoma. Head
Neck. 2016;38(Suppl 1):E1192-E1199.
3. Bernier J, et al. Defining risk levels in locally advanced head and
neck cancers: a comparative analysis of concurrent postoperative
radiation plus chemotherapy trials of the EORTC (#22931) and RTOG (#
9501) Head Neck. 2005;27:843–850. doi: 10.1002/hed.20279.
4. Cooper JS, et al. Long-term follow-up of the RTOG9501/intergroup
phase III trial: postoperative concurrent radiation therapy and
chemotherapy in high-risk squamous cell carcinoma of the head and neck.
Int. J. Radiat. Oncol. Biol. Phys. 2012;84:1198–1205. doi:
10.1016/j.ijrobp.2012.05.008.
5. Johnson JT, Barnes EL, Myers EN, Schramm VL Jr, Borochovitz D, Sigler
BA. The extracapsular spread of tumors in cervical node metastasis. Arch
Otolaryngol. 1981;107(12):725-729.
6. Amin MB, Edge SB, Greene FL, et al (eds). AJCC Cancer Staging Manual,
8th ed. New York: Springer; 2017
7. Lydiatt WM, Patel SG, O’Sullivan B, Brandwein MS, Ridge JA, Migliacci
JC, Loomis AM, Shah JP. Head and neck cancers—major changes in the
American Joint Committee on cancer eighth edition cancer staging manual.
CA: a cancer journal for clinicians. 2017 Mar;67(2):122-37.
8. Prabhu RS, Magliocca KR, Hanasoge S, et al. Accuracy of computed
tomography for predicting pathologic nodal extracapsular extension in
patients with head-and-neck cancer undergoing initial surgical
resection. Int J Radiat Oncol Biol Phys. 2014;88:122-129.
9. Carlton JA, et al. Computed tomography detection of extracapsular
spread of squamous cell carcinoma of the head and neck in metastatic
cervical lymph nodes. Neuroradiol. J. 2017;30:222–229.
10. Chai RL, et al. Accuracy of computed tomography in the prediction of
extracapsular spread of lymph node metastases in squamous cell carcinoma
of the head and neck. JAMA Otolaryngol.–Head Neck Surg. 2013;139:1187–
1194. doi: 10.1001/jamaoto.2013.4491.