Rating visualization in shoulder arthroscopy: A comparison of the visual analog scale versus a novel shoulder arthroscopy grading scale

Introduction

Asatisfactory visual field is essential for surgeons to perform arthroscopic surgery effectively. During arthroscopy, the quality of the visual field is negatively affected by blood mixing with the irrigation fluid. In knee arthroscopy, the visual field can be improved by controlling bleeding with the use of a tourniquet. Since this is not an option in shoulder arthroscopy, a variety of techniques to control bleeding have been utilized to improve visualization. These techniques include the use of flow-controlled or pressure-controlled pumps, the use of hypotensive anesthesia, and the use of epinephrine in the irrigation fluid.^[1-4]

The visual analog scale (VAS) is a subjective tool originally developed by clinicians to assess pain in patients.^[5] Its use has grown recently and it has been used to describe multiple subjective measurements, such as chondropathy of the knee, fatigue, functional capacity, tension, and psychiatric state.^[6,7] Two recent studies applied the VAS to describe the quality of visualization in shoulder arthroscopy. They both utilized the VAS rating to compare the hematocrit level in irrigation fluid with and without the use of epinephrine.^[1,2] While the authors have been able to show utility in the use of the VAS in the setting of visualization in shoulder arthroscopy, it has not been formally validated. Due to its wide scale (0–10) and subjectivity, the VAS may not be an optimal scoring system for rating arthroscopic visualization and a novel rating system with a smaller range of scores and more objective criteria may be needed.

The purpose of this study was to assess the interobserver reliability and intraobserver variability of the VAS for visualization in shoulder arthroscopy and compare it to a less variable, more objective grading scale, the shoulder arthroscopy grading scale (SAGS) validating their use. Our hypothesis was that the less variable, more objective SAGS would show significantly improved interobserver reliability and intraobserver variability among raters.

Methods

After obtaining approval from our Institutional Review Board, 20 separate 30-s length video clips were created from a larger library of shoulder arthroscopies. The library was derived from cumulative procedures performed by all sports medicine-trained physicians. All arthroscopic cases were performed using the same arthroscopic equipment (Arthrex AR-3200 Synergy HD3 Tower with dual wave pump and standard 30° scope). Thirty seconds was deemed appropriate to allow each rater adequate time to determine if full visualization of anatomic structures was achieved and if surgical task was completed. Each video clip was de-identified and created to capture different examples of visualization. The 20 videos consisted of 15 video clips depicting visualization in the subacromial space and five within the glenohumeral joint. The clips were then randomized utilizing a random number generator (random.org) to an order of 1–20 and distributed to six different raters. After their initial evaluation, there was a 1-month interval in data collection following which the 20 video clips were re-randomized and distributed to the six raters for a second round of evaluation. One month was agreed on collectively by raters. All raters were board-certified and fellowship-trained sports medicine orthopedic surgeons with widespread shoulder arthroscopy experience. Each rater was asked to complete an evaluation form during each round [Figure 1]. The evaluation forms asked each surgeon to rate the visualization according to the VAS^[2] and to the SAGS (as detailed below).

Close

Figure 1

Example of evaluation form showing instructions to each rater with descriptions for visual analog scale and shoulder arthroscopy grading scale

Export to PPT

Results

Shoulder arthroscopy grading scale

The interobserver reliability showed an excellent degree of consistency (ICC = 0.97, 95% CI = 0.94–0.98). The intraobserver variability also demonstrated strong consistency, with five of the six raters average ICCs ranging from 0.61 to 0.93.

It is important to note that one rater demonstrated poor consistency regarding intraobserver reliability when using the VAS and SAGS, with ICCs of 0.06 and 0.27, respectively. Tables 1 and 2 demonstrate recorded results of the SAGS and VAS.

Table 1 Interobserver reliability averages for visual analog scale and shoulder arthroscopy grading scale

Table 2 Intraobserver reliability for each rater for visual analog scale and shoulder arthroscopy grading scale

Discussion

Our study found a strong-to-excellent degree of consistency in rating intraoperative visualization using both the VAS and the SAGS. The use of VAS for rating visualization in shoulder arthroscopy was initially supported by Jensen et al.,^[1] who demonstrated that the addition of epinephrine to irrigation fluid seems to reduce intra-articular bleeding during routine arthroscopic shoulder surgery and improved visualization. The VAS was then validated by Avery et al., who also found that visualization improved when epinephrine was added to irrigation fluid. However, Chierichini et al.^[10] evaluated the effects of different additives into irrigation fluid on visualization and found that the VAS was unreliable for this application. The need for a formally validated scoring system is evident and lacking in literature despite its use and support in previous studies.^[1,2,10]

Proper communication is the primary goal of any universal scoring system. A successful system should demonstrate simplicity, accuracy, and reliability.^[6] Due to the wide scoring range and subjectivity of the VAS, we expected lower interobserver and intraobserver variability for rating arthroscopic visualization. However, our results show that the VAS was comparable in consistency to our more objective four-point grading scale. Five of our six raters demonstrated a high degree of consistency (83.3%), providing preliminary evidence of the reliability of the two scoring systems.

Our study has several strengths. First, a large number of evaluators scored visualization in a large number of video clips from a variety of shoulder arthroscopic procedures. This should reduce the bias that any individual evaluator or video clip would have upon the results. Second, all evaluators were sports medicine fellowship-trained physicians who have widespread experience with arthroscopic images; therefore, their scrutiny of scoring visualization should be higher than that of other subspecialties, which comports with previous studies.^[1,2,10] Finally, similar to previous work by Bellamy et al.,^[11,12] our use of ICCs to measure consistency of responses is important for validating each scoring system since small variations (i.e., one-point change in VAS) are unlikely to affect technique.

Conclusion

Given the strong-to-excellent degree of consistency in using the VAS and the SAGS, either can be reliably used as a measurement of visualization in shoulder arthroscopy. The objective criteria and the smaller discrete options of the SAGS may lead to more reliable communication among surgeons. However, the relative ease of understanding and use of the VAS can also allow it to be the easier grading scale to use among different specialties within and outside of orthopedic surgery.