JOM - Volume 36, Number 9, September 1994

Reliability and Validity of a Newly Developed Test of Physical Work Performance

Deborah E. Lechner, MS, PT

James R. Jackson, PhD

David L. Roth, PhD

Karin V. Straaton, MD

This study examined the interrater reliability and validity of a newly developed test of physical work abilities, the Physical Work Performance Evaluation. Eleven physical therapists were trained to administer and score this evaluation. From this group, two therapists at a time simultaneously and independently evaluated 50 patients with musculoskeletal disorders as they performed the tasks of the Physical Work Performance Evaluation. At the conclusion of the evaluation. each therapist determined the safe level of physical work for each patient. A comparison of the two independent evaluations was used to determine reliability. To determine validity, the predicted level of work was compared with the actual level of work. Kappa coefficient between the two therapists on the level of work was .83. Spearman who correlations between the predicted and actual levels of work ranged from .41 to.55. Only 14 to 18% were working above the level predicted by the Physical Work Performance Evaluation. These results indicate high interrater reliability. Given the lack of a perfect standard -for validity comparisons. these results also provide evidence in support of convergent validity. The test can be used in making decisions regarding return to work after injury, preemployment placement, and vocational exploration.

From the Multipurpose Arthritis and Musculoskeletal Diseases Center (Ms Lechner. Dr Straaton) and the School of Social and Behavioral Sciences (Dr Roth), University of Alabama at Birmingham: and the Office of Educational Development. University of Alabama School of Medicine I Dr Jackson).

Address correspondence to: Deborah E. Lechner. MS. FT. Division of Physical Therapy, University of Alabama at Birmingham. B-41 SHRP Building, 1714 Ninth Avenue South. UAB Station. Birmingham, AL 35294-1270.

0096-1736/94/3609-0997$03.00

Copyright C by American College of Occupational and Environmental Medicine

Traditional physical evaluation procedures such as testing muscle strength, measuring joint range of motion, interpreting radiographic findings, and recording medical signs and symptoms are not good predictors of functional abilities.'-'-' Therefore, functional capacity evaluations have developed from a need for a more objective measure of a person's physical abilities to perform work.

Several functional capacity evaluations are available commercially but few provide both comprehensive and objective data. In addition, rigorous reliability and validity studies of these tests are lacking.' The purpose of this study is to determine the interrater reliability and concurrent validity of a newly developed functional capacity evaluation. the Physical Work Performance Evaluation (PWPE).

Methods

Subjects

The PWPE was used to evaluate 50 subjects between the ages of 18 and 65 years with a variety of musculoskeletal disabilities who were recruited from the outpatient rheumatology clinic at the University of Alabama at Birmingham Hospital. Subjects were either working at least 20 hours per week or were not working because of a musculoskeletal disability. "Not working because of a musculoskeletal disability" was operationally defined as those who were receiving Social Security disability payments as a result of musculoskeletal disability. The nonworking subjects also were judged as unable to work by a physician's global assessment. Subjects with major comorbidity were excluded.

All eligible subjects entering the rheumatology outpatient clinic were asked by a rheurnatologist (KS) to participate in the study. The subject recruitment was carried out over 2 years until 50 qualified subjects had agreed to participate. Subjects were reimbursed for their time from funding made available through the University of Alabama Multipurpose Arthritis and Musculoskeletal Diseases Center.

A research assistant contacted the subjects and explained the study to them. Those subjects who volunteered for the study were mailed a consent form and a questionnaire regarding their current work status. The study was approved by the Institutional Review Board for Human Use at the University of Alabama at Birmingham. The subjects were asked to return the questionnaire by mail to a research assistant. The physical therapist administering the test was blinded to the work status of the patients so as not to bias the outcome of the validity component of the study.

Instrumentation

The PWPE is a test of a person's ability to perform physical work and consists of 36 tasks that evaluate dynamic strength, position tolerance, mobility, balance. endurance and coordination, and fine motor skills. Test items include the 20 physical demands of work as defined by the Department of Labor in the Dictionary of Occupational Titles." The test was developed to enhance the consistency and objectivity of the therapist's judgment in determining a worker's maximum safe physical working ability.

The PWPE consists of six different sections: dynamic strength, position tolerance, mobility, fine motor skills, coordination, and balance and endurance. The dynamic strength section evaluates the subject's ability to lift, carry, push, and pull. The position tolerance section evaluates the subject's ability to assume and maintain various positions frequently required for work. The mobility section evaluates the subject's ability to walk, climb stairs, climb a ladder, crawl, and perform repetitive motions. The coordination section is a brief screen of the subject's ability to perform rapid alternating and eye-hand-foot coordination tasks. The balance section screens bilateral dynamic and unilateral static balance. Balance and coordination are screened rather than tested thoroughly because comprehensive balance and coordination testing would add a considerable amount of time to an already lengthy (3.5- to 4.5-hour) evaluation process. For the industrial rehabilitation patient, balance and coordination are less often a problem than are dynamic strength, position tolerance, and mobility. The endurance section evaluates the subject's ability to tolerate an 8-hour work day. In this section of the test, the subject is asked to repeat the first three tasks at the end of the test. Endurance projections are made based on changes in the subject's heart rate response and performance. Because many fine motor tests that have been standardized and tested for reliability and validity are currently available, the authors and the developer of the PWPE (DL) decided not to evaluate these tests further in the current study.

The dynamic strength, position tolerance, and mobility sections of the test are used to determine the overall level of work for which a person is capable and, therefore, were the focus of our reliability and validity study. The Figure provides a list of the 19 tasks included in these three sections and a flow chart of the testing process.