Search This Blog

Friday, February 18, 2011

Ordinal scales - #$#%!

I recently had rotator cuff surgery and now I'm in physical therapy. At every session the therapist asks "how would you rate your pain today on a 10 point scale?". They tell you that 0 is "no pain" and 10 is "the worst pain you can imagine".

How am I supposed to answer this question? It always hurts. Some days only a little, some days a little more, and some days a lot. How am I supposed to tell the difference between a 3 and 4? I've never actually said anything bigger than 5. Does that mean that my 5 is really a 10 because it's the worst pain I have or is my 5 really a 5 because I can imagine much worse pain (being skinned alive, dental drilling without novacaine, etc.)

I'm not saying that there is no such thing as ordinal data. Many phenomena of interest are naturally ordinal (good, plus good, double plus good!). The problem occurs when we impose a numeral-based scale on an ordinal phenomena. People start to think that the numerals represent real numbers. Then they think it makes sense to analyze them as numbers: "The patient's average pain this week was 3.428."

Fortunately, my therapist has never made a statement like that, but course evaluations where I teach are done on numeral-based ordinal scales and every semester the institution computes means and standard deviations for teachers, classes, divisions, etc. It's statistical nonsense.

9 comments:

  1. I feel like this is an example of why it's important to specify what we are trying to analyze when we collect data. For example if your physical therapist would specify whether or not he wanted 10 to be worst pain imaginable or the 10 being the worst pain you've ever felt, he could better analyze the data and possibly doctors could better help their patients.

    ReplyDelete
  2. I recently went through this same situation with being in rehab for my IT bands for track. There was a scale of how much pain I felt that day and everyday I could have said a 1 or a 10 but my own mental toughness could have altered my opinion and been completely different than say what they would have put on their scale of 1 to 10. I do agree with Kait that her way would make more sense. Although I don't think teachers should be based off of these numbers I guess in some respects it is the only real judgement from a business stand point to see how they are performing in their respective classes.

    ReplyDelete
  3. I went through this same situation at rehab when I tore my ACL. Every time I went in, the therapist would ask me my pain level right when I got there (0-10) and then after we did the theraputical exercises. I don't think this scale is accurate because how do you know that when you think it's a 6, that someone else who might be feeling the same pain says they feel a 9 or a 3? I agree with Brian in saying that your mental toughness affects the number you say. I think that using ordinal scales to compute evaluations of the class or teachers is unrealistic because you face the same problem. What if there are 2 people who feel the exact same way about a class or professor but one puts 4 and the other 8? There are too many variables that can affect how a person scores something which makes the ordinal scale ineffective.

    ReplyDelete
  4. I have never gone through rehab but I understand the situation and points well. Giving anything a number on a scale is a matter of someone’s perception. In the rehab situation you may be so used to pain that your three is someone’s ten or your four is someone’s one, it is dependent on the person And the time. Take another ranking system; “measure of someone’s hotness” I may think that Emma Watson is a nine but someone else may think she’s a five; it’s a matter of the persons perception and taste. Now take a coin grading scale; this scale is universal a very fine grade coin is the same to everyone that’s just what it is. Your pain level in rehab is not universal its only you at that time, if the highest number you give them is five and you fluctuate from one-five then they can still tell how you’re doing, so that system is working for what they need it for.

    ReplyDelete
  5. This is so true. This year we played Hope College and there was a play were I dove for the ball and someone jumped on my head resulting in me hitting my head on the court. My nose was pretty messed up almost broken, and I had to go get stitches on my nose as well. When the trainers brought me off the court they were asking me some of the same questions. On a scale one to five how dizzy are you and how bad does your head hurt. Of course I said 1 so I could try and get back into the game but it really was a 5 haha. So asking pain on a scale is unrealistic because most of the time people will not tell the truth. Most people will just through a number out unless they are in a lot of pain. Rating teachers is different because everyone is titled to there opinion, and most ratings today are done by numbers anyways so it is relevant.

    ReplyDelete
  6. I also went through physical Therapy for my Labrum. The doctor would also ask me on a scale of 1-10 how did the pain affect my throwing and my daily life. I dont see how a number can even represent pain. Someone Else's 6 could be a 8 for you and cause the doctor to give you a wrong therapy.

    ReplyDelete
  7. The use of numeral-based ordinal scales to describe issues such as, the feeling of pain or the opinion of how a college course is taught is a typical misrepresentation of expressions. First, it is very difficult for a person to express his or her feelings by using numerals. Expressions are very difficult for people to describe using words alone, and these feelings are too detailed to be expressed by just using a number. Finally, it is extremely hard to analyze data that expresses feelings, because numeral-based ordinal scales cause expressions to become vague. The differences in toleration of pain for example, can cause data to be misrepresented. Two athletes may have suffered significant injuries, and when asked by the athletic trainer about the level of pain, the two athletes' answers may differ. Which may result in the athletic trainer assuming a different type of injury, therapy, and the amount of time sat out of competitions. In the end, the misrepresentation of feelings and expressions by the use of numeral-based ordinal scales can confuse not only the people being asked to express their feelings, but also the people who analyze their responses.

    ReplyDelete
  8. While the one though ten scale is a flawed system due to personal experiences it has its place. Every time I have been in the hospital (which has been more then a few times) I have only declared a 10 once. That was when I came out of an 12 hour procedure. Now someone who had the same surgery could declare a ten and be experiencing more or less pain then I was at the point. However to them it is the worst pain that they have ever felt. So they want the doctors to do their best to fix it as fast as they can. While the goal is for you to obviously be in no pain this scale gives doctors an idea of how much they need to work with you, what they need to prescribe you, and make judgment calls on what treatments to give you. While this scale is easily misinterpreted it is just suppose to be a quick easy guideline to where you are in your recovery. From my experiences every Dr. who has asked me this as often probed further an comes to a conclusion after I have answered several more detailed questions.

    ReplyDelete
  9. I totally agree, there is no way that an ordinal scale is useful in this situation. A scale from (1-10) is too large to accurately define pain levels. I would use a shorter scale from (1-5), not saying that this would make an ordinal scale sufficient for pain levels but it would eliminate the in between numbers. To conclude, there is no way this scale will work because every patient is going to have different pain levels. Unless the doctor can feel each patients individual pain and write it down, it is impossble for an ordinal scale to be accurate.

    ReplyDelete