B-Schooled Podcast Episode #311: Anthony Ritz on Reasoning Traps with Averages
This transcript has been lightly edited for clarity and length. Timestamps refer to the audio.
[00:04] Stacy Blackman: Welcome to B-Schooled, a podcast by Stacy Blackman Consulting, bringing you insight into all aspects of the MBA. I’m Stacy Blackman, the founder of SBC, and today I’ll be your host.
Okay, welcome back to the B-Schooled podcast. I am happy to be here today with Anthony Ritz, Stacy Blackman’s own test prep wizard. He is going to teach me and all of you some new subject matter. So take it away, Anthony.
Anthony Ritz: Oh, Stacy, thank you so much for having me back on your podcast. I always love having the opportunity to meet with you and talk to everyone out there about how to do the GMAT and GRE and all of these other wonderful tests.
In anticipation of coming back on this podcast, I’ve spent a lot of time over the last couple of weeks classifying official GMAT critical reasoning questions, looking for interesting things to talk about with everyone.
And just a side note while we’re on the subject: while I was doing this classification project, I got Claude to help, because there are over 1,100 official GMAT questions that have been published over the years. So I had Claude try all the questions blind along the way, just to see if it was actually understanding everything. Because, of course, if it’s going to classify, it needs to know what it’s doing.
As I was doing it, Claude missed 15 of 96 boldface questions, which is really interesting if you remember what I was on here for last time, because that’s exactly what we just worked on. So I got a fresh copy of Claude, showed it our previous podcast episode and my related Word document, and then I had it try the questions again, blind. It only missed two. It went from 84% to 98% correct by studying that episode and reading the explainer. So everyone, if you haven’t already, go back and watch or listen to that episode, Indicator Words, Episode 301.
Stacy: Yes, amazing, amazing. If you can teach Claude that quickly, I think our test takers have a lot to learn from you. So, Episode 301.
Anthony: Great. It was a very exciting result. And Stacy, you were such a good sport last time. Everyone I showed that episode to agreed- all of my students, everyone else- that what really made it work was seeing you reason through the question using the tools that we laid out. So really, thank you for being the very best guinea pig that I’ve ever had.
[02:56] Stacy: You’re welcome. It’s humbling, for sure.
Anthony: Okay, well, we’re going to do it again, so get ready. As I was doing this classification project, 30 questions among all the ones I looked at had one particular logical flaw, which is kind of a lot. So I thought I’d like to show that flaw to all of you here today. Now, if you’re paying very close attention to the title of this episode, that may be a little bit of a hint.
Stacy: Hey, what?
Anthony: Well, I think it’s going to be a totally average episode, but that might not mean what you think.
Stacy: Okay. Love it.
Anthony: All right. So we’re going to try one. We will, of course, include the question in the notes, and I will read it aloud. And then I really hope that everyone will pause the video, set a timer for two minutes, and give this question a real shot. I think you’ll learn a lot more if you play along at home instead of just letting it wash over you.
One additional note: all of the questions we’re going to be doing are official questions, and they come from GMAC’s Official Guide, which is an amazing resource. So if you have not already done so, I would strongly encourage everyone to get a copy of the current Official Guide and do all those questions. It’s really super helpful.
Okay. So I’m going to read the question now. It starts with “Debater.”
Debater: The average amount of overtime per month worked by an employee in the manufacturing division of the Haglut Corporation is 14 hours. Most employees of the Haglut Corporation work in the manufacturing division. Furthermore, the average amount of overtime per month worked by any employee in the company generally does not fluctuate much from month to month. Therefore, each month, most employees of the Haglut Corporation almost certainly work at least some overtime.
The debater’s argument is most vulnerable to criticism on which of these grounds?
A. It takes for granted that the manufacturing division is a typical division of the corporation with regard to the average amount of overtime its employees work each month.
B. It takes for granted that if a certain average amount of overtime is worked each month by each employee of the Haglut Corporation, then approximately the same amount of overtime must be worked each month by each employee of the manufacturing division.
C. It confuses a claim from which the argument’s conclusion about the Haglut Corporation would necessarily follow with a claim that would follow from the argument’s conclusion only with a high degree of probability.
D. It overlooks the possibility that even if, on average, a certain amount of overtime is worked by the members of some group, many members of that group may work no overtime at all.
E. It overlooks the possibility that even if most employees of the corporation work some overtime each month, any one corporate employee may, in some months, work no overtime.
[06:25] Stacy: Okay, should I jump in with what I thought it was?
Anthony: I was actually thinking, if you wanted to briefly screen share, I believe you have a copy of this question. Maybe you could put it up on the screen for just a moment for everyone to see. Okay. So yes, I would love it if you let me know what your thinking is here.
Stacy: Okay. So the first one, A, is not correct because we’re only talking about manufacturing. It says that overtime per month worked by employees in manufacturing is 14 hours. Oh, wait. Oh, wait, wait, wait. Okay, now I’m getting confused.
Anthony: Tell you what, don’t worry about explaining every wrong answer. Just try to figure it out. Take a minute, please. Take your time, like everyone else will.
Stacy: So basically it’s saying that in the manufacturing division, the average amount is 14 hours. So then it’s drawing this conclusion about the average amount of overtime in the company. Okay. So A, it takes for granted that the manufacturing division is a typical division. So that is kind of correct. B, it takes for granted that if a certain average amount of overtime is worked each month, then approximately the same must be worked each month by… oh my God.
Okay. I mean, I think they’re all kind of true. Like D: just because it’s 14 hours, that can mean half of them work 40 hours a week of overtime and half work zero. An average doesn’t mean that it’s even.
Anthony: Okay.
Stacy: Right. But then A is also correct, because it’s saying that the overall company reflects manufacturing.
Anthony: Well, what I can tell you for sure is there is only one right answer, and all four of the wrong answers are actually broken in some way.
Stacy: Okay. It takes for granted that the manufacturing division is typical. Well, it is typical, because most of them are in manufacturing. Then it takes for granted that a certain average amount of overtime is worked each month by each employee. Okay. Do you want to walk it through? Because I feel like it’s going to be annoying, having me sit here thinking about it.
[09:43] Anthony: Sure, I’d be happy to. I think you landed on a couple of really interesting potential answers there. But first I want to talk through what this question is actually testing, which I think you nailed, by the way. You just didn’t have the confidence to go, “Bam, that’s the winner right there.” And that answer is D.
You gave the perfect description. It was fantastic. Yay you, absolutely. You said, and you were dead on, that just because the average is 14, that doesn’t mean that anyone actually worked 14 hours of overtime. It might not be a single person. It might be, as you said, something like half of them work zero and half of them work… I think you said 40, but in that case it’d actually be 28, right? Half zero and half 28. Or it could be a third of them work 40 and two-thirds of them work zero. There are a lot of ways you could break it down.
In fact, I went and actually did the math on it. If one crazy dude worked every minute of every day, that’s 557 hours of overtime per month, give or take, for just that one guy. And that would mean the average overtime per employee would be 14 hours a month, even if there were 40 employees in the department and none of the others did any overtime at all.
So when they say most employees work some overtime, that’s a very different statement than “the average amount of overtime is 14 hours.” The point here is that a positive average doesn’t mean that most individuals have positive values.
One more way I’ll put this: an average can hide subgroups that look very different. You really have to keep straight two different statistical measures: the mean, which is the plain old average where you add everything up and divide, versus the median, which is the middle value. When there are a few extreme values, the mean gets pulled a lot more than the median, and there can be a really big gap between the two. So when a question is talking about one and then shifts to the other, that can be an issue.
Stacy: Are people going to hate me if I say that when you’re not actually studying for these tests and under pressure, this content is actually really interesting?
Anthony: I think it is really interesting. I find it interesting. And by the way, pretty challenging. This question is probably over a 605 level on the GMAT, so it wasn’t easy, and you shouldn’t feel bad if any of you out there struggled a little bit with it. But I do agree with you. It’s interesting, and I actually think it’s a really big, important life skill too, even if you’re not studying for the GMAT.
[12:42] Stacy: It’s like figuring out assumptions. And okay, just for the record, I think if I was sitting at my desk alone, hunkered down and really focusing, I would have gotten that.
Anthony: It’s also my fault. I didn’t get it all fitted to the page, and I’m so sorry. I should have, because I noticed you had to scroll up and down. That makes it much harder too. So my bad.
Stacy: I was going back and forth. But no, that makes complete sense. The first time I read it, I read it wrong. So then I had to go back and go, “Oh, now we’re not talking about manufacturing.” You have to keep track of the details.
Anthony: Those details are super important. I was happy you caught that. Allow me to briefly talk about a couple of the wrong answers here. You specifically mentioned A. And the most-picked wrong answer here is actually E. You can look this question up on GMAT Club and a few other places on the internet, and they have some stats on what people often pick.
E is kind of a funny answer, so I’m going to start briefly with that. The first thing it says is, “Even if most employees of the corporation work some overtime each month.” That’s where I check out on E, because the conclusion was that most employees work at least some overtime. It wasn’t all. So as soon as you grant me that conclusion, it doesn’t matter what you say after that. You gave me the conclusion. Game over. That’s the biggest problem with E for the people who are picking it: it gives you the conclusion, so it can’t criticize the argument for that conclusion. And even if you get to the part about how an employee may work no overtime in some months, that can still be true even while most employees work overtime, which is, after all, what they concluded.
But if I go back up to A, that’s the one I think you were looking at. I mentioned there’s only one right answer, and the other four are truly broken. Here’s the problem with A. In the argument, they said most employees work in the manufacturing division. So even if manufacturing is not typical, even if, let’s say, all the other divisions do no overtime, as long as everyone in manufacturing works overtime, that’s still enough to say that most employees in the company work overtime, since most of the employees in the company are in manufacturing. It doesn’t have to be typical for this argument to work.
So, everyone clear? Hopefully. I guess it’s just you, Stacy, but do we feel good about this one?
Stacy: Stacy’s clear. Stacy’s clear.
Anthony: Okay, great. Well, I hope everyone else out there is as well. By the way, I went and got a couple of fun examples of this, because I think there are a lot of really great real-world places where this comes up. So if you’ll indulge me, I’d like to give you a couple.
[15:40] Stacy: Okay, should I stop the share?
Anthony: You can stop sharing for now. Thank you. So here’s one. Did you know that the average net worth of U.S. families is over a million dollars?
Stacy: Wow, that’s interesting.
Anthony: It’s true. $1.24 million, to be exact, according to the U.S. government, no less.
Stacy: How much?
Anthony: $1.24 million.
Stacy: Okay.
Anthony: But before you balk at how unrealistic that sounds, let me note that the median net worth of U.S. families, so in some sense what the typical family has, is only $215,900. That average number you just heard? That’s just the top 1% pulling the mean, baby.
Stacy: Right, right. Yeah.
Anthony: Another example, and this is a fun one. Stacy, did you know that you have an above-average number of legs?
Stacy: I didn’t.
Anthony: It’s true. Almost everyone has two legs, and the median is therefore exactly two. But a small number of people have fewer, and almost nobody has more. So the mean comes out just under two, which means you, like almost everyone else, have above-average legs.
Stacy: Okay. Okay. Interesting.
Anthony: It’s the same trick running the other way, which is sort of a preview of where we’re headed. One more fun story, but I’ll caution that this one is apocryphal. It may not have actually happened. But legend has it that when UNC compiled average starting salaries by major, geography came out on top, because Michael Jordan was in the department.
Stacy: Oh, wow. Yeah. So averages are… it’s like when the schools report average salary. Just like what you said, I’m thinking about MBAs. It could be that there are a couple of people who, yeah, same thing.
Anthony: Yeah. All of us have to be careful, not just on the GMAT, but in life. That’s what I was going to say earlier: even if you somehow stumbled in here and you’re not taking the GMAT or GRE at all, and you’re like, “What am I even doing here?”, I think this sort of data literacy is a really important life skill. Any time you’re looking at any sort of statistics, you need to be an informed and intelligent consumer of them, because they can be very misleading if you’re not careful.
Stacy: Yeah. So interesting.
Anthony: Now that we all understand the trick here, can I show you one more question? A harder version that plays the same game?
Stacy: Am I going to be the guinea pig again?
Anthony: Do you not want to be the guinea pig?
Stacy: I’m signing up. Okay.
Anthony: All right. So again, for everyone out there, we will include the question in the notes, and I will read it aloud. I strongly encourage you to pause for two minutes and give it a real shot. And again, this is from the Official Guide for the GMAT, which is an excellent resource. I strongly encourage you to go out and grab that and its supplements.
Stacy: Is there something for me to read, or am I just listening?
Anthony: Yes, it’s also on the next page of that document. For anyone wondering, I cleaned out all the answers from the version that Stacy is looking at, so she doesn’t get to cheat.
Stacy: Yes.
[19:11] Anthony: Okay.
Educational theorist: Recent editorials have called for limits on the amount of homework assigned to schoolchildren younger than 12. They point out that free-time activities play an important role in childhood development and that homework in large quantities can severely restrict children’s free time, hindering their development. But the actual average homework time for children under 12 (little more than 30 minutes per night) leaves plenty of free time. In reality, therefore, the editorials’ rationale cannot justify the restriction they advocate.
Which of the following, if true, would most seriously call into question the educational theorist’s conclusion?
A. Some teachers give as homework assignments work of a kind that research suggests is most effective educationally when done in class.
Stacy: Totally irrelevant.
Anthony: Wow, you’re quick. I haven’t even finished reading the answers, but okay. I’m with you 100%.
B. For children younger than 12, regularly doing homework in the first years of school has no proven academic value, but many educators believe that it fosters self-discipline and time management.
C. Some homework assignments are related to free-time activities that children engage in, such as reading or hobbies.
D. A substantial proportion of schoolchildren under 12, particularly those in their first few years of school, have less than 10 minutes of homework assigned per night.
E. Some free-time activities teach children skills or information that they later find useful in their schoolwork.
Stacy: Okay, I have no clue where we’re going with this. My gut is saying C, but I have no idea.
Anthony: Well, let’s start with this. The 30 minutes that they gave was an average again, right?
Stacy: Yes.
Anthony: And so now we know that we should be very suspicious of that average. That average might hide people who have much more or much less homework than the average shows.
Stacy: Oh, so it would be D.
Anthony: Well, that’s interesting, because D seems to say that a lot of students have much less homework. Isn’t that the opposite?
Stacy: Which of the following would call into question… Right. But basically, okay, a lot of them have less than 10, but that means there’s still a portion of them that have way more than 30. And so those are the kids we should look at and say, actually, they should have less.
[22:06] Anthony: Exactly. That is exactly the perfect explanation of why D is the right answer. And by the way, this is an 800-level question. Fantastic.
Stacy: Yeah. And what I want to say is, if I didn’t have this lesson with you and tune into the fact that, oh, we’re looking at averages and this is the trick they play with averages, then I would have gone in another direction. But since I know it’s an averages thing, it makes you hone in on D.
Anthony: That’s wonderful to hear, and that’s exactly why I wanted to do this episode. I think there are things students can start to identify in some of these questions, so that when they see a certain setup, they can hone in. They can go, “Wait a minute, hold on. There’s an average here. That’s a little fishy.” They’re talking about some of the kids, right? Does anyone need a limit? The average is 30, but maybe there are some kids who have way more.
And by the way, the move you made was really smart, but I want to go back over it for anyone following along at home for whom that might have been pretty quick. What Stacy said, and she’s exactly right, is that if D tells us a lot of students are way below the average time, that implies there have to be a substantial number of other students who are significantly above the average time in order to balance that out. That’s simply how averages work. You can’t get an average of 30 with a bunch of people below 30 and no one above it.
So there’s a way in which the right answer here feels very backwards. Even with you, Stacy, when I questioned you, you went, “Oh, wait, no, hold on.” I’m glad you stuck with it, because I was playing a little bit of devil’s advocate. But I’m also showing you what gives a lot of people pause. They go, “Wait, I needed students with more homework. These are the students with less.” And that’s a trick the test makers really love to use. They love to give you an answer that is flipped from the way you’re expecting to read it.
The simple version is, you’re looking for “A is greater than B,” and they say “B is less than A.” Sounds different. Literally the same statement, right? The more complex version is that I’m giving you information about the low end, but it’s like a seesaw. When the average is fixed and there are more kids below the average… I guess it’s like a reverse seesaw. I don’t know, that’s not a thing. But you get the idea. When the low end goes up, the high end has to go up as well. So again, they’re giving you sort of the opposite of what you expected, but in the end, it’s exactly what you need.
[24:51] Stacy: Yes. No, I think that makes so much sense. I’m clearly so rusty on these tests, and it’s kind of fun to go back and quiz myself. But I see how your tips, just having that, change the way you look at the question. And I never claimed to be a test prep wizard. I’m good at the rest of the application.
Anthony: You just got an 800-level question, right? I don’t think anyone is doubting your cred. I think you’re just fine.
Anyway, the big point here: averages can hide subgroups. So when you see a question that involves an average, you need to take a closer look, because it’s possible that the right answer may point you to a subgroup that’s very different from that average. And that could be exactly the key point the problem is all about.
I think the upshot is that, in the end, this may have been an average episode, but that just means we’ve had some really great highs to balance out the lows.
Stacy: Yes, absolutely. Averages are interesting. Thank you, Anthony, for your brilliance and wisdom. By the way, our entire team at SBC hires Anthony for our own children. He is the man. Amazing results. If you’re struggling with your test scores, this is your guy.
Anthony: Thank you, Stacy.