Here are relay DQ's by year and gender.
1998 Girls - 3.6% (3/84)
1998 Boys - 3.6% (3/84)
1998 Total - 3.6% (6/168)
2000 Girls - 1.8% (3/168)
2000 Boys - 0.6% (1/168)
2000 Total - 1.2% (4/336)
2002 Girls - 4.2% (8/190)
2002 Boys - 2.6% (5/189)
2002 Total - 3.4% (13/379)
2003 Girls - 1.6% (3/191)
2003 Boys - 2.1% (4/190)
2003 Total - 1.8% (7/381)
2004 Girls - 1.6% (3/190)
2004 Boys - 2.1% (4/192)
2004 Total - 1.8% (7/382)
2005 Girls - 1.1% (2/190)
2005 Boys - 3.7% (7/191)
2005 Total - 2.4% (9/381)
2006 Girls - 3.1% (6/191)
2006 Boys - 2.6% (5/191)
2006 Total - 2.9% (11/382)
2007 Girls - 2.6% (5/189)
2007 Boys - 4.2% (8/189)
2007 Total - 3.4% (13/378)
2008 Girls - 2.1% (4/190)
2008 Boys - 4.3% (8/188)
2008 Total - 3.2% (12/378)
2009 Girls - 2.1% (4/191)
2009 Boys - 3.7% (7/189)
2009 Total - 2.9% (11/380)
2010 Girls - 4.9% (9/185)
2010 Boys - 5.4% (10/186)
2010 Total - 5.1% (19/371)
2011 Girls - 7.0% (13/187)
2011 Boys - 2.6% (5/190)
2011 Total - 4.8% (18/377)
This showed a definite "spike" in relay DQ's since machines became more important than humans.
Still, after I'd crunched all the numbers for the "project", the second-guessing began.
Older results didn't tell why a relay was DQ'ed. Could it have been due to stroke, turn, or finish violations? Was the call a false start on a lead-off swimmer and not a relay take-off violation?
Did I just miss my exit?
Rather than go crazy with all the variables, I decided to use only free relays for this "study" and not worry about lead-off DQ's. How convenient, right?!?!
Using just free relays, here are DQ percentages by year and gender.
1998 Girls - 3.6% (2/56)
1998 Boys - 3.6% (2/56)
1998 Total - 3.6% (4/112)
2000 Girls - 0.0% (0/112)
2000 Boys - 0.0% (0/112)
2000 Total - 0.0% (0/224)
2002 Girls - 3.9% (5/127)
2002 Boys - 2.4% (3/126)
2002 Total - 3.2% (8/253)
2003 Girls - 0.8% (1/128)
2003 Boys - 1.6% (2/126)
2003 Total - 1.2% (3/254)
2004 Girls - 1.6% (2/127)
2004 Boys - 1.6% (2/128)
2004 Total - 1.6% (4/255)
2005 Girls - 0.0% (0/128)
2005 Boys - 3.1% (4/128)
2005 Total - 1.6% (4/256)
2006 Girls - 1.6% (2/127)
2006 Boys - 3.1% (4/127)
2006 Total - 2.4% (6/254)
2007 Girls - 1.6% (2/127)
2007 Boys - 4.0% (5/126)
2007 Total - 2.8% (7/253)
2008 Girls - 2.4% (3/127)
2008 Boys - 2.4% (3/127)
2008 Total - 2.4% (6/254)
2009 Girls - 1.6% (2/128)
2009 Boys - 3.2% (4/126)
2009 Total - 2.4% (6/254)
2010 Girls - 2.4% (3/126)
2010 Boys - 4.0% (5/125)
2010 Total - 3.2% (8/251)
2011 Girls - 5.6% (7/126)
2011 Boys - 3.1% (4/127)
2011 Total - 4.3% (11/253)
That made things "level out" a bit, but we still saw a jump in "jumps" in 2010 and 2011. From 1998 through 2009, free relay DQ's happened at a rate of 2.0%. In 2010 & 2011, that rate shot up to 3.8% - nearly double!
Finally, just for fun, here's the breakdown for medley relay DQ's.
1998 Girls - 3.6% (1/28)
1998 Boys - 3.6% (1/28)
1998 Total - 3.6% (2/56)
2000 Girls - 5.4% (3/56)
2000 Boys - 1.8% (1/56)
2000 Total - 3.6% (4/112)
2002 Girls - 4.8% (3/63)
2002 Boys - 3.2% (2/63)
2002 Total - 4.0% (5/126)
2003 Girls - 3.2% (2/63)
2003 Boys - 3.1% (2/64)
2003 Total - 3.1% (4/127)
2004 Girls - 1.6% (1/63)
2004 Boys - 3.1% (2/64)
2004 Total - 2.4% (3/127)
2005 Girls - 3.2% (2/62)
2005 Boys - 4.8% (3/63)
2005 Total - 4.0% (5/125)
2006 Girls - 6.3% (4/64)
2006 Boys - 1.6% (1/64)
2006 Total - 3.9% (5/125)
2007 Girls - 4.8% (3/62)
2007 Boys - 4.8% (3/63)
2007 Total - 4.8% (6/125)
2008 Girls - 1.6% (1/63)
2008 Boys - 8.2% (5/61)
2008 Total - 4.8% (6/124)
2009 Girls - 3.2% (2/63)
2009 Boys - 4.8% (3/63)
2009 Total - 4.0% (5/126)
2010 Girls - 10.2% (6/59)
2010 Boys - 8.2% (5/61)
2010 Total - 9.2% (11/120)
2011 Girls - 9.8% (6/61)
2011 Boys - 1.6% (1/63)
2011 Total - 5.6% (7/124)
Okay, folks. Why did the percentage of medley relay DQ's increase so markedly in 2010 and 2011? Hint: The DQ rate from 1998 through 2009 was 3.8%. In 2010 & 2011, that rate jumped to 7.4% - nearly double! Seeing a pattern?
I say that no, medley relays didn't skew the data. Seems DQ's of both types of relays follow a similar pattern. DQ rates almost doubled for both medleys and free relays when machines were allowed to overrule humans.
Finally, what did I learn from all this?
I learned that we need a whole lot more data (this is really just a tiny portion of what we'd need to "prove" anything, right?) and someone else to fool around with it...
From a pure statistics perspective - it is clear that a change occurred for the 2010 and 2011 results. So what changed? Only thing that makes sense... automatic relay judging.
ReplyDeleteDING DING DING!!!!!
ReplyDeleteWe have a winner!
Now, let's either:
a) use a video back-up, or
b) leave the platforms in the equipment room and go back to dual confirmation.
Awesome effort, you deserve an extra helping of mandarin oranges.
ReplyDeleteGuess those students got worksheets today, huh, or did they assist in this math project, lol?
Really, way to go, kudos and kumquats!
If the "Lids" are the problem, would you expect to see a spike in DQ rates for false starts on all events?
ReplyDeleteIs there someone at the UIL or the NFHS whose job it is to continually review this type of data and look for changes or oddities? In a manufacturing environment, these would be the quality engineers or the statisticians. When a new rule is put in place, there should be metrics to see if any change occurs. Nearly all high performing businesses do that. How does someone know that a new ad campaign works - they measure it. How does someone know that a new teaching method works - they test the students.
ReplyDeleteWhen the UIL (or any organization for that matter -- think red light cameras :)) puts a new rule in place, they need to ask - how will I know this rule is effective and how will I know if this rule is causing any unintended consequences. Metrics should then be put in place to evaluate the effectiveness of the new rule. Common sense...
The 10:20pm Anonymous commenter deserves a medal. Great thinking, great argument, great candidate for another sweep-under-the-rug or "we'll take it under advisement" response at the UIL office, based on what we've seen so far.
ReplyDeleteNo disrespect intended. Just sayin'.
Great job with this whole unfortunate situation Coach Button. I really feel for the athletes affected by the unintended consequences of the rule removing officials from having a say in DQ's. The equipment malfunctioning and no one able or willing to step in and figure it out, is disappointing. Everyone involved should be putting the swimmers interests first. In answer to anonymous, no there is no one overseeing any effect a new rule change has on the competition.
ReplyDelete7% for girls versus 2.6% for boys. It seems that this rule has hurt girls and their relays at the state meet at a disproportionate amount that is not statistically possible if the equipment was working fairly and equally for both genders. Why is that? This is something I hope the UIL will also care enough to investigate.
ReplyDeleteTo Anonymous 8:54PM - The false start must be initiated by the starter-- at least for now, still a real human being.
ReplyDeleteBased on my experience as a timing operator, I have seen just as many DQ "saves" as DQ's with the take off pads. These don't get recognized,contested or communicated to the coaches.
ReplyDeleteIsn't it the whole point of RJP's to catch false starts that are missed by humans?
ReplyDelete'Based on my experience as a timing operator, I have seen just as many DQ "saves" as DQ's with the take off pads. These don't get recognized,contested or communicated to the coaches.'
ReplyDeletegreat point. we know there are some relay take-offs that are illegal. if both officials didn't see it, they went uncalled.
'Isn't it the whole point of RJP's to catch false starts that are missed by humans?'
again, great point. it's the whole point of using the technology. unfortunately, our high school rules kept us from undoing errors that have now shown up as after-the-fact evidence.
to keep this from happening again, let's use video back-up in case of a malfunction. if we're using technology to disqualify kids, don't you think we owe it to them to use technology to let them off the hook?
if we're not willing to give our kids the benefit of the doubt, what message are we sending them? have we gone 'gotcha' with a sort of 'zero tolerance' policy?
Don't forget too - that if the rule makers don't want to add video or secondary proof, then they at least need to indicate that any time a pad fails during a race, then any resulting technology called DQ is not approved/vacated/reversed. Oh wait, they can do that now can't they...
ReplyDelete