Brian Friend and Mike Halpin propose a new method to determine who competes at each stage of the CrossFit Games season
Research provided by Mike Halpin Image Credit - Athlete’s Eye
It is no secret that CrossFit has made an effort this year to present its community with a system which is intended to create a more accurate representation of the competitive landscape within the sport of fitness.
It’s also no secret that from the onset I have been adamantly opposed to the system they put forth; not because it’s a bad idea (I’ve been doing something similar to this for years), but because it is not an accurate representation of the CURRENT competitive landscape.
This is not an article explaining what’s wrong with that system though, that has been done more than enough. Instead, this is my solution to the questions everyone’s been asking: if their system is flawed, what system would you use?
——-
This began as a hypothesis, but in the end we were pretty happy with what our first effort looked like. There are definitely questions that need to be answered in other areas of the administration of the CrossFit Games season, but we cannot do much about those for now. So, given the tests we’ve been given in the first two stages of the season, and the review process (also severely flawed) that has been carried out, let us explain our 15% threshold system for more accurately capturing what is actually happening at the elite level of the sport this year.
Note: This entire study is done based uniquely on the 2023 Open and Quarterfinals. The athletes competing would not have known about this study, and therefore may have put in varying levels of effort. The same can be said for the current Open to QFs to SFs process.
We acknowledge that because we are applying a study to a group of athletes who did not know we would be doing so, it does have the potential to skew the results. However, in changing as little as we could, we wanted to run a test to see what it would look like to use only this year’s field of athletes, compared to each other, to assess the competitive lay of the land.
THE OPEN
Every stage of the CrossFit season is unique, and yet, all the qualifying stages should also have one thing in common. They should be a vetting stage for the next round.
- The Open should be testing a baseline of strength, skills and capacity that will be necessary for Quarterfinals
- The Quarterfinals should be doing that for Semifinals
- And the Semifinals should be doing that for the Games
Whether that is actually something Adrian Bozman and his team consider is unclear. We saw a large percentage of athletes advance from the Open to Quarterfinals this year that were clearly (based on their Open performances) not qualified to be taking the tests presented in Quarterfinals.
So, we introduce the 15% Threshold: in order to qualify for Quarterfinals athletes must finish in the top 15% worldwide on EVERY Open test against the field of PARTICIPATING athletes.
WOMEN
For the Women they would have needed to meet these thresholds:
Minimum Work | Under 15% | |
---|---|---|
23.1 | 30 Cleans by 11:34 | |
23.2A | 5 Rounds + 2 burpee pull-ups | |
23.2B | 147lbs | |
23.3 | 8 strict HSPUs |
Here are the number of women who did so:
Region Summary | Starting | Qualified | % |
---|---|---|---|
NAE | 31,650 | 1,293 | 4% |
NAW | 23,011 | 961 | 4% |
EUR | 31,222 | 1,056 | 3% |
OCE | 7,574 | 311 | 4% |
ASA | 4,538 | 95 | 2% |
AFR | 2,352 | 63 | 3% |
SAM | 5,565 | 329 | 6% |
105,912 | 4,108 |
MEN
For the Men they would have needed to meet these thresholds:
Minimum Work | Under 15% | |
---|---|---|
23.1 | 8 RMU | |
23.2A | 5rds + 18 Burpee Pull-Ups | |
23.2B | 228 lbs | |
23.3 | 20 sHSPU + 50 DUs |
Here are the number of men who did so:
Region Summary | Starting | Qualified | % |
---|---|---|---|
NAE | 40,898 | 1,916 | 4.68% |
NAW | 29,941 | 1,356 | 4.53% |
EUR | 57,033 | 2,089 | 3.66% |
OCE | 9,369 | 473 | 5.05% |
ASA | 10,474 | 402 | 3.84% |
AFR | 4,144 | 147 | 3.55% |
SAM | 9,031 | 548 | 6.07% |
160,890 | 6,931 |
That means a total of 4,108 women and 6,931 men would have been ELIGIBLE to compete in Quarterfinals (you can see the breakdown by competitive region above the blue highlighted numbers in those columns).
Looking at what was required to meet the 15% minimum it makes sense considering the weights and skills required in the Quarterfinal stage as well:
- Particularly the ability to do strict handstand pushups (which only became more important and difficult in QFs)
- And the ability to thruster a minimum threshold that is far more in line with the required strength for the front squats and clean and jerks in QF workouts 1 and 3 respectively
THE QUARTERFINALS
Since the 15% threshold worked so well for vetting athletes in the Open, we decided to start there for Quarterfinals as well. This time we only took the athletes who qualified for Quarterfinals using our 15% thresholds and competed in Quarterfinals. Taking those performances we once again require athletes to perform in the top 15% of all five Quarterfinal tests against the field of participating athletes only. Knowing this stage has to be critical in terms of screening for the Semifinal stage, it’s imperative to measure against those who are actually doing the test, not just people eligible for it or who signed up but did not do it.
WOMEN
The minimum thresholds for 15% on each Quarterfinal test were:
Minimum Work | ||
---|---|---|
1 | 85 reps & 11:00 Tiebreak | |
2 | 5 rounds + 8 DB snatches | |
3 | 9:18 | |
4 | 2 rounds + 160-meter row | |
5 | 9:01 |
And the number of women by competitive region who met these thresholds:
Region | Count | % |
---|---|---|
North America West | 58 | 22.8% |
Europe | 75 | 29.5% |
North America East | 79 | 31.1% |
South America | 12 | 4.7% |
Asia | 4 | 1.6% |
Africa | 1 | 0.4% |
Oceania | 25 | 9.8% |
TOTAL | 254 |
MEN
The minimum thresholds for 15% on each Quarterfinal test were:
Minimum Work | Under 15% | |
---|---|---|
1 | 11:53 | |
2 | 5 rounds + 7 DB snatches | |
3 | 9:19 | |
4 | 2 rounds + 540-meter row | |
5 | 7:10 |
And the number of men by competitive region who met these thresholds:
Region | Count | % |
---|---|---|
North America West | 45 | 15.7% |
Europe | 109 | 38.1% |
North America East | 77 | 26.9% |
South America | 21 | 7.3% |
Asia | 6 | 2.1% |
Africa | 4 | 1.4% |
Oceania | 24 | 8.4% |
TOTAL | 286 |
And finally, we have an actual picture of what the Current state of competitive CrossFit, for this season, in the elite individual field looks like. There are a total of 254 women and 286 men who have proven they are capable of taking the Semifinal test. However, not necessarily all of them would; and this is slightly outside the scope of this article, but at this point we’d likely still cap all Semifinal fields at 60*.
*In general we are trying to change as little as possible from the current layout for the season.
GAMES SPOTS DISTRIBUTION
CrossFit Games organizers have talked about testing different size fields when using the D’Hondt method for distributing Games spots. But all of them were based on arbitrarily chosen numbers and using a WWR system which does not really capture what this year’s competitive field looks like.
That is why even though not all 254 and 286 will necessarily compete at Semifinals, those are the numbers we’ve chosen to use when allocating Games spots for the 2023 season.
Africa and Asia
In this model we determine that Africa and Asia do not have enough eligible participants to warrant hosting their own Semifinals; so we would propose a Africa/Asia hybrid Semifinal in the UAE to host the 10 men and 5 women combined from those two continents to participate in the Africa-Asia Semifinal for 2023.
Maintaining that every competitive region gets a minimum of 1 Games allocation, this is what the distribution of Games spots would look like for 2023:
WOMEN
Region | Region % of All Qualified | D’Hondt Allocation (1 min) | SF Spots |
---|---|---|---|
North America West | 22.83 | 9 | 58 |
Europe | 29.53 | 12 | 60 |
North America East | 31.10 | 12 | 60 |
South America | 4.72 | 2 | 12 |
Africa/Asia | 1.97 | 1 | 5 |
Oceania | 9.84 | 4 | 25 |
TOTAL | 100.00 | 40 | 220 |
MEN
Super Regions | Region % of All Qualified | D’Hondt Allocation (1 min) | Sf Spots |
---|---|---|---|
North America West | 15.73 | 6 | 45 |
Europe | 38.11 | 15 | 60 |
North America East | 26.92 | 10 | 60 |
South America | 7.34 | 3 | 21 |
Africa/Asia | 3.50 | 2 | 10 |
Oceania | 8.39 | 4 | 24 |
TOTAL | 100.00 | 40 | 220 |
The European Misperception
For those who have been following my work for years, you may recall that on multiple occasions I’ve suggested that there is no group in the world being slighted more than the women in Europe. And that was in fact true, at that time.
However, unlike the current WWR and SoF system that CrossFit is using, our system does not account for what used to be. Rather, it accounts for what actually is now.
And in the current competitive landscape, we see that it is in fact the European Men, not the European Women, who are not being adequately represented this season.
Based on their performance this season, they have earned 15 Games spots, one less than North American men (6 + 10).
The nice thing about this model is that it keeps up with the actual field of competitors year to year.
To Summarize:
We believe this model is the true representation of the strength of field for this year’s group of competitive athletes competing in the CrossFit Games Season.
- They were vetted through the Open and performed in the top 15% on ALL tests against all worldwide participants.
- They were vetted through the Quarterfinals and performed in the top 15% on ALL tests against all worldwide participants (who previously had proven themselves in the Open AND chose to participate in Quarterfinals).
- Once we established the group who, based on performance, are qualified to tackle the next stage of the test (Semifinals), we used them to distribute Games qualifying spots.
- We believe this will yield Semifinal fields that are composed of athletes who are actually capable of taking the tests.
- And ultimately a Games field that will be both a global representation of athletes, and an accurate depiction of where the best athletes this season are coming from.
Reader’s Contributions
We know that we pour alot into efforts like this, but we also know there are things we have the potential to overlook or not consider. If you feel there is something we have not considered in the production of this study please drop a comment below.
Brian C. Jackson
May 5, 2023 / at 7:22 pm
I understand your rationale, but isn’t the final result skewed by the size of the super region?
Europe and North America East are the biggest regions (most athlete dense). I don’t think taking the 37th qualified athlete in the Europe region would compare accurately to the 15th place athlete in the North American East, although fitness of those athletes could be equivocal.
CF’s model is taking the best of the best in my opinion.
Keep this up I love how your keeping us on our toes!
Thanks!
Brian
May 8, 2023 / at 3:03 pm
I’m not sure I understand. Al 286/254 athletes did well enough against the field of competition to prove themselves ready for the Semifinal test. We did not factor anything with regards to where they are from up until that point.
At some point there is going to be a live qualifier, and logistically speaking, it makes sense at that point to have athletes in similar regions of the world compete against each other for ease of travel etc.
From there all we did was assess where the athlete’s who proved themselves good enough were from and applied a Games allocation distribution method to that field of athletes.
Brian
May 8, 2023 / at 3:05 pm
The one thing I can definitely say is that CF’s model does not find the 40 fittest men and women in the world to compete at the Games.
J
May 10, 2023 / at 5:31 pm
Cool idea. Two questions and an idea:
Did any Top 100 2023 QF finisher, women or men, change using your 15% SoF formula? I like the idea of “minimum fitness”, and it prob has some impact in the Open, but I’d be surprised if many Top 100 QF finishers don’t hit the bar and even more surprised if anyone in the Top 50 doesn’t.
Would it be easier, and just about as good/accurate, if you used the distribution of the Top 100 QF finishers (not WWR; QF finish) to compute strength of field for Games spots?15% is a lot of spots, and your formula rewards the “15th” (lowest) percent finishers just as much as the “1th” (highest) percent. It doesn’t feel like a region with lots of finishers in the 10-15% percentile range should get extra spots vs. a region with lots of finishers in the top 5%.
Idea for Games spots: standardize the SF tests (I think that’s happening in 2023). Every SF region winner gets a spot, and the remaining spots go to the Top 50 SF finishers regardless of region. CF could add a handful of Games spots for developing regions.
Brian
May 10, 2023 / at 11:13 pm
Appreciate the thoughtfulness J.
I think there is definitely room to re-evaluate the best method for assessing the strength of field rather than counting every spot as one. We spoke about this on this week’s Talking Elite Fitness episode a bit in fact.
I don’t think I did the exact study you’re asking for, but there are athletes in the top 100 of CrossFit’s WWR who would not have qualified for SFs in this model- we spoke about that on an episode of the Sevan Podcast which focused on this article.
Joe A
July 9, 2023 / at 4:42 am
Hi I love the idea and content and think it has a lot of potential but I have a couple of questions that arose.
Firstly, what about the case of a specialist who performs poorly in an early event and fails to reach the top 15%. For example, let’s say an endurance-based athlete fails to place in the top 15% of a strength-based event. I understand that this athlete should not advance due to their weaknesses and limitations, but there is a concern that they might quit (either literally or mentally) and not compete in the workouts that follow. In order to test the overall fitness of an individual, it is likely the later workouts would align with their strengths. With them withdrawing or mentally checking out, this would lower the bar of the top 15% on the later workouts in the competition and skew the results to favor the earlier events oppose to the later ones. Essentially, the order of the events may favor a certain type of athlete, leading to biased results.
Secondly, I’m not sure about the familiarity with the ELO ranking concept. Originally developed as a chess power ranking system, ELO has been successfully applied in various sports to rank opponents based on head-to-head matchups. Has any research been conducted to explore the possibility of implementing the ELO ranking system in order to improve the efficiency of selecting the fittest athletes?
I love the idea and appreciate the research and thoroughness and am looking forward to more content.
Brian
July 11, 2023 / at 4:30 pm
By no means do we think this is a perfect system. If anything we would hope that it’s a launching point for a conversation. Additionally, it forces the programmer(s) to consider these variable not just in writing the workouts, but being thoughtful about the order etc.
I am familiar with the ELO rating for chess, haven’t considered that before as a carry over to CrossFit, but it is intriguing!