Saturday, January 24, 2015

CFG Analysis Podcast - Pilot Episode

I'm trying something new here at CFG Analysis.  In addition to the articles I'm posting, I'm adding a podcast to complement what I do with my written work.  These podcasts are going to be much more informal than what I typically do on the blog, and eventually I hope to chat with some athletes, coaches and everyday CrossFitters.

Today's Pilot Episode is just me, discussing the background on this site, how it came to be and who exactly I am.

Stay tuned for our next podcast here in a couple of weeks, and bear with me as I get the hang of this podcast thing.

The Pilot Podcast and all future episodes can be found here: http://cfganalysis.libsyn.com  or below.  I'm currently working on getting it set up on iTunes.

Sunday, January 4, 2015

How Many Years Do CrossFit Athletes Last in the Open?

What you need to know from this post:
  • About 50% of athletes who compete in the Open on year will continue to the following season.
  • Athletes who have competed for multiple years have a higher likelihood of returning the next season than athletes who are competing in their first season.
  • Based on data from the past four years, approximately 20% of athletes who start competing in the Open will still be competing in their fourth season. I estimate that approximately 10% will still be competing in their seventh season.
  • The higher an athlete ranks in the Open, the higher the probability that they will return the next season.
As I mentioned a few weeks ago, I will be missing the Open for the first time this season.  I had arthroscopic hip surgery to repair a torn labrum.  Without getting into too much detail, the injury was a chronic wear-and-tear type of injury that wasn't a direct result of CrossFit, per se, but rather the fact that I had been so active for the past 10-15 years.  I had a hip impingement that I was born with that put me at risk for this type of injury.  I hope to return to CrossFit eventually, but I won't be ready to compete by late February.

So although my injury isn't directly attributable to CrossFit, it's forced me to face the fact that competing and training at the level I had been is not always easy to sustain.  For me, it was my hip.  For others, it is a shoulder or a knee.  This is true in any sport, and CrossFit is not immune.  This is not a commentary on whether CrossFit as a training methodology is dangerous.  That's a third rail I don't intend to touch.  My point is simply that injuries are inevitable when competing in any serious sport.

With an assist from my wife, who came up with the idea for this post, I decided to try to answer the the question: how long do CrossFit athletes tend to compete?  I'm not talking just about the Rich Fronings of the world, but the everyday athlete who signs up for the Open with no hope of even sniffing Regionals.  We see the Open growing in size each year, but that doesn't necessarily mean athletes are continuing on for multiple years; we could just be replacing the vast majority of athletes one year with an even bigger crop of newbies.

Of course there are multiple reasons an athlete won't compete: injury, lack of interest, disappointment from poor performance in a previous season, work commitments, etc.  With the data I have available, I can't identify which factors are most important, but I can get a pretty good idea of the rate at which athletes are dropping off.

Using the Open data from 2011-2014, I looked at how many athletes continued on each year and what attributes about the athlete (prior Open experience, prior Open finish, age, weight, height) may have an influence on their likelihood to continue.  Because I only have athlete names without any other identifier, this analysis is limited to "truly unique names," which are those that never appeared more than once in any year.  I also focused my analysis on men, because women's names are frequently changed due to marriage or divorce, making it hard to track which athletes truly dropped out and which simply changed name.*

The way I am defining it, an athlete successfully survives from one year to the next if he completes all the Open events in both years.  Failing to complete all the Open events in a given year is counted the same as not competing.  Keep that in mind, since typically about 30% of the Open field from week 1 is gone by the end of the Open.  For purposes of this analysis, those athletes never even competed.  I don't care that HQ still got their money.

Finally, in this study, once you're out, you're out.  If an athlete skips a year, I ignore whether or not they returned the following year.  This makes things much cleaner for the analysis.  In case you are wondering, only about 15% of athletes skip a year and return in a subsequent year (I hope to be one of those 15%!).

OK, so let's get to the results.  First, the simplest way to look at this is to evaluate the athletes that started in 2011 and see how many were left in each subsequent year.  Let's take a look at those results below.


You can see that nearly 60% of athletes survived to Year 2 and nearly 30% were after Year 4.  The issue with this analysis is that it ignores the athletes who started after 2011, which is a huge chunk of the current athlete pool.  The type of athlete who is competing today may be characteristically different than those who started in 2011, and we want to capture that.

The next chart estimates a survival curve for today's athlete population using only the most recent information.  To get the survival rate from Year 1 to Year 2, I looked at athletes who first competed in 2013 and see how many returned in 2014.  To get the survival rate from Year 2 to Year 3, I looked at athletes who competed in 2012 and 2013, then found out how many of those returned in 2014.  To get the survival rate from Year 3 to Year 4, I looked at athletes who competed in 2011-2013, then found out how many of those returned in 2014.

To get the cumulative survival rate for year 3, I multiplied the survival rates for years 1-2 and years 2-3.  To get the cumulative survival rate for year 4, I multiplied the survival rates for years 1-2, years 2-3 and years 3-4.  I then took it one step further, estimating survival rates in years 5-7 based on the rates for the first 4 years.  These are obviously estimates, as we don't have enough data to know the true likelihoods beyond year 4.


Here we see the survival rates are much lower than we previously observed.  About 50% of athletes remain after year 2, about 20% are left after year 4 and I'm estimating only about 10% will be left after year 7.  Clearly the majority of athletes don't make it too many years in this sport, but there is still a decent chunk of the population that sticks with it for years.

However, what's not obvious in the chart is that the chances that an athlete continues in the following year increases with each subsequent year of participation.  Below are the year-to-year survival rates:
  • Year 1-2: 47%
  • Year 2-3: 61%
  • Year 3-4: 72%
This is good news in my opinion.  We see that athletes who stick with it beyond the initial year are not likely to "burn out" the longer they compete.  Once an athlete is sufficiently invested, they are pretty likely to keep at it.**

In total, when you combine first-, second- and third-year competitors, about 52% of the field returns in the following season.  One offshoot of this is that if we consider how fast the Open has been expanding, we know that it must be largely made up of first-time competitors.  In order to maintain the size of the Open field from one year to the next, we need a lot of new athletes each season.  If there were 200,000 total athletes last season, we likely need about 100,000 new athletes to enter the field in 2015 simply to maintain the same size field as before.

The last piece of this analysis was to try to identify other factors (aside from number of years of prior experience) that made certain athletes more or less likely to continue in subsequent years.  For this, I limited the data again to only 2013 and 2014, then further limited the data to athletes who submitted a height and weight (I used the 2013 height/weight for all athletes).

Using a logistic regression model, I looked to see if 2013 percentile rank, age, height or weight had statistically significant impact on the likelihood of returning in 2014.  As it turned out, all but height had a statistically significant impact (p-value less than .01 for percentile rank, age and weight).  However, in my opinion, we can basically ignore weight because the predicted probability of returning did not vary a whole lot (only about 4% higher probability at 180 lbs. vs. 220 lbs.).  

The big key was the 2013 percentile rank.  As you might expect, athletes who finished near the top of the rankings had a much higher likelihood of returning.  Holding all other items at their mean, the predicted probability of returning was 74% for an athlete finishing in the 1st percentile, compared with 55% at the 50th percentile and 35% at the 99th percentile***.  If you aren't convinced by those somewhat opaque predictions from the logistic regression, the chart below shows the observed percentage of athletes who returned, by 2013 percentile rank.




As you can see, the pattern is very evident.  I'm not sure this will come as a surprise to anyone, but it's always nice to see your intuition confirmed in the data.

Age was an interesting factor.  There are two things going on here:
  1. Without controlling for the 2013 percentile rank, it appeared that age had basically no effect.  
  2. Older athletes tend to have worse rankings than younger athletes in general. As we just showed, athletes with lower rankings have lower persistency.
What happens here is that the logistic regression showed that all other things being equal, older athletes actually have a higher likelihood of returning than younger athletes. If we hold the other items at their mean, the predicted probability of returning was 51% for a 20-year-old and 65% for a 50-year-old.

Of course, the big question moving forward is how things will change with the changes made to the Open in 2015.  The addition of a scaled division is likely to siphon off some athletes who had previously competed in the Rx'd division, and it's possible that with only 20 Regional invitations in each region (as compared to 48 in 2013-2014 and 60 in 2011-2012), some additional athletes will drop out.  I also have to believe that the overall participation in the Open will not continue to expand the way it has since 2011, where it has roughly doubled each season.

There is no way to know the answers to these questions right now, but understanding what has happened in the past will certainly help us understand the impact of these changes in the future.

[Thanks a lot to Andrew Havko, Michael Girdley and Jeff King for pulling this data for me and/or making it publicly available]

*It appears that overall, the survival rate for women is very similar to that of men.  Initially, it appears about 4% lower, but using some very rough data for the marriage and divorce rates, that discrepancy could very easily be attributable entirely to name changes.

**For my estimates beyond year 4, I assumed this would continue to flatten out, so I used 77% for year 4-5, 79% for year 5-6 and 81% for year 6-7.

***These predicted probabilities are all probably about 4% too high.  That's because the subset of athletes used for the logistic regression was limited to those that submitted a legitimate height and weight.  In general, these athletes have slightly higher finishes and are slightly more likely to return than the average athlete.

Saturday, December 13, 2014

Understanding Year-to-Year Improvement in the Open

In this and most future posts, I'll start by giving you a very brief summary of the key takeaways from the post.

What you need to know from this post:

  • On average, athletes that compete in the Open in multiple years improve their rank percentile in the second year, but their absolute ranking declines.
  • The more years an athlete competes in the Open, the less they improve their percentile ranking in subsequent years, although the average improvement is still positive. 
  • There is pretty strong evidence that if the Open has a higher load-based emphasis on lifting (LBEL), this will favor taller and heavier athletes. If the Open has a lower LBEL, this will favor smaller and lighter athletes.
  • It is unclear how age is related to an athlete's percentile ranking improvement from year-to-year.

Today I want to get into a topic that has interested me for some time.  Many readers of this site have been competing in the Open for several years and likely have used their performance each year to judge how much their fitness has improved from the past year.  There are two underlying assumptions that we use here:
  1. The Open is a pretty good test of overall fitness;
  2. Each year of the Open is a relatively similar test of fitness, compared to other years.
I'm going to leave the first assumption unchallenged today, although there could certainly be debate about that.  But let's assume that the Open is indeed a good test of fitness.  What I will try to do today, using data from 2011-2014, is try to test the second assumption and get a feel for how much impact, if any, variations in the programming might have from year-to-year.  Along the way, I'll also look for other interesting observations about how athletes are improving across multiple years in the Open.

Before we get into the results, here's a quick background on the data I'm using and my basic methodology:
  • For both men and women, I started with the Open results for athletes under the age of 55 (meaning no scaling in the Open).
  • I removed any athletes whose first/last name combination was not unique.  This is due to the fact that my data does not have any other identifiers for each athlete.  Since there are something like 9 or 10 Ben Smith's, I just threw them all out.
  • I removed all athletes that did not complete all five events.
Next, I split up the analysis into six cohorts: 2013-to-2014 male, 2013-to-2014 female, 2012-to-2013 male, 2012-to-2013 female, 2011-to-2012 male and 2011-2012 female (an athlete could be multiple cohorts).  For each section, I identified athletes that competed in both years.  For all athletes that submitted it, I also mapped on age, height and weight information from 2013 (except for the 2011-2012 cohorts, in which case I used the 2012 information).  I had to make the simplifying assumption that an athlete's weight did not change from year-to-year, which is probably not true for some athletes.

OK, with the background out of the way, let's move onto the findings.  The first thing I wanted to know was just how much improvement athletes were making from year-to-year.  The initial results might surprise you:


On average, athletes who continued from one year to the next actually finished lower in the second year than in the first.  In fact, between 60-80% of athletes had a lower rank in the subsequent year across all six cohorts.  

So what gives?  Well, the key here is that the field has been expanding, nearly doubling in size each year.  The easy way to account for that is to look at the change in an athlete's percentile rank from year-to-year.  If an athlete is 5,000th out of 50,000 in 2013 and 8,000th out of 100,000 in 2014, then the percentile rank actually shows an increase of 1% (5% to 4%), despite the 3,000-spot drop in absolute rank.


Now we see that in general, athletes are improving year-to-year, although it became a bit more difficult each year.  Approximately 89% of athletes improved their percentile rank from 2011-2012, compared to 80% from 2012-to-2013 and 71% from 2013-to-2014.  

Each year, we have more and more athletes who have been competing for several years.  Most of us who have been CrossFitting for a long time know that making incremental improvements becomes harder and harder (thought not impossible) as the years go on.  For evidence of this, I looked at the average percentile improvement from 2013-to-2014 of athletes who also competed in 2012 vs. those who did not.*


These numbers make it fairly clear that the amount of past Open experience is a factor in how much improvement athletes make from year-to-year in the Open.  But how about other variables, such as height, weight or age?  This is where things get a little tricky.

First, let's look at age.  The three charts below show the average improvement by age for males (orange) and females (blue) in each time period.




From these charts, we see that there is not a simple answer here.  In two cases (male 2012-2013 and female 2013-2014), improvements were generally higher at older ages, but in the other four cases, the reverse was true.  Across the four cohorts, the correlation between age and percentile rank improvement ranged from -12% to +14%.  Unfortunately, the results are not consistent by gender or by year, in which case we might be able to make some sort of generalization about what these results mean.  For now, I will simply conclude that there is no clear relationship between age and year-to-year improvement in the Open.

How about height and weight**?  Well, again, the results were mixed, but in this case, the mix of results might actually be able to provide some insight.  Let's focus on weight for now.  Four of the six cohorts did show a clear linear relationship between weight and percentile rank improvement, but the other two did not.  Here are charts showing the relationships for those cohorts.


From 2012-to-2013 for females (third chart), it appears that heavier athletes tended to show more improvement than lighter female athletes.  In the other three cohorts shown, heavier athletes tended to show less improvement than lighter athletes.  

Another way to look at this is by examining the correlation between and percentile rank improvement  in each cohort.  A positive correlation means that higher weights tend to have higher percentile rank improvements; a negative correlation means that higher weights tend to have lower percentile rank improvements.  The correlations for the four cohorts shown above tell the same story: -11% correlation for 2011-2012 males, -15% correlation for 2011-2012 females, +8% correlation for 2012-2013 females and -6% correlation for 2013-2014 males.

What could be the reason for these results? 

For each of the Opens, I've evaluated the programming using a few metrics.  One that I reference quite frequently is the load-based emphasis on lifting (LBEL).  This metric attempts to quantify how "heavy" a CrossFit competition is based on the portion of the competition that was made up of lifts (as opposed to bodyweight movements), as well as how "heavy" those movements were.  After seeing the charts above, I went back and looked at the LBEL in each of the years for both males and females.  What I found was that in situations where there was a strong negative correlation between weight and improvement, the LBEL decreased significantly, and in situations where there was a strong positive correlation between weight and improvement, the LBEL increased significantly.

The chart below shows the following for each cohort:
  • Percentage change in LBEL;
  • Correlation between weight and improvement in percentile rank;
  • Correlation between height and improvement in percentile rank; and
  • Correlation between age and improvement in percentile rank.

While we still can't seem to tell much about the relationship between age and improvement, this chart above does seem to clearly indicate that a higher LBEL benefits larger athletes and a lower LBEL benefits smaller athletes.  Note that the color scales for the percent change in LBEL mirror the weight correlation almost exactly.

While this my seem intuitive, I think it is a very important result to lend credibility to the LBEL metric.  It is still my belief that LBEL is not a particularly useful metric when looking at individual events, but when used to evaluate a multi-event competition such as the Open or Regionals, I think it is very useful to help us understand the type of athletes that might benefit from the programming.  Keep in mind, of course, that these correlations are relatively small, so there are plenty of other factors that determine how much an athlete will improve from year-to-year.

Obviously, none of this is meant to de-emphasize the importance of training in determining how much an athlete will improve from year-to-year.  Rather, this can help us understand all the factors that might be impacting an athlete's improvement, which can in turn help evaluate how successful all that training really was.

[Thanks a lot to Andrew Havko, Michael Girdley and Jeff King for pulling this data for me and/or making it publicly available]

*I did also take a look briefly at the 2013-2014 improvement for athletes who competed back in 2011.  As expected, they were slightly lower than those who just competed in 2012.  However, they did still show a positive improvement in their percentile rank, on average.

**Many athletes did not submit their height or weight (or typed in something ridiculous, like 1,000 pounds).  Any time I looked at correlations between weight/height and percentile rank improvement, these are based only on the subset of athletes that reported a reasonable height and weight.  This ranged from about 50%-80% of the field (women generally reported less often).


Monday, November 24, 2014

What to Expect from the 2015 Open

So I'm a little late on my annual "What to Expect" post this year, but you've still got about 3 months left before 15.1 is announced.  That means there's still plenty of time to fine-tune your game, work on your weaknesses and get yourself as prepared as possible for this year's Open.  To help you do that, this post gives you an overview of the type of programming you should expect to face.

For those familiar with my past "What to Expect" posts, nothing here should come as a major shock to you.  But I have updated the numbers based on what happened last year, and I've also got some thoughts on what might change this year due to the changes mentioned by Dave Castro following the CrossFit Invitational a few weeks ago.  For those of you who are new, this should be a good primer on what the first stage of the CrossFit Games competition is all about.  To be sure, it is quite different from what you may have seen on ESPN (hint: you probably won't be pushing any sleds or doing 315-lb. squat cleans for time).

The way this post will be laid out is the same as last year.  I'll pose some questions that many newbies might have, and I'll answer them as best I can based on the information we have from the 2011-2014 Opens.  As I have noted in the past, I won't be covering how you might want to train for the Open in this post, but rather what you should be training for.

Let's get started.


What movements will I need to do?

Fantastic question.  I like where your head is at.  Although the CrossFit Games is all about preparing for the "unknown and unknowable" (do they still use that tagline?), you can expect to face a small subset of movements in the Open.  By my count, there have been 52 different movements tested at the CrossFit Games finals, there have only been 15 tested in the Open.  The chart below shows how much value each of these movements has been given in the past four years, along with the best estimate of what might be to come this year.  All of the "best estimates" in this post give more weight to recent years, but they are influenced to some extent by all years dating back to 2011.


Last year's open had a little wider spread of movements than in the past, but still, you can see that historically, there are a few movements that are tested heavily again and again.  This year, you can expect about 60% of the points to come from six movements: snatch, burpee, thruster, pull-up, double-under and box jump.  Just in those six alone, you can see that there is a jumping theme.  All but the pull-up and thruster involve some sort of jump or explosive hip movement (and you could argue the thruster does, too, especially at higher weights).

It's also worth noting (for all you home-gym folks) that you may need a Concept 2 rower available to be able to compete in the Open.  From 2011-2013, nothing was required other than a barbell, weights, rings, a box and a pull-up bar, but in 2014 there was a workout that involved rowing.  This came as a surprise to many, but now that it's been done once, don't be caught off-guard if it happens again.

In the previous chart, there was also a "Subcategory" listed for each lift (I know a wall ball isn't actually KB or DB, but I put it in there because it's a lift that doesn't use a barbell). Grouping movements by the subcategory can be useful, because it reduces some of the noise and gives us an idea of the type of movements that will be used. For instance, cleans haven't actually been used that much in the Open the past three years, but I wouldn't recommend skipping them in training - the movement pattern is similar to that of a snatch, which is highly valued. Below is a table similar to the one above, but looking at subcategories instead of specific movements.



The big theme here has not changed since 2011.  The focus of the Open is on two things: Olympic-style lifts and basic bodyweight gymnastics. This is partly due to equipment restrictions in the Open, but partly due to the fact that HQ seems to really value those two types of movements when making the first cut of athletes.

For those wondering why I have a category for "Uncommon CrossFit Movements" that has 0% value each year, it's because that category contains things like sled pushes, sandbag carries and swimming that have appeared at the Games but are rarely seen at other levels of competition.


How heavy will the lifts be?

Two years ago, I came up with the concept of average relative weights last year as a way to understand this topic a bit better. There is more background in this post from 2013, but here's the concept: depending on the movement, a certain weight may be heavy, medium or light, so I have normalized the weights prescribed on each workout so that we can get a fairer indication of how "heavy" the lift was. After looking at the normalized loads that were prescribed in the past three years, I applied the average relative weight we've seen to the various lifts to show the average expected weights in this year's Open.


The above graph is useful, particularly for new competitors who'd like to get a feel for what they will have to handle in the Open workouts.  However, there are some other factors to consider:
  • These are only averages. The "heaviest" load required in the Open was a 165-lb. clean and jerk (squat clean and jerk technically), which is roughly equivalent to a 130-lb. snatch, a 290-lb. deadlift or a 145-lb. overhead squat. Last year, I said that HQ would likely never require something that heavy again, but with the scaled division being added this year, I actually think it is possible (but unlikely) to see something that heavy required in an Open workout this year.
  • Once in the each of the past three years, HQ has programmed workouts where the weight starts light (to allow everyone to participate) but gets progressively heavier.  Expect this to happen again in 2015.  For those looking to make the Regionals, you'll likely need to be able to comfortably move weights that are about 75% heavier than the loads shown above (for instance, 165-lb. snatch for men).
  • From 2011-2013, HQ never really went heavy on a "heavy" lift (like the deadlift), rather going with heavier loads on the Olympic lifts.  Last year in 13.3 (deadlift/box jump), they showed that you need to have plenty of weights on hand.  Anyone qualifying for Regionals would have needed to handle plenty of deadlifts at 315 for men or 225 for women.
The chart below illustrates the distribution of weights that have been used in the past.  For workouts where the weights got progressively heavier, the weight used for this chart was the average that would be lifted for a typical Regional-level athlete.  Above each bar on the chart, I've given some examples of the type of movement/loading combinations that would fall into that range.



What types of WODs will be programmed? Will there be any ridiculous chippers like 121201?

First of all, there will not be anything like that workout.  There should never be anything like that.  If you are a gym owner, do not program 20 different movements into one workout.  Please.

Anyway, HQ has made it clear that they believe couplets and triplets are the bread-and-butter, so I'm sorry if you're really awesome at "Filthy Fifty," because it's probably not getting programmed.  We did see a little more variety last year with 14.4 and 14.5, but still, it's going to be pretty straightforward.

In the past four years, we've seen 21 workouts:
  • 20 of them included either one, two or three movements.  One involved five movements.
  • All were metcons (no max-effort lifts)
  • 18 had a fixed time limit, and 2 had a time limit that could be extended if you completed a certain amount of work in the first time limit.  Only one has had a required amount of work.
  • All fixed-time workouts have been between 4 and 20 minutes.  Anyone competing at Regionals would have finished the "for time" workout in less than 15 minutes.
Is it impossible that a max-effort workout will be used this year?  No, not after seeing a few of those in the team series.  But I still wouldn't count on it.  Work on your metcons if you want to qualify for regionals.

The chart below helps to get a sense visually of what combinations of loading, duration and number of movements we have seen in the Open.  Each ball represents a workout, and the size of the ball indicates the load-based emphasis on lifting (LBEL) of that workout. The LBEL is a metric that tells us not only how heavy the loads were, but what percentage of the workout was based on lifting. So if the workout had 135-lb. cleans and burpees, the average relative weight is 1.00 but the LBEL is 1.00x50% = 0.50.  The left-right arrows indicate the time domain was variable; the plus-symbol indicates the weight was variable.  In those cases, I've used the average time/load for Regional competitors.





You'll notice that as the workouts get longer, the weights tend to decrease and the number of movements tends to increase. This is also typically what we see in CrossFit programming on the main site and in most gyms. For instance, programming 7 minutes of burpees (Open 12.1) is torturous, but not unreasonable; programming 20 minutes of burpees is a recipe for Rhabdo.


Are past years really a good predictor of what's to come this year?

In many ways, they are.  We have not seen big shifts in the types of movements tested, and the loading has not drastically changed in the past four years of the Open (although it has been declining slowly each year).  There were a couple of curveballs last year with the 5-movement chipper (14.4) and the for-time workout (14.5), but those workouts were still not wildly different than what we had seen in 2011-2013.

That being said, the introduction of the scaled division this year could change things.  It's all speculation at this point, but as I said a few weeks ago, HQ may be more willing to push the required loading up a bit in the Rx division.  I'd also say that we could see some slightly more challenging movements (such as handstand push-ups).  That being said, I doubt that HQ is going to change the over ally feel of the Open dramatically.  The Open is likely still going to be a test of conditioning primarily, with a modest amount of strength required in order to move on.  The big weights will probably come into play more at the Regional level, and the truly unexpected movements and workouts will be reserved for the Games.

I hope that new and old athletes alike can get some value out of this.  As I've said before, I'm not going to tell you how to train, but I'll try to help you understand what you should be training for.


*Note that for this chart, I considered Open 11.1 a single-modality despite technically being a clean and jerk. Also, in calculating the LBEL for the snatch workouts with varying weights, I took the average weight lifted for someone who reached regionals. This reflects the fact that, while only 75 lbs. is required, for a regional-level athlete, they'll be moving somewhere around 130 lbs. on average throughout the workout.

Wednesday, November 12, 2014

Quick Hits: What Impact Will Next Year's Games Season Changes Have?

In a surprise announcement following the conclusion of the 2014 CrossFit Invitational, Dave Castro unveiled some changes to next year's Games season.  The details are still a little sketchy, and it sounds like HQ may be leaving themselves a little wiggle room to change their minds, but on the surface, the changes seem pretty substantial.

There were three key changes that Castro mentioned:

  1. Add a scaled division to the Open
  2. Limit each of the current regions to somewhere around 20 athletes qualifying for the second level of competition (formerly "Regionals")
  3. Regions will then be combined (likely in pairs) to produce "Super-Regionals," each of which will qualify 5 athletes to the Games
First of all, I'll say that like many of the changes HQ has made in the past, I am initially skeptical.  That being said, many of those past controversial decisions (moving the Games from Aromas, partnering with Reebok, replacing in-person sectionals with the Open) have worked out well.  So I won't pass judgment just yet.

But that doesn't mean we can't start trying to assess what the impact will be (without labeling that impact "good" or "bad").  I'll address each of the three changes and what impact I believe they will have next season.


Add a scaled division to the Open
  • The reason has to be money, right?  Not that there's necessarily anything wrong with that.  But HQ is likely seeing that there is an untapped market of athletes looking to compete who may not have quite the capacity to handle the Open.  
  • That being said, I have to believe that market is not huge.  What I would expect is a lot of beginner-level athletes who would otherwise have entered the Open will switch to the scaled division.  I'm not sure there will be a ton of increased participation solely due to this division.  Measuring this will be tricky, to be sure.
  • For the Rx division, my guess is this means the loading could increase a bit, but what is more likely is that HQ will start workouts with heavier loads.  For instance, 14.3 was a heavy workout, but it started very light.  It wasn't until the third round that things even got close to a 1.00 relative weight.  I'd expect that workouts like 11.3 (AMRAP-5 of squat clean thrusters at 165/110) could be fair game again, as well as something like the snatch ladder in the Team Series, which started at 135/95.
  • I don't expect the types of movements tested to change much.  I believe HQ still wants the Open to be a test of conditioning with mostly basic movements.  They will leave the more advanced movements and extremely heavy loads to regionals and the Games.  Maybe we'll see some handstand push-ups thrown in, but I still wouldn't expect rope climbs or dumbbell snatch.

Limit regions to 20 qualifying spots out of the Open
  • Yikes.  The Open will be even more cutthroat this year, and absolutely only the very serious athletes will be contending for spots in the next stage of competition.  
  • My fear is that this further intensifies any potential judging problems.  Athletes are going to be pushing the boundaries on range-of-motion in order to get into that top 20, which means more likelihood of complaints from those who don't approve.  The good news is that HQ requires video for all qualifiers, so the evidence will be out there.
  • Here is the list of 2014 Games-qualifying athletes who would not have made it past the Open last year under this format:
    • Quinton Z Van Rooyen (Africa, 40th)
    • Tyson Takasaki (Canada West, 22nd)
    • Cody Anderson (North West, 45th)
    • Kenneth Leverich (Southern California, 31st)
    • Paul Tremblay, who competed on Team Canada at the Invitational, would have been right on the border in 20th.
    • Interestingly, no females would have missed the cut.  This is likely due to the fact that the women's field still doesn't have quite the depth of the men's field.
  • It will be interesting if this number varies by region.  Will 20 athletes still qualify from Africa? Will only 20 be allowed from the South East, which had nearly 5,900 athletes complete all five workouts last year?

Combine regions to make "Super-Regionals"
  • It seems this was done to address some of the issues that have arisen the past few years with some elite athletes from talent-rich regions missing out on the Games despite posting results that would have won many other regions (Graham Holmberg 2014, Nick Fory 2013, Kristine Andali 2014).  
  • The issue here is combining the right regions.  Sticking Northern California and Southern California together isn't going to make it any easier to get out, and putting South West and Latin America together would still make for a pretty weak Super-Regional.  However, it sounds like HQ wants to combine regions based on geography, so we still may end up with some of the same issues.
  • You could (and probably will) see some of the old regions not have any athletes qualify.  For instance, if Central East and North Central were combined last year, Kyle Kasperbauer would have been the only male North Central representative in the top 6 based on the Cross Regional Comparison (even after my adjustments for the week of competition, this would still be true).  Depending on how they combine the regions, we may not see any athletes from Asia, Africa or Latin America.
  • We will potentially see the loading and skill levels increased even further since the bottom half of each region has been weeded out.  However, I think any changes here will be modest, since HQ likely wants each level of competition to maintain its own identity.  I believe the Regionals are still a test of classic CrossFit movements tested at high loads and short-to-medium time frames.  I think HQ will leave the crazy stuff for the Games.
I'd be interested to hear the thoughts of others on this.  I'm still planning on publishing my "What to Expect from the 2015 Open" post in the next few weeks, but there are certainly a few more unknowns than there have been the past two years.  It should be interesting, if nothing else!

Tuesday, November 4, 2014

Team Series Thoughts and Site Updates

The first edition of the CrossFit Games Team Series has come and gone, with Team Reebok East pulling off a somewhat surprising win.  One of the big questions coming into the Team Series was what the programming would look like, considering HQ had never put together anything like this before.  With scaled divisions being offered for the first time, how tough would the Rx division be?  Would things look like the Open, Regionals, Games or something entirely different?  With all 12 events in the books, let's take a look at how things shook out.

Loading

As far as the weights used, the Team Series fell somewhere between the Open and the Regional level.  For men, the load-based emphasis on lifting (LBEL) came out to 0.56 for men and 0.39 for women.  The heaviest Open to date was 2011, which came in at 0.51/0.35 (men/women), but typically it has been somewhere around 0.45/0.30.  The 2014 Regionals were in the same ballpark as the Team Series, at 0.58/0.36, but the Regionals have averaged 0.69/0.44 since 2011.  This year's Regionals were particularly low because lifting made up such a small portion (37%).

Weighted movements made up 51% of the Team Series, which is just about the average across all HQ competitions since 2011.  When weights were used in metcons, the average load was 0.87/0.59, which equates to about a 115/80-lb. clean-and-jerk, a 210/140-lb. deadliest and a 90/60-lb. snatch.  These are by no means hefty weights, and in fact, they are quite close to the historical averages for the Open and well below the averages for the Regionals.  But what differentiated the Team Series was the addition of two max-strength events, something we've yet to see in the Open.  These are a big reason that the Team Series seemed to have a bit more emphasis on strength than the Open.






Types of Movements

With 12 events, we did see a pretty good variety of different movements tested, much more so than a typical Open.  In all, 17 movements were tested, including at least one movement from each of the seven subcategories I typically use ("Uncommon CrossFit Movements" were not used, which makes sense due to logistics).  The largest focus, not surprisingly, was on Olympic-Style Barbell Lifts, which comprised 44% of the points, including 14% on snatch and 12% on front squat.  The 44% actually ties for the most emphasis on any one of these subcategories in any HQ competition, tying the 2011 Open, when Oly lifts also made up that same portion.

Aside from the Olympic lifts, we did see that Basic Gymnastics (24%) were not used as much as in the Open (36% average), but more than Regionals (18%).  We also saw Powerlifting-Style Barbell Lifts make up 13%, which is higher than the historical average for the Open (6%), Regionals (10%) and the Games (3% since 2011).  That was due in large part to a 2-rep max bench press, which is actually the first time bench press has ever been used in an HQ competition.  No wonder Camille said the last time she maxed on bench press was "in a dream."







Time Domains

This one is hard to assess because the team setting often makes it difficult to compare the length of an event in this competition to, say, an event from the Open.  There is often plenty of built-in rest throughout some of these team chippers that a 20-minute team workout with four athletes working together might not feel any more grueling than an 8-10 minute Open workout.  All-in-all, the focus in the Team Series seemed to be more on power output over short time frames and being able recover quickly, rather than the ability to grind out long workouts and keep a steady pace.

Overall

The team series, to me, seemed like a nice blend of the inclusiveness of the Open with the heavier, more challenging movements of the Regionals.  Athletes who had the ability to go heavy or excel at some more challenging gymnastic movements were able to do so, but anyone who is capable of finishing all the Open events would be able to complete the Team Series workouts (provided they had capable teammates).  Although some of the events were a little bland (still can't believe they opened with that dull 14.1/11.1 remake), there were some pleasant surprises that kept things interesting (such as the burpee-box jump/squat clean ascending ladder).

Are there kinks to be worked out?  Certainly.  I also don't think the Team Series will ever be quite as popular as the Open.  I think the feel of the Team Series is necessarily going to be more low-key than the Open due to its position on the calendar.  But that doesn't mean there isn't a place for something like this.  Hopefully HQ will continue to refine what they have and make this an annual event worth looking forward to for the masses.



Site Updates

The Team Series was the first open HQ competition since 2009 that I haven't been a part of (I even competed back in the "Sectionals" back in 2010).  The Open will almost certainly be the second competition I sit out.  The day after last season's Open, I injured my back deadlifting, my second back injury in 6 months.  I have taken things very slow coming back from this one, and opted to skip the Team Series despite feeling 75-80% healthy.  However, in my recovery from the back injury, I looked into some lingering hip pain and found that I had a pretty significant hip impingement and labral tear (the impingement is genetic and was bound to flare up at some point). I will be having surgery to repair the tear and clean up the impingement soon, but that means a recovery time of 4-6 months.

I've also been studying for my latest (and hopefully last) actuarial exam, which I just took last week (results still pending).  All this combined with dealing with a one-year-old baby have made it challenging to make time for the web site.

I say this not because I plan to retire this site or give up on CrossFit.  Rather, I hope that you can continue to bear with me over the next few months if updates are not quite so frequent (though I don't plan to go dormant).  My hope is that in time, I can be back to CrossFitting and blogging on a much more consistent basis.

In the meantime, get your SWAG's ready - the Open is only 3 months away!

Wednesday, September 24, 2014

A Closer Look at the 2014 Games Programming

Today's post will basically put a bow on the 2014 CrossFit Games season.  What's that, you say?  Didn't the Games end 2 months ago?  Haven't we moved onto the Team Series already?  Isn't the 2014 Games season in the distant past?  NO!  It's not over until I say it's over, and since I've been too busy to wrap it up until now, it's not over yet!

Sorry.  Anyway...

Like last year, I'll break things down based on the five goals I think that should be driving the programming of the Games, in order of importance:
  1. Ensure that the fittest athletes win the overall championship
  2. Make the competition as fair as possible for all athletes involved
  3. Test events across broad time and modal domains (i.e., stay in keeping with CrossFit's general definition of fitness)
  4. Balance the time and modal domains so that no elements are weighted too heavily
  5. Make the event enjoyable for the spectators
For each goal, there will be some brief discussion and analysis, and I'll conclude by pointing out suggestions for improvement, because simply identifying the problems only gets us halfway there. Additionally, I'll point out things that I felt worked out particularly well.  For those who've been following this site for a while, this is basically the same way I broke things down at the end of last season.

So let's get started.

1. Ensure that the fittest athletes win the overall championship

I'll start by saying that I do think the two fittest athletes won the titles this year.  Rich Froning came through when it counted, and despite looking more human than in years past, I think there is no doubt that he deserved his fourth straight title.  Camille was ridiculously consistent the whole season, and her combination of Olympic lifting prowess and gymnastic ability is pretty much unrivaled.  Also, as discussed in this post from August, several different scoring systems would have produced the same champions.

That being said, if we're being honest, we have to acknowledge that Sam Briggs realistically could have won the Games if she qualified.  She dominated the Open, and aside from one event (an extremely specialized event at that), she was among the best in the world at Regionals.  While maybe not the most skilled, I feel she has the best conditioning in the world.  Looking at the programming that came out at the Games I think she would have fared well, but it would not have been easy for her.

Looking at the 36 events she competed in during 2013 and 2014, she's only finished outside the top 20 worldwide in 7 of them: 2013 Regional Event 2 (3-RM OHS), 2013 Games Zigzag Sprint, 2013 Games Clean & Jerk, 2014 Open Event 2 (C2B and OHS), 2014 Regional Event 1 (max hang squat snatch), 2014 Regional Event 2 (max HS walk) and 2014 Regional Event 7 (pull-ups and heavy OHS).  The theme here is heavy Olympic lifts and extremely short time domains.  Other than that, she's at the very top of the world.

Applying that to this year's Games, she potentially would have struggled on the 1-RM OHS, the Clean Speed Ladder, the Midline March (due to HS walks) and Thick and Quick.  Other than that, I'd expect her to be top 10 in basically everything else.  Would that have been enough to win?  It's hard to say.

How We Can Do Better: Avoid programming extremely volatile events at Regionals (like a single attempt at a HS walk).  Remember, the goal is to find the fittest athletes for the Games, not just to see who can avoid screwing up (this isn't American Ninja Warrior).  Also, I hope the qualification system gets tweaked to allow more athletes from the elite regions to make it (Central East men, for instance).
Credit Where Credit is Due: The Games test was not overly grueling this year (see discussion in my last post), and while we did have a couple of top contenders fall out due to injury (Kara Webb and Anna Tunnicliffe), it seemed like most of the athletes were competing at their best the entire week.


2. Make the competition as fair as possible for all athletes involved

I think HQ learned from the mistakes of 2013 as they programmed the Open this year.   Judging was pretty straightforward on every event in the Open, and really judging wasn't a major issue throughout the season.  To me, that's a huge key for this sport moving forward.  The less we can have spectators talking about the judging, the better.

I also like the improvements in the way ties were handled at the Games.  We saw very few big logjams the way we did in the in Cinco 1 and 2 in 2013.  All the athletes were able to separate themselves throughout the field on every event.

How We Can Do Better: Though I personally liked the workout, the inclusion of a rower in Open Event 4 goes against the fact that HQ has consistently said that the Open is for anyone in the world.  Rowers are far more expensive than the other pieces of equipment that have been required in previous years.
Credit Where Credit is Due: Judging is becoming less and less of a factor throughout Open, Regionals and Games.  Also, there were fewer massive ties in the standings at the Games.


3. Test events across broad time and modal domains (i.e., stay in keeping with CrossFit's general definition of fitness)

Like last year, let's start by looking at a list of all the movements used this season, along with the movement subcategory I've placed each one into. I realize the subcategories are subjective, and an argument could be made to shift a few movements around or create a new subcategory. In general, I think this is a decent organizational scheme (and I've used it in the past), but I'm open to suggestions.


As we've seen in recent years, the season as a whole is testing a very wide variety of movements.  This year saw 27 different movements, compared with 29 last year.  The only movements that did not appear at all this year that have appeared in at least two other seasons are: bike, KB swing, wall climb-over and push-up. I don't think leaving those out are too much of a concern.

Another key goal is to hit a wide variety of time domains and weight loads. Below are charts showing the distribution of the times and the relative weight loads (for men) throughout the entire 2013 and 2014 seasons. The explanation behind the relative weight loads can be found here.  Two notes: 1) some of the Regional and Games movements had to be estimated because I don't have any data on them (such as weighted overhead lunge and pig flips); 2) the time domains for workouts that weren't AMRAP were rough estimates of the average finishing times.


What was lacking this year, as discussed previously, were the extremely long events.  There were no events beyond 45 minutes this season, and there were twice as many sub-5:00 events as last year.  You'll also notice that there were a smaller number of lifts in all of the ranges except the very low (0.4-0.8) and the very high (2.4+).  Part of this is just the fact that there were more lifts last season, though that was not the case at the Games.  We have previously noted that the Regionals were particularly bodyweight-focused this season, but the Games made up for that by being particularly heavy.

How We Can Do Better: I'd like to see us go really long at least once (though probably not more than once).
Credit Where Credit is Due: The season really didn't miss out on much.  It's really not likely that an athlete could finish well these days with a hole in any area of their fitness.


4. Balance the time and modal domains so that no elements are weighted too heavily

Based on the subcategories of movements I've defined above, below is a breakdown of movements in each segment of the 2014 Games Season. As in the past, these percentages are based on the importance each movement was given in each workout, not simply the number of times the movement occurred (so an OHS in the 1 RM at the Games was worth more than the OHS in Open 14.2).


We see this year that there was consistently focus on the Olympic lifts at all phases of the competition, but we saw changes in the other categories.  Pure conditioning (running/rowing/double-unders) increased in value steadily throughout, whereas high skill gymnastics increased dramatically after the Open.  Conversely, basic gymnastics decreased in value steadily and was not much of a factor in the Games.  As is typical, we didn't see many of the uncommon CrossFit movements until the Games.

In my opinion, the Olympic lifts were maybe a touch overvalued (a third of the competition seems like a bit much).  Also, I'd like to see the three phases of competition be a little more similar, so that you don't have athletes qualifying for the Games without being competent in a particular area, only to be exposed at the Games.  For instance, we continue to see running have little value until the Games, which means we may be letting in athletes who are poor runners.  Of course, those athletes won't win the Games anyway, but you may be keeping out athletes who would have been better Games performers.

That being said, there's no such thing as perfect balance here.  We know the Open can't have a ton of high skill gymnastics movements, and we know that Open-style workouts might not be spectator-friendly at the Games.  You could also argue that powerlifting-style lifts are undervalued, but I'm of the opinion that the Olympic lifts are a much better test, and HQ seems to agree, so I doubt we will see that change.  I don't see any glaring problems in the chart above, so that's good news.

As far as time domains, I think there was pretty good balance, even though a bit more weight was given to shorter workouts than in years past.  Like I mentioned previously, I can't complain too much about this, because it seems to keep the athletes fresher throughout the Games.  As long as HQ continues to throw in a few nasty workouts in the 15-25 minute range, plus one or two really long ones, I think that's sufficient.

Another way to see if we're not weighting one area too much is to look at the rank correlations between the events. If the rankings for two separate events are highly correlated, it indicates that we may be over-emphasizing one particular area.  As I did last year, I focused only on the Games, since it's not really a problem if we test the same thing in two different competitions since the scoring resets each time.  It's the overemphasis within the same competition that's a problem.

The chart below shows all the combinations of men's events at the Games, excluding the finals, since the entire field did not compete.


The cells highlighted yellow had a correlation above 50%, and the cells in red had a correlation below 0% (for some reason, the "-" sometimes doesn't come through well on the picture, but those are negative numbers).  Only two combinations had a correlation above 50%, and one of those were the two sprint sleds (each of which were worth only 50 points, so this actually isn't really a problem that they're highly correlated).  The other combination was event 2 (1-RM OHS) and event 9 (clean speed ladder) - this shouldn't be surprising.

For women, the results were similar, although there were also high correlations between Event 1 (The Beach) and Event 3 (Triple-3) and between Event 2 (1-RM OHS) and Event 6 (21-15-9).  Not sure about the OHS/21-15-9 combination, but the other one also makes sense intuitively since both events are very long.

It's also interesting to note the negative correlations, most notably the -31% correlation between Event 3 (Triple-3) and Event 9 (Clean Speed Ladder).  Women showed a -14% correlation here as well.

All in all, this is similar to what we saw last year, but probably a bit more balanced.  Between men and women, there were 6 correlations above 50%, compared to 8 last year, and as mentioned, 2 of those (sprint sled events for men and women) were only given half-weight.  

How We Can Do BetterI've said it before, but it bears repeating: We need to test running earlier in the season (and more than just 50-yard jogs between rope climbs).  Also, I'd prefer we lighten up just a bit on the Olympic lifting emphasis.
Credit Where Credit is Due: Things were really balanced overall, and cross-correlations between events at the Games were lower, indicating we weren't testing the same things a bunch of different times.


5. Make the event enjoyable for the spectators

I was lucky enough to attend the Games in person for the third straight year (thank you to my lovely wife), and I can say without a doubt that they are improving the spectator experience each year.  I was originally concerned about the fact that the morning and afternoon events would all be in the soccer stadium, but I think they made it work.  There's still no doubt that the tennis stadium is the more exciting venue, but the issue there is that you can't fit as many athletes in each heat, so the sheer number of heats tends to make things drag on.  In the soccer stadium, they were able to keep most events to 3 heats, which really is a big improvement over 4 heats.  Not to mention that all the silver ticket holders get to watch those events as well.

Another change was that the team events were all done in the morning prior to any individual events, so it really separated that out from the individual competition.  While some of the teams may disagree, I liked this move because it shortened the breaks between individual events.  Some of the team events can really drag on, and while the top heats are often exciting, the majority of fans probably don't want to sit through tons of the team competition if they don't have to.  And of course, those who do want to watch can show up in the morning and get great seats.

I think ultimately, the Games are going to have to have all of the events in the soccer stadium, or some other large venue.  There is simply too much demand to continue limiting the Gold tickets to only 10,000 fans.  From my understanding, they sold out in a matter of minutes this year, and even some people who were logged in prior to the tickets going on sale could not get them.  Let's hope HQ can find a way to satisfy the demand while still keeping the event as exciting as possible for those who do get tickets.

How We Can Do Better: Do not let Swizz Beatz perform again.  In fact, replace the musical act completely with those crazy gladiator dudes.
Credit Where Credit is Due: The experience continues to improve each year.  The views in the soccer stadium were improved, there were fewer lulls in the action and the spectators were more engaged than in previous years.  The conclusion of that men's Push-Pull event with Bridges narrowly holding off Froning was still the highlight of the weekend for me.


And now... the 2014 CrossFit Games season is in the books.