I’ve written about the evolution of draft pick value in Major League Soccer before and how I’ve used that history to evaluate the performance of players and the teams that drafted them. The idea of draft pick models is to relate the draft position to the expected career value of the expected player. A subset of this idea is that clubs care a lot more about drafting players who will create value for them immediately. By building a model of draft pick value I hope to understand which players were over- or undervalued and which teams have done the best jobs at identifying talent in the draft.

Here was the draft performance of the 2017 SuperDraft class after the 2017 regular season. The Total Draft Performance Rating is the difference between expected career value and actual career value which is then scaled by draft position. (How this career value is computed is really up to you — the measure that I use is quite simplistic — but it should be consistent). As I said, it’s way too soon to tell which teams did well after Year 1. But the Philadelphia Union stands out for selecting Jack Elliott late and seeing him win a starting position. And the LA Galaxy traded away all their draft picks, so they don’t even appear in the figure.

How does the 2017 SuperDraft class look after a second season in the league? The figure below displays the cumulative draft performance rating of all MLS teams that participated in that draft (so the Galaxy are missing again). Again, these performance ratings are over an expected tenure with a club, so it’s not unusual for these ratings to be in negative territory.

As of the end of Year 2 only four teams are in positive territory: Philadelphia Union, Vancouver Whitecaps, Columbus Crew, and FC Dallas. Two of the Union’s draftees have seen playing time in the league, but one was a second-round pick (Marcus Epps) and the other a fourth-round selection. Vancouver’s rating was entirely that of Jake Nerwinski, who was selected 7th and has played close to 4000 minutes. Columbus Crew saw three of their four draftees win playing time, but two of them (Lalas Abubakar and Niko Hansen) were among the first ten picks so they were expected to start; Connor Maloney at #49 was a pleasant bonus. FC Dallas made seven selections, two of which saw playing time, and those two were selected in the top 40% of the draft.

Overall, 49 of the 81 selections have yet to appear in a league match (Sporting KC’s Colton Storm was the highest-selected player yet to appear). Some of these team ratings will change after next season, but the end of year 2 is when we see options declined by clubs, so these class performances are starting to set in.

]]>With FC Cincinnati entering the league there are now 24 selections in each round for a total of 96 slots. The values are expected career values of players selected at each draft position, taking into account performance of players drafted in the previous four seasons. The values are then scaled so that the top draft pick is 100.0.

The relative values of the draft picks have changed very little from last year but the decline in value is apparent when one compares previous draft curves. The figure below compares curves of expected career value relative to the normalized draft position for the years 2013, 2015, 2017, and 2019. (Position 0.0 is the first pick, 1.0 the final pick, and 0.5 the final pick of the second round.) Beyond the first seven or eight players of the first round, the value of the remaining slots has declined significantly, and especially so for the mid-draft slots.

In light of this apparent reality, it’s little wonder that Philadelphia Union’s sporting director decided to give away his team’s draft picks.

We also have this helpful information on the complex nature of MLS transactions. FC Cincinnati first executed a trade with LAFC in which the West Coast side sent this year’s #16 draft slot and $175,000 in GAM for Cincinnati’s fourth-round selection and their Allocation Ranking in 2020 (provided that they finish outside the top five in Allocation Ranking Order). Then, FC Cincinnati sent $150,000 in GAM to Philadelphia in exchange for all their draft picks in 2019.

As we receive more data on the amount of GAM that is included in trades involving MLS draft picks, we should be able to extend our models to come up with the expected value of a draft pick in terms of allocation money. All this sounds like a nice academic exercise, but given the current and declining values of these draft selections, how much longer can the college draft last?

]]>As you know, I’ve written several articles that summarize my research into draft pick valuation from the performance of previously selected players. I’m using that research to create a draft value card for the 2018 MLS SuperDraft. Here it is below.

The values are expected career values of players selected at each draft position, taking into account performance of players drafted in the previous four seasons. The values are then scaled so that the top draft pick is 100.0.

As has been the case in recent seasons, the relative value of the draft pick falls off exponentially, stabilizes midway through the second round, and declines steadily in the third and fourth rounds. According to the first draft selection is more than eight times more valuable than the first pick of the second round, more than 200 times more valuable than a selection in the third round, and more than 1000 times more valuable than a pick in the fourth round. That’s not to say that latter round picks can’t make contributions — they certainly can, and even as first-year players — but that’s not the way to bet.

At any rate, it will be interesting to see if the teams see the draft values in the same way as these estimates.

]]>I’ve written about the draft valuation models in part 1 of my draft pick value posts (scroll down to the “Valuation Models” section), so I’m not going to repeat myself too much. Draft pick models relate the draft position to the expected career value of the selected player. How you calculate “career value” is up to you, but it must be consistent. I used a simple valuation metric that used basic performance data that is available over the entire history of MLS. I also created what I called a “Club” model which related draft position to the expected value of the player over his tenure with the team that drafted him. The expectation is that clubs want to draft players who will create value for them immediately, not in five or six years when they are (likely) somewhere else.

The draft performance rating is the difference between expected career value and actual career value which is then scaled by draft position. Late selections that make significant contributions are strongly weighted positively, and early selections that do not are strongly weighted negatively.

In part 2 of my draft pick value posts, I calculated the draft performance rating of all draft selections made between 1997 and 2013 from the club perspective. If one views the draft as an opportunity to select players who will make the most contributions to the club, one sees a cluster of three clubs that have done this very well: Chicago Fire, LA Galaxy, and Sporting Kansas City (ex-Kansas City Wizards).

If we look at the 2006-12 period, which covers the Designated Player era, the same three clubs — Kansas City, LA Galaxy, and Chicago — are at the top of the list. (Miami and Tampa Bay had already been contracted four years earlier, and Montreal had not yet entered the league.) There is significant movement of the teams outside of the top three; Dallas’ and Chivas USA’s draft performance ratings were just barely ahead of expectations during this period. In fact, more than 80% of Chivas USA’s draft overperformance came from two draft years — 2005 and 2006 — in which the club drafted Brad Guzan, Sacha Kljestan, and Jonathan Bornstein.

If we look at the period between 2012 and 2017, we see very dramatic changes in the ratings and the teams at both end of the list. The draft performance ratings are negative for most teams, not necessarily because most teams underperformed in the draft, but because it is too soon to tell whether the draft selection was a success or a bust. We see a significant overperformance by Colorado in the draft (thanks to selections of Dillon Powers, Jared Watts, and Dominique Badji), some modest overperformance by Philadelphia, DC United, FC Dallas, and Seattle, and a broad middle of draft performance in line with expectations or a little negative.

It’s way too premature to calculate draft performance for the 2017 draft class, but I’ll do it anyway. Most draftees have yet to appear in a league match, so most of the team performance ratings will be negative. Philadelphia Union’s total rating was the only positive rating of all MLS teams thanks entirely to the Jack Elliott pick. Selected late in the fourth round, he earned his place into the starting lineup, held it for most of the season (only lost it due to injury), and finished third in the balloting for MLS Rookie of the Year. So why was Atlanta United’s draft performance rating so negative despite having a Rookie of the Year winner? Their other first-round selection, Miles Robinson, didn’t play a minute in the league last season. The other selections aside from Julian Gressel didn’t appear either, but they weren’t expected to do so anyway. One other thing that I’m fairly sure that I’ve read about but have just noticed is that the LA Galaxy is missing entirely from the SuperDraft. They traded away their draft picks for player acquisitions and passed on their lone fourth-round selection. Whether that becomes a strategy among other teams that want to prioritize their academy will depend, of course, on results.

I’ve written before that the expected lifespan of a draft class with the original club is close to three years, so it will take a couple of years to see which clubs got the most from their 2017 draft classes. It does appear that the Galaxy, who have dominated the draft historically, have transitioned away from it. Who will dominate the draft in this new (and maybe final?) era remains to be seen.

]]>Major League Soccer has undergone a lot of changes over its 20-year history, and the value of the draft to its member clubs has been among those changes. The most obvious use of the draft pick value model is to estimate the relative values of picks so that front-office officials have a better idea of its exchange power — the famous Dallas Cowboys draft card, for example. I plot the expected career values relative to the maximum value (first pick) of the MLS College and SuperDrafts every other year between 1998 and 2012. The value of the early drafts was so much higher than in previous years because a majority of players entered the league via the draft system. The relative draft value of all picks fell at the turn of the century, then stabilized between 2002-2006, which were part of the Golden Era of the drafted player (over 60% of all players in the league were drafted during this period), then dropped again to its current level. (I’m not sure why the relative draft value saw such a big uptick for the 2008 model. That would be worth examining!)

The punchline here is that relative draft value can change significantly year-on-year. It can also remain static, as the periods between 2002-2006 and 2010-present indicate. But it does mean that holding on to the tattered six-year-old draft card to make trade decisions on draft day make little sense.

This was one finding that popped out at me late in the summer, and it was interesting to see it again when I created a Bayesian model with credible regions and all. For the draftees taken in the first half of the SuperDraft, there was a significant difference between the expected career value of the Present model and the expected career value of the Club model — there was no overlap in the 95% credible regions of the two curves. Furthermore, this difference in expected career value has persisted over the history of the league. Except for some brief moments in the early years of the league, there is an overlap between the expected values of the Present and Club models, which indicates that almost all of the expected value of these late draftees will be with the drafted club.

Another use of the draft pick value models is to retrospectively assess draft selections and practices. Here we calculated the draft performance rating of the drafted players using Present or Club valuation models. Regardless of whether you use the draft to get the best talent or the most compatible talent for the team, Davy Arnaud (5th round pick in 2002 SuperDraft) and Kevin Hartman (next to last pick in 1997 College Draft) were the biggest steals in MLS draft history with over 700 MLS regular season appearances between them. Jeff Parke (the final pick in 2004 SuperDraft) came close, but Arnaud had almost 100 more appearances.

I’ll post the full list of draft selections and their draft performance ratings on my Project Data repository in the near future.

When analysts and pundits draw up their lists of the worst MLS draft selections ever, Real Salt Lake’s Nik Besagno is almost always at the top of the list. The selection of a US U-17 international as the first pick of a new MLS franchise (and first pick overall) was stunning at the time, and the subsequent lack of performance brought into question John Ellinger’s judgment. The draft performance rating for this draft selection in 2005 confirms the popular observation of this pick. (I shouldn’t be too hard on the guy; he appears to be a smart guy — a math/CS major! — and one who has moved on with his life.)

In my opinion, Ben Parry should get consideration for worst field player selection ever. The third overall pick from San José Clash in 1997, Parry went straight to injured reserve and then to a USISL touring side called US Project-40, and then never returned to MLS again. Parry’s draft performance rating (-5.1) puts him at the bottom of the draftee list.

Another use of the draft value model is that we can look at the collective and year-on-year performance of the draft war rooms at each MLS club.

If we look over the College Draft and SuperDraft from 1997 to 2013, and calculate the draft performance rating of all draft selections, we see a couple of interesting patterns. First of all, if the goal of the drafts is to identify and select the talent with most upside in the league, the Galaxy dominate the draft by a wide margin:

I find it fascinating that Miami Fusion — a team that was only in the league for four seasons — had a cumulative draft performance rating that was higher than Colorado Rapids, an original MLS team. (The Rapids are almost certainly ahead on this score now.)

If one views the draft as an opportunity to select players who will make the most contributions for the team selecting him, then the cumulative performance rating is calculated for the Club model and looks like this:

Now the situation changes. LA Galaxy are still in the mix, but there are now three teams that, according to this model, are good at finding players who deliver big value for them: Chicago Fire, Kansas City Wizards, New England Revolution. One can identify two or three other groups that describe degrees of success in the MLS draft system.

If one sets aside goalkeepers (the current valuation metric for goalkeepers dominates that for field players), the picture changes further. When it comes to selecting field players that have high impact for the drafting team, Kansas City Wizards, New England Revolution, and Chicago Fire stand out.

If one drills down into year-on-year draft performance graphs, he/she will see a lot of noise and a few spikes where teams had exceptional success with that year’s draft. The median draft performance rating over the history of the league is about 1.2, which usually translates to moderate success at the early draft positions.

There are some draft classes that stand out because of the value gained by the team relative to the draft position. They are:

- 2005 Chicago Fire (Gonzalo Segares, Chris Rolfe, Chad Barrett)
- 2002 New England Revolution (Taylor Twellman, Shalrie Joseph, Marshall Leonard)
- 2004 MetroStars (Jeff Parke, Seth Stammler, Zach Wells, Michael Bradley)

Some teams have good consecutive years at the draft, but for the most part that should not be expected, and it seems that in today’s complex player acquisition environment, it’s no longer assumed that the draft will fill all of the team’s needs.

There is a ton of additional insight that can be pulled from these results, but it’s a good idea to stop here.

]]>A draft pick is an asset that can be exercised, sold, traded, or allowed to expire unused. With that in mind, it’s of interest to know what a draft pick is worth. You could respond that the draft pick, like an illiquid or intangible asset, is worth what a willing buyer and seller agree upon, and that’s true. A draft pick’s value can also be defined from the value of transactions involving a draft pick, or from the value (however we define it) of previously selected players at similar times of the draft, or any other method. We’d expect the top overall selection to be the most valuable because it provides a chance to select from the entire draft pool, and selections in an early round have more value than selections in later rounds.

Let’s say we agree upon a valuation of every slot in the draft. There are a few things we can do with that information. We can create a chart of the relative value of all draft picks to inform transactions involving a draft selection (which the Dallas Cowboys made famous). We can combine the absolute value of these picks with projected values of candidate players to determine the best players available at a certain draft position. If we go further and create values for every slot in previous drafts, we can go back and understand which draft selections in hindsight were clever or foolish, and which organizations were historically brilliant or suspect in their draft performance.

A valuation represents one perspective for defining value. We could define draft pick value from career performances of past players or from what is offered in exchange for players, money, or other picks, which we could use to determine if one party is over- or under-estimating draft values, in our opinion.

Other analysts have considered the draft valuation problem and made their own contributions to it. Tim Swartz and his team in the aforementioned paper created valuation curves by making lowess (local regression) fits of peak salary levels and minutes played as a function of draft position over a 12-year period. Ford Bohrmann also used a lowess fit of career minutes played by drafted players to create his valuation model. A contributor to the Sounder At Heart website charted a moving average of minutes played by draftees in four consecutive SuperDrafts in order to identify over- and under-performing draft selections and organizations. There are others who have attempted to identify draft performance through descriptive statistics or their own scoring systems.

There are some things about this analysis that make it stand out against the prior art:

- It covers almost the entire history of the MLS College and Super drafts, from the 1997 College Draft to the present day. (Supplemental Drafts are not considered but dealt with separately.)
- Every player selected by the MLS draft is incorporated in this analysis — over 1,700 players.
- Draft pick valuation models are created that are valid only for a specific MLS season (except the first one), and incorporate player data from the previous four seasons only.
- As an alternative to local regression, a Bayesian regression is applied to the data in order to determine credible bounds of expected draft values.
- Create two types of valuation models that communicate the following: the ability of clubs to identify talent able to thrive in the league, and the ability of clubs to find talent that benefits them.

There are a couple of things that we have to define before we formulate valuation models. Here they are below.

The number of slots in MLS’ drafts has changed significantly over the years, from 160 picks over 16 rounds in the 1996 Inaugural Draft to 88 over four rounds this year. We transform the draft position \(i \in [1, N]\) so that it lies on a \(\alpha \in [0, 1]\) interval — draft position 0 is the first pick, and draft position 1 is the final pick.

\[

\alpha = \frac{i – 1}{N – 1}

\]

This will be controversial, and I recognize that it’s imperfect, but I chose to create a simple measure of career value using summary statistics that are accessible in every MLS season:

\[

V = \sqrt{\left(\frac{M}{M_{max}}\right)^2 + \left(\frac{G}{G_{max}}\right)^2 + \left(\frac{A}{A_{max}} \right)^2}, \, \mbox{field players}

\]

\[

V = \sqrt{\left(\frac{M}{M_{max}}\right)^2 + \left(1 – \frac{G_A}{G_{A, max}}\right)^2 + \left(\frac{S}{S_{max}} \right)^2}, \, \mbox{goalkeepers}

\]

The basic idea is that I wanted to capture player participation (minutes played) and player performance (goals scored and assists made for field players, goals allowed and clean sheets for goalkeepers). All of the metrics are scaled by the league leaders in respective categories. Goals allowed is a “negative” metric, so a low number relative to the league maximum is seen in the same way as being among the league leaders in goals or assists.

A more robust metric such as adjusted plus-minus or points above replacement can be dropped in place of the above metrics, but this simple value allows us to compare drafts across the entire history of MLS.

As I wrote before, one of the applications of a draft valuation model is to evaluate past draft selections. To this end I’ve created an expression that scales the difference between and expected and actual career value by the normalized draft position. The idea is that late selections that go on to make significant contributions should be strongly weighted positively, and early selections that turn out to be duds should be strongly weighted negatively.

\[

R = \Delta e^\alpha,\, \Delta > 0

\]

\[

R = \Delta e^{1-\alpha},\, \Delta < 0

\]

Essentially the valuation model is a function that relates the draft position to its expected career value:

\[

V = f(\alpha)

\]

Our goal is to determine \(f(\alpha)\), and from there, estimate the career value associated with a draft pick.

One method to determine \(f(\alpha)\) is a lowess regression, which is a locally-weighted and smoothed linear regression. This is what Swartz et al. and Bohrmann used to estimate their valuation curves. It’s non-parametric, so the resulting curve can’t be described easily by an equation.

The other method is a Gaussian process model, which is a non-parametric Bayesian regression model. We assume that the points that make up the curve are drawn from a multivariate Gaussian distribution with zero mean and variance defined by a covariance matrix. It’s this covariance matrix that defines the amount of continuity and smoothing in the resulting curve. A fuller description of the model is in my Atlanta presentation slides, but the main idea is that we can produce uncertainty regions around the expected draft value curve.

The valuation models are trained with the career values calculated for all players drafted in the College Draft or SuperDraft. The scope of the training data determines the model’s type. I’ve defined two models — a “Present Model” and a “Club Model.”

The **Present Model** is trained with cumulative player value data of draftees the four years before the year of interest — a 2009 model is built with data from players drafted between 2005-2008. Even if a player goes on to have a longer career, we only consider his performance in those four years. The reason is that in 2009, we’re not aware which drafted players will go on to long career in MLS, so we can only work with the knowledge available at that time. It’s “living in the present”, which inspired the name of this model.

The **Club Model** is similar to the Present Model, except that the cumulative player value data is calculated only for the period where the draftee is playing for the club that drafted him. The thinking is that clubs are more interested in drafting players who will best benefit them than players who are likely to have long and productive professional careers.

There is a significant difference between the valuation curves associated with both models. It’s especially pronounced in early draft picks and in previous MLS seasons before 2011.

So why apply a four-year window? From my survival analysis of MLS draft classes, the median lifetime of a draft class is four seasons, so that seemed like a reasonable cutoff.

This post is getting way too long and I want to publish part of it now, so I’ll stop here and create Part 2 later.

]]>It is one thing to be drafted by a Major League Soccer team. In recent years, that honor has been extended to somewhere between sixty and eighty (mostly) college soccer players every January.

It is another manner for a drafted player to make it onto a MLS roster and enter the field of play for a MLS league match. Over the 20-year history of the league, just over half of players drafted have appeared in a league match, even if just for one minute.

To be sure, being drafted by a MLS team and making the team are major achievements. The real battle is the fight to stay on a club roster and remain on the pitch. On average, within four seasons half of a year’s draft class is out of MLS.

That last figure shows how long a cohort of draftees remains in Major League Soccer as a whole. But let’s drill down to the individual clubs. How long do those draft classes stay with their original teams? The thinking is that MLS clubs — and NFL teams and NBA teams and MLB teams and NHL teams — are drafting players who they believe will benefit *them*, not merely the league as a whole.

To answer this question, I used a local regression (lowess) plot to calculate the median number of seasons that the members of a team’s draft class remained with their original team. I considered players drafted in the College and SuperDrafts between 1997 and 2013. The data was time-shifted so that they all fell on the same axis (0 = year of draft, 1, 2, 3 = seasons after draft), and the data was produced to fit the plot, from which the expected proportion of the draft class to receive a league contract (year 0) and the number of seasons to for the proportion of the class to reach half that of year 0 are calculated.

The below plot shows the median draft class longevity for the league teams in drafts between 1997 and 2013. If you don’t see your team, that’s because there wasn’t enough data to calculate a median value (Portland, Seattle, Montreal, NYCFC, Orlando City).

Keeping in mind that these figures only consider the subset of draftees given a contract, they throw up something interesting. Sporting Kansas City (ex-Kansas City Wizards) had one of the highest draft mortality rates in the league, but the players who do stay remain at the club the longest. Chicago Fire’s draft classes stay at the club the longest, followed by New England Revolution. It is interesting to see that most of the clubs in this list see about half of their draft classes leave within 2.5 years, and three see their draft class turn over within two seasons. Toronto FC’s draft classes would only last a season or two at the most, but this was back in 2013, and it seems that the club ownership has recognized the situation that they were in and made significant changes since then.

From the player’s perspective, it is an accomplishment to be drafted, to make the roster, and to play. But it is a greater accomplishment to stay on that roster and keep playing. From the front office perspective, one has to wonder constant turnover in the squad is a good thing, even with the reduced proportion of drafted players in the league.

]]>