Attempting to put an end to the Stat-versus-Scouting debate

by Shaun

A lot of the heated discussion on this website boils down to whether a statistical approach is useful or whether teams should only use scouts.  It seems most of baseball has already settled the issue, if there ever was an issue.  I think one could argue that the issue of stats versus scouts was overblown.  It seems most Major League organizations are always looking for ways to improve and at the same time save money, so most have grasped that a more statistical/scientific approach to evaluating players is extremely helpful.  But there are some out there, and maybe a few within the game, who still think it’s an issue worth debating.

First of all, it’s clear that a scouting approach is absolutely necessary.  Scouting has obviously always been a part of the game.  No statistic is ever going to match watching, if a scout knows what he’s looking for, and most who work in pro baseball probably know what they are looking for or else they wouldn’t be working.

A scouting approach is even better than a bad statistical/scientific approach.  I heard a podcast a few weeks ago on which ESPN’s Keith Law, a scout with an affinity for “advanced” statistics, said that it was better for a team to not look at any statistics than to look at “baseball card” statistics.  What he meant by that is that some stats are not very useful when it comes to evaluating what a player can and is likely to do on a baseball field and aren’t going to be as helpful as sending scouts out to watch.  Some of those statistics include pitcher wins and losses, runs batted in, and runs scored.  These stats largely depend on a player’s opportunities rather than a player’s skills.  So an organization, like the Philadelphia Phillies, who reportedly use more or less only a scouting approach can obviously do just fine.

Also, a team can misinterpret more useful statistics.  A player who posts a high on-base percentage, a high slugging percentage, a high walk rate and walk-to-strikeout ratio who is 25-years-old and playing in Double-A is not as impressive as a player with those same statistics who is 19 and in Double-A.

An organization that uses the less-useful stats or misinterprets the more-useful stats is going to fare worse than an organization that more or less purely uses scouting to evaluate players.  I’m not sure what approach the Mets take (sometimes I wonder if they know).  But their trade of Ryan Church for Jeff Francoeur last season was an example of an organization doing a poor job of player evaluation.

It’s not that Ryan Church was a great player and that he was sure to outperform Francoeur.  But the Mets traded an average-to-slightly-below-average player with a lower salary who they could non-tender at the end of they year for an average-to-slightly-below-average player who was more expensive.  It’s hard to see how this move improved the Mets.  The only advantage was that Francoeur was younger than Church.  Aside from that, there was absolutely no reason for the Mets to make that trade.  They either put too much emphasis on Francoeur’s “tools” or they looked at the wrong stats, namely Franceour’s two 100-RBI seasons that were a result of Francoeur being in the lineup literally every day and hitting somewhere behind high on-base guys.

If the Mets had done their homework, their scouts would have seen that Francoeur was an out-machine that wasn’t likely to improve because he didn’t have the skill set to take a walk; he swings at bad pitches too often and gets himself out.  If they had looked at the appropriate stats, they would have basically seen the same things:  A low on-base percentage, a low walk rate and a rather unimpressive slugging percentage for a major league corner-outfielder; plus he was more expensive than the guy they traded and the guy they traded at least had a much better track record of getting on base and not getting himself out.

Back to organizations and their approaches, it’s clear that a statistical approach is helpful.  Back in the offseason of 2002-2003, the Boston Red Sox made big changes to their organizational philosophy.  They attempted to hire Billy Beane, a champion of sabermetrics within the baseball establishment, as their general manager.  They ended up hiring then 28-year-old Theo Epstein, a Yale grad with a law degree from the University of San Diego and a “stat head.”  They also hired Bill James as a consultant, the man who invented the word “sabermetrics” and who brought a more scientific, data-based approach to baseball into the mainstream.

The Red Sox didn’t do away with their scouting department.  Instead they used both a statistical/scientific approach and a scouting approach and ended up as the only team with two World Series wins during the first five years with Epstein as their GM.  The Red Sox saw the value of gathering as much information as possible, from scouting reports to detailed statistical reports, and it paid off for them.  (It also didn’t hurt that they had a brilliant business man running the show in John Henry.)

Some will criticize the statistical approach because of its apparent arrogance, acting like they are reinventing the wheel.  I’m sure there are arrogant “stat heads” out there, just like I’m sure there are arrogant scouts out there.  But a statistical approach is not about inventing or reinventing anything.  It’s about gathering as much honest data as possible and using all the technology on-hand.  Branch Rickey is probably the most famous “stat head” pre-Bill James.

All of the current baseball “stat heads” realize they aren’t the first to attempt to use advanced statistical data to a large degree in evaluating baseball players.  The differences between now and Rickey’s time in regards to baseball statistical analysis are the advances in computer and multimedia technology.  There are entire companies dedicated to things like tracking batted balls and to what degree pitches break.  There are ways that weren’t available in Rickey’s time that can calculate what all hitters did relative to their leagues and his ballparks, within a split second.  Technology is why the statical/scientific approach is better now than ever and most “stat heads” realize this.  They realize it’s not about them;  it’s a matter of working through the data using the tools that are now available to answer questions about baseball.

This is one advantage to a statistical/scientific approach over a scouting approach.  A statistical/scientific approach, because of technology, can grab lots of data and spit out something useful about every player in baseball within a split second while scouts must go watch players and make an evaluation based on their own eyes and their own judgments.  A scouting approach takes more time to gather data, interpret data and is more susceptible to bias.  This is not an indictment of scouting.  Obviously there are things that scouts can see that stats can’t possibly account for.  A stat can’t tell you about a hitter or a pitcher’s mechanics.  A stat can’t tell you about a catcher’s release.  But a scout can’t tell you how much better a hitter is than every other hitter in the league and adjust for the ballparks within a matter of a split second.

In general, within the game it seems both each sides respect each other a great deal.  In fact it seems there is a great deal of crossover because most organizations aren’t going to ignore one side or the other.  For a major league front office, it’s about gathering as much information as possible on players.  If you have two useful approaches, along with many others, most organizations realize it’s quite appropriate to use them all.  It’s all about getting to what a player can and can’t do on the baseball field.  There is more than way way to get to that point.  Some ways have different advantages and disadvantages, some ways are more useful than others, and some ways aren’t very useful.  Scouting and statistical approaches have different advantages and disadvantages but both are useful and both can get you to what a player can and can’t do on the baseball field.

Tags: ,

Recent Posts

11 Responses to “Attempting to put an end to the Stat-versus-Scouting debate”

  1. Hossrex Says:

    I wonder if Shaun typed this out, thinking Chuck would give him a tearful standing ovation, while loudly admitting that he’s seen the light.

    This isn’t anything different from what was in the other discussion thread… what would make anyone think restating your position, only this time using MORE of the same words, would change anyones mind.

    In the end… as much as I think both scouting, and statistical analysis are important… what this boils down to is what Chuck says (a man who’s spent years in the process of scouting, and seen how said process works in minute detail), versus what Shaun says (a dude who owns a computer).

    That’s an argument from authority, I agree… but if “both!” isn’t the answer, it’d sure be hard to give Shaun the edge.

  2. John Says:

    Shaun: “The Red Sox didn’t do away with their scouting department. Instead they used both a statistical/scientific approach and a scouting approach and ended up as the only team with two World Series wins during the first four years with Epstein as their GM.”

    Let’s look at the 2004 Boston Red Sox who made that title possible. I’ll even rank them by WAR…

    Curt Schilling: One of the top 5-10 pitchers in the game for about a decade before his signing. Good move by Epstein, but you could’ve hired 14 year old me as a consultant instead of Bill James and I would’ve been perfectly capable of telling you “yeah, Schilling is a good at pitching!.”

    Pedro Martinez: Acquired in 5 B.E. (before Epstein), and was a dominant pitcher whether you used WAR, ERA+, WHIP, or CHUCK (a stat I’ve invented that measures how highly Chuck thinks of you as a baseball player).

    David Ortiz: Epstein signing, acquired after the Twins released him. Nothing about this acquisition was based on SABR-anything. He sucked, and magically became good *COUGH* STEROIDS *COUGH* HE WAS USING STEROIDS *COUGH COUGH*.

    Jason Varitek: Acquired in 6 B.E. in one of the most lopsided deals in baseball history along with Derek Lowe for Heathcliff Slocumb.

    Manny Ramirez: Acquired in 2 B.E. during the Duquette tenure, and was perfectly good based on traditional metrics…he had 165 RBI in 1999. 165. From 1998-2000, he averaged more than an RBI per game. Also, showing how flawed WAR is, Manny is ranked as less valuable than Ortiz (a DH!) based on defensive value.

    Bronson Arroyo: Very good pick-up, so +1 pt to Epstein…except that he was traded by Epstein two years later for WILY MO PENA.

    Johnny Damon: Acquired in 1 B.E. as a free-agent after a career-worst season.

    Mark Bellhorn: Finally, an Epstein signing probably based on numbers (based on his very good 02, not his pitiful 03).

    Bill Mueller: Epstein signing. Good move.

    All in all, Epstein’s done a good job. But he has a TON of resources. And I’m not talking computers and Bill James’s, I’m talking green resources with presidents on them. I mean, he’s out-GM’d the collective idiots running the Mets and Cubs who have similar payrolls, so kudos…but there’s nothing inherently new-agey or statty about figuring out that Curt Schilling is a terrific pitcher.

  3. Hossrex Says:

    Shaun: “The Red Sox didn’t do away with their scouting department. Instead they used both a statistical/scientific approach and a scouting approach and ended up as the only team with two World Series wins during the first four years with Epstein as their GM.”

    Technically it was his first five seasons as GM, and there aren’t too many people who would credit a general manager as the man responsible for winning the world series in his second year of tenure.

    I actually think Epstein does a fine job… but like a president who presides over a strong economy… it’s easy (and not unreasonable) to credit his predecessor with the prosperity.

  4. Shaun Says:

    “That’s an argument from authority, I agree… but if “both!” isn’t the answer, it’d sure be hard to give Shaun the edge.”

    That’s my point. There should be no debate. Both (and more) is the best answer. A team should hire stat experts, scouting experts, medical experts, financial experts, any kind of expert that can tell the front office as much info as possible about a player’s performance, what a player could or is likely to do and what a team should pay that player. I even admit that if a team feels they must choose just one approach, scouting is the way to go rather than rely on a half-assed statistical approach. The problem is not stats versus scouting. The problem is that some on this site only see value in scouting and don’t think a team should even use any other approach.

  5. Shaun Says:

    My point wasn’t necessarily that Theo Epstein has done a great job and is a brilliant GM, although I do think he’s done as well as pretty much any other GM. I do agree his predecessor deserves a great deal of credit and that some of his moves, particularly the ones that led to the first World Series win, weren’t all that outside-the-box. And I think it’s like field managers and players; we should be careful not to overvalue one guy.

    My point was more that the Red Sox changed their organizational philosophy more or less around the time Epstein became the GM. If any single person deserves more credit than another, it’s probably John Henry.

    By the way, I’ve made the correction on World Series wins in Epstein’s first five years instead of four. Good catch.

  6. Raul Says:

    Branch Rickey is known for a lot of things. Stats analysis being one of them. Branch Rickey also had a good eye for scouting talent. There are a few nice paragraphs about him scouting Roberto Clemente in “Clemente” by David Maraniss.

    If Theo Epstein was half the GM Branch Rickey was, the Redsox would win the next 12 World Series.

  7. Shaun Says:

    Right, Raul. I agree about Branch Rickey, based on everything I’ve read about him. Another reason why it’s appropriate to use a variety of methods–scouting and statistics, among others–to build a baseball franchise.

  8. Raul Says:

    Then again I suppose that same sentiment could be directed at Brian Cashman too.

  9. Shaun Says:

    I would say the front offices of the Red Sox and some other teams could beat the brainpower of Branch Rickey alone. We probably have more specialization in front offices these days, which probably means GM’s and other decision-makers get much more information. The game has changed not only on the field but in the front office. Just like it’s difficult to compare say Ruth and Pujols, it’s also difficult to compare Rickey to a modern GM with access to scouts who have access to tons of video and statisticians who have advanced degrees in mathematical fields.

    I think we can say Rickey is the Babe Ruth of GM’s in that he was the innovator and he was a great GM.

  10. brautigan Says:

    Branch Rickey was also a skin flint that hurt his team. I remember only too well Bill James parroting Rickey in regards to trading players before they reach their prime……and what genius trades Johnny Mize and Joe Medwick at the point in their respective careers (look what he got in return, the best part of the deals for the Cardinals was the money involved) like Rickey did. Those trades did not help the Cardinals, they basically hurt the team.

  11. Shaun Says:

    brautigan, well Mize only had one season in New York that matched his best seasons with the Cardinals (according to OPS and OPS+), although he was still very good or great for a while after he left the Cardinals.

    Medwick was not near the player he was with the Cardinals for the rest of his career, although again he was pretty darn good.

    Also, I don’t think we should ignore the money involved.

    But most importantly, those trades didn’t seem to hurt the franchise at all. They were a dynasty in the 1940’s after Mize and Medwick left. And by the 1950’s when the dynasty ended, Medwick had retired and Mize was just hanging on from 1951-1953.

Leave a Reply