The PFF raters have a defense to that argument. They say that they don’t rate the performance if the assignment is unclear and (2) there are over the course of a season a large enough sample of plays where there is no issue about the assignment to make a valid evaluation of the player.
I thought they published ratings on a game by game basis.
Yes, one game can be misleading, but over the course of the season you get a large enough sample size to make a reliable evaluation of the player— at least that’s what they say.
I thought they published ratings on a game by game basis.
Yes, one game can be misleading, but over the course of the season you get a large enough sample size to make a reliable evaluation of the player— at least that’s what they say.
I know this stuff is not really scientific. But as a retired engineer, I have a real bias towards the rules about publishing scientific papers. In an ideal world you must include complete details on all of your source data and how you calculated each result. Done for peer review.
All the papers I see published this way and do not have all this are highly suspect IMO.
I don't believe PFF has ever done anything like that. If you don't, it is easy (and meaningless) to say "They say that they don’t rate the performance if the assignment is unclear and (2) there are over the course of a season a large enough sample of plays where there is no issue about the assignment to make a valid evaluation of the player."
Yes, one game can be misleading, but over the course of the season you get a large enough sample size to make a reliable evaluation of the player— at least that’s what they say.
I know this stuff is not really scientific. But as a retired engineer, I have a real bias towards the rules about publishing scientific papers. In an ideal world you must include complete details on all of your source data and how you calculated each result. Done for peer review.
All the papers I see published this way and do not have all this are highly suspect IMO.
I don't believe PFF has ever done anything like that. If you don't, it is easy (and meaningless) to say "They say that they don’t rate the performance if the assignment is unclear and (2) there are over the course of a season a large enough sample of plays where there is no issue about the assignment to make a valid evaluation of the player."
Actually , if you go on their website, there’s an article on what they do and how they do it. I read it a few years ago, but I assume it’s still there.
I know this stuff is not really scientific. But as a retired engineer, I have a real bias towards the rules about publishing scientific papers. In an ideal world you must include complete details on all of your source data and how you calculated each result. Done for peer review.
All the papers I see published this way and do not have all this are highly suspect IMO.
I don't believe PFF has ever done anything like that. If you don't, it is easy (and meaningless) to say "They say that they don’t rate the performance if the assignment is unclear and (2) there are over the course of a season a large enough sample of plays where there is no issue about the assignment to make a valid evaluation of the player."
Actually , if you go on their website, there’s an article on what they do and how they do it. I read it a few years ago, but I assume it’s still there.
But what I was referring to allows a peer reviewer to examine the work on an observation by observation basis. Allowing for criticism on an extremely granular level.
Actually , if you go on their website, there’s an article on what they do and how they do it. I read it a few years ago, but I assume it’s still there.
But what I was referring to allows a peer reviewer to examine the work on an observation by observation basis. Allowing for criticism on an extremely granular level.
As a person who used to teach experimental design to graduate students, id say you’d need more than that. How do we know that the peer observer is doing a good job of implementing their methodology? I don’t recall whether they they have more than one person view the same game film to do the ratings. But I vaguely recall that they do have a quality control procedure.
But what I was referring to allows a peer reviewer to examine the work on an observation by observation basis. Allowing for criticism on an extremely granular level.
As a person who used to teach experimental design to graduate students, id say you’d need more than that. How do we know that the peer observer is doing a good job of implementing their methodology? I don’t recall whether they they have more than one person view the same game film to do the ratings. But I vaguely recall that they do have a quality control procedure.
PFF is next to meaningless b/c they do not grade based on scheme or what the scheme asks the player to do or how to do it. They also don't know the play call so they on a 2nd level don't know what the player is asked to do.
This means that any value they have is only to a team that can look at their grades on a game by game basis and use it to look at their own grading. Anyone using PFF to decide if a player is good or not is being led by the blind.
As a person who used to teach experimental design to graduate students, id say you’d need more than that. How do we know that the peer observer is doing a good job of implementing their methodology? I don’t recall whether they they have more than one person view the same game film to do the ratings. But I vaguely recall that they do have a quality control procedure.
PFF is next to meaningless b/c they do not grade based on scheme or what the scheme asks the player to do or how to do it. They also don't know the play call so they on a 2nd level don't know what the player is asked to do.
This means that any value they have is only to a team that can look at their grades on a game by game basis and use it to look at their own grading. Anyone using PFF to decide if a player is good or not is being led by the blind.
I know your opinion has been well-established for years. If it were totally worthless, NFL teams wouldn’t be wasting their time and money on it. It’s not perfect. Nothing is perfect except God.
I’m pretty sure players are not “asked” to whiff block, miss tackles, or drop passes.
PFF is next to meaningless b/c they do not grade based on scheme or what the scheme asks the player to do or how to do it. They also don't know the play call so they on a 2nd level don't know what the player is asked to do.
This means that any value they have is only to a team that can look at their grades on a game by game basis and use it to look at their own grading. Anyone using PFF to decide if a player is good or not is being led by the blind.
I know your opinion has been well-established for years. If it were totally worthless, NFL teams wouldn’t be wasting their time and money on it. It’s not perfect. Nothing is perfect except God.
I’m pretty sure players are not “asked” to whiff block, miss tackles, or drop passes.
David, re read what I said and comment on it. =)
What teams get out of PFF, has nothing to do w/what random people outside of the org can do with it. Again we, nor PFF, know WHAT they are asked to do, and they do not just grade on the obvious; if they did, they'd be no better then random drunk fans watching on TV.. They've stated as much. Keep trying to defend that terrible org though, it's glory days are well past and they haven't even been around that long.
PFF is next to meaningless b/c they do not grade based on scheme or what the scheme asks the player to do or how to do it. They also don't know the play call so they on a 2nd level don't know what the player is asked to do.
This means that any value they have is only to a team that can look at their grades on a game by game basis and use it to look at their own grading. Anyone using PFF to decide if a player is good or not is being led by the blind.
I know your opinion has been well-established for years. If it were totally worthless, NFL teams wouldn’t be wasting their time and money on it. It’s not perfect. Nothing is perfect except God.
I’m pretty sure players are not “asked” to whiff block, miss tackles, or drop passes.
I read the PFF data and grades with a big grain of salt. For me it is not a definitive measure of a player's performance, but it is a source of information that I like to consider. Sometimes the grades don't pass the eye test. But sometimes it affirms what you believe you are seeing in a player's performance. As you say, it's not perfect.
I know your opinion has been well-established for years. If it were totally worthless, NFL teams wouldn’t be wasting their time and money on it. It’s not perfect. Nothing is perfect except God.
I’m pretty sure players are not “asked” to whiff block, miss tackles, or drop passes.
David, re read what I said and comment on it. =)
What teams get out of PFF, has nothing to do w/what random people outside of the org can do with it. Again we, nor PFF, know WHAT they are asked to do, and they do not just grade on the obvious; if they did, they'd be no better then random drunk fans watching on TV.. They've stated as much. Keep trying to defend that terrible org though, it's glory days are well past and they haven't even been around that long.
I understand your point, but I will repeat that you don’t need to know the play called or the assignment to see a guy miss a block, whiff a tackle, drop a pass, sack a QB, fumble the ball, holding penalty, false starts, etc., etc.
The plays you are talking about where a receiver runs the wrong route, a lineman forgets his assignment, etc. are only a subset of plays where you could give a wrong grade because you don’t know the “ask.” I believe the PFF people have addressed the issue, and say they don’t grade the player on a play if there is clearly some ambiguity about his assignment on the play. On most plays, it is not hard to see what he is trying to do and hard to imagine him “asked” to do something different. It varies by position.
As a former professional sociologist trained in multivariate statistics, I would like to know how much “predictive validity” a player’s PFF grade has. For example, how well is it statistically correlated with other events, such as:
>Moving up/down on the depth chart. >Getting cut before next season > Salary and contract >Getting voted to Pro Bowl >Getting a contract extension after rookie contract expires (I especially like this one.)
If there is a statistical correlation at the .01 confidence level, I would conclude that PFF score correlates pretty well with playing ability – not necessarily for a single game, but over a season.
