Cricket Performance Ratings - Opinions seeked

I'll have a look at it later.

Thanks

Interesting. I tend to regard the ICC rankings as a pretty good guide to merit, with a few significant caveats, the two largest being overall sample size issues and over weighting of historical performances for obviously deteriorating players (well, apparently obviously - one could argue that saying a player is past it is itself getting very caught up on small sample sizes).

this is so hard.

it's an idea i've been mulling over for some time as well.

i'm not suggesting that it's impossible to improve on the current way the data is analysed and compiled to provide better indicators of performance, but i still think that data alone isn't enough.

my problem is that so often the nature of dismissal as well as missed chances and unrewarded spells can all only be compiled from watching a match. and that batting is as dependent on the quality of the bowling as it is on a batsman's skill. I noticed you gave Brathwaite a high rating in the world t20 final, but what choice did he have, and how awful was Stokes' decision making. I couldn't give Brathwaite any more than a third of the rating points I would award Kohli for his semi final knock against Australia. I find that a huge factor in t20. **** or bust situations regularly present themselves and there will always be a risk that noise is mistaken for achievement.

i came to the conclusion that you'd need knowledgeable cricket fans, familiar with cognitive biases, to amend a technically derived score (the structure of which they understood) at the end of each match, and you'd need to keep an audit of amendment process to ensuring consistency.

I can see your point with Brathwaite. I was not sure about using that example as I was not happy he was given so many more points than Samuels.

The trouble with T20 internationals is there are so few games and the associate teams play the associate teams and the top teams play the top teams. Brathwaite is rated 11th in t20 internationals and 214th in T20 league games! Clearly that is not right but he only had a handful of matches in T20is.

Kolhi is an interesting one. Clearly an amazing player but 9 years with RCB and still no title despite thousands of runs. I think he is too slow in the powerplay and the middle overs when setting totals. He then explodes from 16 onwards. He doesnt seem to increase it much in the middle overs even if there are plenty of wickets in hand.

With ratings I do on horseracing they also sometimes produce strange results but as a whole they are a good guide. I find if you start adding human input you cannot repeat them in an unemotional way. But I can see how for cricket this might be required. Thanks for the feedback.

Very interesting post DS. I think all you really have are the data. Yes, there is a lot of noise and some serious sample size issues but as you clearly know and imply, there's no objective or reliable way of analysing performance otherwise. Clearly it is important to understand the limitations of the method used though.

The T20I thing is also interesting. I don't rate the ratings for them at all, because there are so few games played. The guys at the top might be the best players but I don't think there's much more than a vague tendency towards having any real idea. As you say, frog2, looking at league games with a far bigger sample size is probably more sensible but even then you're assessing a rather random format.

I agree the leagues are random and sometimes strange. But due to the number of games I think things tend to balance out. There is so much more data to go on but I cannot find anyone rating the players on the web. So for my T20 league batting you have a top 8 of:

1    AB de Villiers    56.9    Barbados Tridents
2    Andre Russell    53.6    Sydney Thunder
3    David Warner    52.5    Sunrisers Hyderabad
4    Kieron Pollard    48.8    Karachi Kings
5    Kevin Pietersen    47.9    Quetta Gladiators
6    Chris Lynn    47.5    Brisbane Heat
7    Virat Kohli    46.7    Royal Challengers Bangalore
8    Jos Buttler    46.6    Lancashire

Which I think feels reasonable.

I think the point about who is bowling to who etc is important. Maybe the algorithm needs to be more iterative to include this information once the initial ratings are worked out.

If I'm reading your ratings correctly then some of the T20 League batting ratings look miles out to me (they're the only ones I've looked at so far). The top ranked players seem reasonable but as you go down the rankings then they seem to deteriorate.

Chris Broad ranked at 68 with so many good batsmen rated below him e.g. Bairstow ranked 99, Stokes 106, Hales 156, Faf 198, Root 220. Common sense tells us that something's deeply wrong with these.

Just noticed Dernbach rated slightly higher Sam Billings!

higher than Sam

Broad got one high rating when he won a game for Hobart in the final over. I think he has only batted 3 times in T20 league games since 2014.Thanks for pointing that out. I think there needs to be a min amount of matches to be rated.

Bairstow has had one T20 league win in the last 13 matches he has played in. His last good performance was 92 for Yorkshire off 42 balls in July 2015. His win rate in League matches is 38% since 2014

Stokes' recent scores are 18, 56, 1, 11, 5, 40, 19, 33, 4. Average of 19 since 2014 with a win rate of 45%

Hales averages 23 for Notts which as an opener is not great.

Faf averages 22 in 48 matches with a strike rate of 121. He had a terrible CPL.

Root has only played 5 matches for Yorkshire and they lost 4 of them.

.....


The algorithm can only go on the data supplied but I see your point. These are class international players. Should the algorithm combine league and international performances?

Initial thoughts were that combining data from different leagues is quite difficult as not all leagues are equal.

Apologies will poster later as watching footy now.

Dernbach had 3 bats only. I think you are right there needs to be a min amount.

But Billings is only averaging 17 since 2014
http://www.cricsq.com/players/sam-billings-t20l

How many match winning innings has he played? 78 off 50 v Quetta. 13 matches for Kent last summer he got 142 runs off 129 balls. Top score 55 from 30 when the team had a solid base.

Bowlers that dont bat much need to be taken out I think. But some of the batsmen actually dont have that great a record in T20 league matches.

Half-time in footy.

You're covering quite a lot of things very quickly here and need to think about them.

Another early thought I had is what would you do with the ratings? Horse ratings I can see as used to do my own speed ratings for AW meetings only. That was initially why I joined Betfair.

If you had two teams where one team had better ratings for every player than the other then I'm not at all certain that I would back that team. Odds, whose batting first, pitch, weather, need to win, recent form, individual match ups, etc, so many other things to consider than just player ratings.

Not trying to be negative just throwing some thoughts onto the pitch.

Not negative at all. Very useful thoughts.

What I have been doing is putting the ratings for all the players, pitch info (spin/pace ratings, average winning 1st score etc), who bats first, and so on into a neural network with the target value the result (win lose (or draw)). So I have rating histories use.

Have to wait for the toss and then run it to give a probability for each team. Then look for value in the odds.

As mentioned you have a sample size issue. In addition, the majority of dismissals are caught which is beyond the control of the bowler (as are stumpings and runouts). Dollies are dropped and crackers are taken. And edges behind sometimes hit the stumps and sometimes go for boundaries. In combination, these factors will give a biased picture of both bowling and batting proficiency. What I would like to see is a weighting for how good dismissal chances are irrespective of whether they are taken or not. Although I suspect that is too difficult.

You say you use a Neural Network, but don't say how. What does a NN give you over a simple summing and averaging of individual players performance?

Otherwise, I'd say you are on the right track, and it would be good enough to give you a betting edge. How much of an edge I can't say.

What the NN does is slightly a black box. But the batting order is set by the average historical position to go to 1 to 11. So in T20 the NN will probably give less weight to batsmen further down the order (I would hope it does anyway!). This to some extent gets rid of the problem mentioned above of Broad and Dernbach getting too high ratings. For bowlers it uses average overs bowled for T20 so it should, for instance, not weigh much for Joe Root's bowling.

Just averaging the totals would not do this.

The NN also gives the chance to set the target value when building the model so you get a win probability from it. With averaging you just have, say, Team 1 rating 42, Team 2 rating 25. But what should the odds be?

What the NN does is slightly a black box.

As we say in the business, 'Magic happens here'.

How much value are you awarding for "winning" a game? Only I think I'd give roughly equal weighting to a guy getting 30 off 12 setting a total that happened to get overhauled off the last ball versus the same game where the final ball was a dot rather than a 4 so the defending team won. The Braithwaite innings getting that many points smacks a bit of results merchantry. What matters in terms of prediction for future games is the skill that was displayed (as far as you can fathom that from what happened in a match, which may not be all that far), I think, rather than the eventual match outcome. You say something similar about Kohli not winning the IPL but as one of 22 players on the pitch each time, I don't think that's the way you'd work out how good he is.

Just averaging the totals would not do this.

So weight them.

Personally, I'd use a Monte Carlo simulation.

Just another quick observation. On your T20I batting rankings you have Afghans ranked 6 and 7, a Hong Konger at 9 and a Jock at 15. If you ask anyone to name their best twenty T20I batsmen I doubt if they would include those players. I imagine that the NN doesn't take in to account the strength of the opposition so high rankings can be achieved by performing well against relatively weak opponents.

Good point Charlie. The algorithm needs a second level to take into account who you are playing against. Who is bowling or batting. The major problem is in T20 the associate teams generally just play against themselves and the top teams play against themselves.

They are hard to compare and then when they do play you get results like Afghanistan beating the West Indies.

For the t20 team ratings I split them into associate and full members otherwise Afghanistan would be rated 2nd. Maybe I should have two lists for the player ratings as well?

Fatslogger,

From what I can make out the machine is doing it does not rate first innings performances on the final result. Scoring 30 off 12 balls in exactly the same situation in the first innings would get the same rating regardless if the team beating second won of lost.

Batting second it would be dependent on keeping up with the required rate and getting the team across the line.

pbx,

The NN does basically weight the inputs. But it works out those weights from historical data rather than having to guess the weights by hand.

Montesim is something I have tried a few times but I seem to always get slightly strange odds forecasts.

Is there a difference in the inputs to the net depending on whether it's say a T20I or a T20 league game?

Yes. The same algorithm is used for all T20 matches. The reinforcement learning component that works out the inputs has formed 132 inputs based on the match situation.

Currently it does not take account of opposition. Looking at the league team ratings (which is just average ratings of the last 3 teams fielded) IPL teams are in the top 5 of domestic teams which implies the IPL is the best league in the world.

Very useful feedback. A fair bit of revision is needed to iron out the flaws.

Is there a different algorithm (different net?) for calculating Team Rankings and Player Rankings?

Presumably the players are some of the inputs? For a T20I if you only have one set of player rankings (as you currently do) then surely associate team players will be overrated when playing against a top team?

If you aggregate each team's players (maybe just a simple sum) does the aggregate correlate with the Team's Ranking?

There is several layers to it.

So firstly you have player batting and bowling ratings for each match. These are a rough guide to impact each player performance has on the match outcome.

Secondly you have the master bowling and batting ratings for each player for each format. This is calculated from previous match performances for each player.

Then you have the teams. The team ratings do not use NNs at the moment. The teams are the average of the sum of the master ratings for players who played for that team in the last 3 matches they played. Generally this seems to look ok but you get situations where Australia play a weak team V Sri Lanka for a 3 match T20i series so they have a low current rating.

On the day of each match I use the players in the actual team playing rather than the team ratings on that website to calculate probabilities.

To your last post. Yes. That is how they are calculated at the moment just using the individual players. Yes on the rare occassion like a world t20 when associates play full teams it would not make sense.

These are very clear issues. Do the test numbers (where this issues shoudl not exist) look weird to you as well?

Have only looked at T20 so far!

How do you deal with player's making their debut?

Does DL screw things up?

Retrospective DLS messes up slightly but it seems to manage out as it uses target and balls left etc as inputs. So this game for instance:

http://www.cricsq.com/match-ratings/psl-2017-02-17-islamabad-united-v-karachi-kings
The ratings look okish I think

Debut some as for horseracing I used have one input as 0 for debut and 1 for not debut.

I have to go out now. But thanks for your advise.

frog - I like the idea of trying to rate players in order to gain an edge. However I think horses/dogs and humans are light years apart given the variables.

Kohli's lack of runs versus Aus compared with his glut against England is purely a form issue. As you will know form is an oddity. An average player in form can be better than a legend out of nick.

May I suggest you look at the IPL and study the markets and their reactions? Markets rate certain players/teams and oppose others. This is very clear to observe but I'm ashamed to say I've never written it down.

So the markets are doing the rating for you, especially in T20s. They rate certain bowlers (say Narine) and love AB or Gayle. Even a leg bye from Kohli will influence the price!

Obviously I'm talking about trading prices rather than fixed oddds betting so this may not appeal.

I've brushed over some of the above points very superficially and there are several which I'd like to investigate further but for now I'me had a quick look at your test batting rankings.

The first one that stood out as a bit peculiar was Sabbir Rahman ranked at seven. So I drilled down into the ratings you gave him and noticed that out of the last 10 ratings eight were negative, one was 2.1 and the other was for 1142.6. Digging a bit deeper I see you gave the 1142 rating for top scoring with 65 not out against England. Moeen top scored with 68 for England in first innings and was given a rating of 56. No matter which way I look at it I can't see any reason for the difference in ratings awarded it just seems bizarre to me. If you take out that particular rating for Rahman what would that do to his ranking?

So how ratings are calculated would be a concern for me. Plus maybe you need to look at a player's ratings and apply some sort of normal distribution disregarding highest and lowest scores.

I wouldn't expect batting scores to be a normal distribution. I'd expect a more bimodal distribution, with a bunch of low scores and a smaller bunch of bigger scores. Although it could vary quite a bit by player. Some players seem good at getting to 50 and then out.

I searched for this kind of statistical analysis some time ago without success.

Charlie,
Thanks for the test feedback. Immediately I asked I saw Rahman stood out as a bad rating. I think it is the case again of one performance having too much influence on the overall rating. You are right on this for sure. The same size is too small like Broad and Derbach.

But for the match itself would you really give Ali the same rating in the first innings of a match as Rahman being 64 not out in the fourth innings almost getting Bangladesh up to win the match?

I didn't mean to imply that a player's scores would follow a normal distribution was concerned that one or two very high scores would produce an unrealistically high rating for that player so maybe there needs to be some way of excluding these scores.

I agree. It is disappointing that the NN is not down rating one off good scores. I can only assume it has historically seen players after only a few matches getting one or two good ratings and going on to do well. Or it is just not picking up.

I need to reverse engineer what the algo is doing more.

It's a bit subjective for me to say whether Rahman's knock was better than Ali's - that's what software is for! It's the difference in the ratings that is concerning me.

I notice that Steve Smith got a rating of 190 for scoring that century in India. I certainly don't believe that Rahman's knock was six times better than Smith's!

Injera,
using the markets themselves is surely going to include the bias for the players the market thinks are the best players. For instance the market always over rates players like Perera and Stokes.

Frog, I doubt you are the first to try to apply the kind of statistical analysis that works for horses on cricket. I suspect Peter Webb gave it a try.

And FWIIW I think Winviz has done a decent job. Which is not to suggest you can't do better.

Charlie,
The ratings are about how much influence the player has on the match outcome. Rahman almost got his team to victory. Australia were 155 ahead in the first innings when Smith came out to bat. India only got 107 in second innings. Other players chipped in for Austalia's 285. So how did Smith innings contribute to winning the match?

I see your point though. Smith is overall top rated for the test batsmen.

frog - indeed. And if one trades there lies the opportunity. Narine comes on lay the batting side, Stokes walks out, lay the bowling side..

That's crude I know but I've found the best way to trade is to ignore the cricket and watch the market. The inherent bias is incredible.

Winviz is an interesting project but it is based on 60s montecarlo tech. They look at inplay as well which is extremely difficult and I cannot see how it can ever be better than just giving the implied probabilty from betfair odds.

Injera,
Good to see I am not the only person that cannot understand the $2m+ for Stokes!

Narine is so far ahead of the rest in T20 bowling its crazy.

Smith was the glue that held the innings together. He faced a lot of balls in a very tricky situation if he had got out early then India may well have won. But again it's subjective.

I'm thinking about "influence on a match" but not for much longer as want to watch the darts.

Winviz is an interesting project but it is based on 60s montecarlo tech. They look at inplay as well which is extremely difficult and I cannot see how it can ever be better than just giving the implied probabilty from betfair odds.

Inplay is not much harder than preplay with Monte Carlo and statistical models. If a player has scored x runs, you know how much he should score statistically. Statistics haven't changed since the 1960s.

What I know about NNs is from a long time ago. Sometimes people who take novel approaches come up with better answers. So I wish you luck.

I'm thinking about "influence on a match"

There is a saying in scientific circles,

'If you can't measure it, it doesn't exist.'

frog
In the Bangladesh-England Test Match where Rahman got his big rating if we imagine that the Bangas two innings had been reversed i.e. the actual first innings now becomes the second and vice versa would his rating still have been the same?

No. The ratings are generated relative to the match situation.

So in the reverse innings idea even though they've contributed exactly the same amount of runs their rating differ depending whether they score them in the first innings or second with the second innings being weighted more heavily than the first?

Its to do with the impact on the match rather than the innings number.

For instance;

If both openers got 50 not out in the second innings chasing 100 they would not get as much rating as a 50 in the first innings when the number 10 batsman went in at 104-8 and helped the team get up to 200+.

I think i know where Charles is going with this..Lets see..

frog
It's your model so don't want to labour the point. If you want I'll try and look at bowling ratings but have a suspicion they will be harder to tweak.

How do your (NN) findings compare to actual odds and results?

Hi Charlie,

I have been going over them all again after your (and some others) input. I am fairly happy with the match by match ratings as impact on the match outcome.

I am not happy with small sample sizes and how they work for the player master ratings. Areas of concern are when a tail endder wins a match in the last over and that has a high impact on his overall rating.

I am looking at getting more inputs or just displaying more information that could be useful. For instance Chris Lynn has a strike rate of 170 against seam and only 107 against spin in 55 t20s since 2013. This is the kind of information I have never had to hand before.

In terms of NN model for actually predicting probs compared to odds there is a margin to be made for t20s but it is not currently huge. I think it can be with more work. It really needs to be done after the toss and the teams are known.

I hope to have some revised ratings based on more inputs by the end of this coming week.

I have made alot of changes to the rating following the advise.

The main ones are:

1. T20 ratings are combined for leagues and internationals.

2. The rating of the team you are playing is taken into account for the match ratings. This means no more Hong Kong and UAE players in the top ten! It means a good performance against Ireland gives less points than a good performance against India.

3. Have to have faced or bowled 200 balls in the format in the last year to get a rating. This stops people like Broad getting a high T20 batting rating for one good performance.

4. The team ratings are now based off team performances rather than the individual players. They are formed from 3 years for international and 1 year for T20 Leagues.

Again I would be grateful for any feedback and opinions.

Thanks.

Also added some charts to the player pages.

Hi Frog hope you're well.

Had a superficial look at new T20 ratings and they do seem better than previously, well done. A question I was going to ask but never got around to it is the comparison between batsmen and bowler ratings. Your batsmen ratings are much higher than than the bowlers, does this mean that batsmen are considered more important than bowlers?

Hi Charlie. All good thanks.

It seems that Bowlers are important in Tests but in one day and T20s the batsmen are more important. It is easier to be consistent as a batsman in white ball cricket than be a bowler.

Most of the top T20 players bat in the top 3 and have a chance to have a big influence on the whole match. A bowler has a max of 24 legal balls.

Good look butyYou're trying to plait Mercury - cricketers form is more diverse than horses owned by JP Mac.

*but you're

Frog2, don't dissapear on us. I'm interested in how it works out.

I put IPL batting and bowling previews up.

The pace versus spin strike rates are interesting.

For instance:

Chris Lynn 170 against pace and 107 against spin.
Dwayne Smith 109 against pace and 141 against spin.

These are on all T20 balls faced since Jan 2014.

Frog
Enjoyed your preview of IPL good stuff (Duminy's apparently not playing).

Gareth Batty ranked 33 for Test match bowling seems generous based on two games mainly because of rating of 499 for three wickets against Bangas.

Broad ranked 92 and Finn 95 for Test batting seem very high compared to most of the players below them (is system still overrating bowler's batting performances?).

Is it much trouble to have names in Surname order as I find it a pain when sorting on first names? Also could the number of games a player's rating is based on be shown?

Titles with T20I should ditch the I now (it confused me).

Not sure about my last post all games rated are for Internationals. Thought you'd combined all T20 games?
T20 batting David Lloyd! Server error - is it Bumble? Quite a few at bottom of rankings give an error.

Hi Charlie,

Thanks for all that feedback.

Yes there is a server error as rating cards go back to 2014 but player pages only one year. I need to address that when back from holiday. Also the t20i needs to be changed.

I will change the sort order when I get a chance. A player search function would be useful as well because currently no way to quickly find a player.

Duminy, de Kock, Rahul and Kohli for the start of the IPL are out so far. I have updated the article.

Thanks again

Looking at the SH v RCB page. It's hard to digest all those numbers and I feel more summarization is needed. Although, you'd need to be guessing the starting 11.

I agree pxb. Its just an automated stats page. Will be writing a text preview for each game.

Excellent stats, Frog2, thanks. I'm probably being a bit thick here but what is MO strike rate and DO strike rate on the player's ratings and where do I find the stats for the diff batsmen against spin/fast bowling, pls?

Hi,
The split the 20/20 overs
PP (powerplay) 1st 6
MO (middle overs) next 10 overs
DO (death overs) final 4

In ODIs
its PP first 15
MO next 25
DO is last 10.

I haven't put the strike rates v spin and pace on the players pages as already a lot of data there. Do you think they should go up?

Thanks, and yes, put everything you can think of up! I love your stats, I think you've done a great job and the more info I think the better! It doesn't really have to be that useful, just so long as it's quite interesting as far as I'm concerned!

I have to be honest I used the stats to see if it works to bet on two matches...SL vs Bangla and SRH vs RCB and in both the cases it didnt work out..I am back to me old strategy..see the game bet the game rather than using statistics..

@Steam If stats worked like that everyone would be winning. Assuming you placed the bets purely on the basis of stats, two games is not enough of a sample to reject the usefulness of stats.

I agree with you regarding staying wtih your old strategy. But sometimes it might be easier to see the value in your bets if you have the stats?

There are stats, and there are ratings. It's far from clear to me where Frog's stats end and his ratings begin.

Secret sauce.

The stats is are just raw data. You can draw your own conclusions from them.

@detraveller yes you are right , key is do the performance lead to stats or stats are defining the performance. You are right over the period of time it might work out but never let go off the game in front of u. Like yesterday while the avg score was around 160 and SRH ended up 207. I for one like mugs thought there was value in laying 190 at 3 without actually seeing how good the surface was playing . Lessons learnt

As long as you learn from your mistakes(without going broke).

Not a huge fan of stats but some are definitely useful and can't be ignored.
Imo you should be using stats to try and get an idea of what could happen and not just follow them blindly, assuming that it will happen

the only stat u need is back the chasing team in T20

I see you've added an IPL Betting Diary. Nice winnings, well done.

Charlie I would love to read your IPL betting diary

Charlie, I hope you will vouch for me that the diary is true and I have not changed the selections. A long way to go this IPL to return a profit at the end. it is my first experience of betting on cricket.

Vry nice diary

Frog
I'd certainly vouch for you but don't see the need as you've got the tips in your preview.

The only thing I'm a little uneasy with, as mentioned previously, is it's not clear how your system derives bets/odds for the match. Still appears like manual input from you and not system derived.

If I created a test team from your top 11 I'd have a 9 man team. You sure you are not missing a criterion there?

Plus we’ve still not seem Charlie’s IPL betting diary.

You want my IPL diary?

It's short and red.