Mirror, Mirror on the Wall - How good is CricketX, after all?

(An assessment of how the CricketX model has fared in picking winners_..)

Time for introspection. There are several features unique to CricketX. We are the only web-site to post forecasts of cricket matches, both static (pre-match) and dynamic (over-by-over). But are our forecasts reliable? Here are the bare facts. You, the thinking fan, sit in judgement.

The propreitary model of CricketX was first developed by Dr. Surjit S.Bhalla in his book, "Between the wickets: The who and why of the best in cricket" (1987). This basic model has since undergone many foliations that has seen it transformed from just an authoritative rating system to a more holistic model of forecasting. The core of the CricketX model is based on crimetrics (from: cricket econometrics). Crimetrics allows the use of sophisticated statistical techniques to adjust player performances according to the nature of the pitch and the quality of the opposition. For teams, it leads to the construction of batting and bowling indices that are vital inputs in the forecasting model.

The CricketX database covers every single international match ever played - now 1508 Tests and 1624 ODIs. However over-by-over scores is a comparatively new development. The run rate comparison statistic has featured more prominently since the World Cup in England in 1999. Our database tracks this for all matches beginning World Cup'99 - about 150 matches. The analysis of our predictions, would be based on this sample and will not take into account abandoned matches. We do not rate or predict matches involving non Test-playing nations like Kenya, Holland or Scotland.

There are three sets of results. One, for our static forecasts, the second for predicting the score at the end of the 50 overs in the first innings and third, for the dynamic winning probabilities in the second innings. Besides, there are forecasts on individual performances.

With the static forecasts, we got a creditable 69% right. This means that out of 100 matches, we have correctly predicted the outcome of 69 of them before a ball is bowled. For the World Cup, this figure was around 73%. Initially, we display two sets of probabilities on our home page, depending on which side bats first. The record also shows that the errors have been uniformly right or wrong for both the team batting first and batting second. This shows that the model is fairly robust.

For the first innings Dynamic Score Predictor (DSP), the standard error (SE) at the end of 10 overs is 10 runs. Till about the 25th over, the error stays around 13 runs. By the 35th over the difference between the predicted score and actual score at the end of 50 overs falls to 8 runs on average. Five overs from close, the SE is less than 0.5 runs. Not only the SE, the range of the errors too has progressively fallen. At the end of 15 overs, when the SE was 13 runs, the error in predicting is in the range of -18 to 31 runs. By the 35th over, the range narrows down to -11 to 22 runs.

Once the target is known and before the second innings gets underway, we have called it right 78% of the times. At the half-way mark, our record improves to 84%. This means that half-way through the second innings, we were able to correctly say whether the chasing team would win (51% probability of winning) or not 84% of the times. This percentage steadily increases to 89 by the 35th over.

Besides match forecasting, we also make predictions for individual players. Our predictions for the last Australia-South Africa series is still featured on our site. This followed our hugely successful maiden venture for the Asia Cup. You can read about it in "Lest we forget_" How did these predictions stack up?

The forecasts for the Asia Cup were spot on with most leading batsmen averaging within two runs of the forecasts.

Forecasting requires a sound crimetric (cri for cricket, metric for econometric) model. It's also a risky business. An inventory of expertise expertise and risk appetite probably explains why CricketX are the only ones sticking their necks out!