========================================================================= ANALYSIS OF INSIDER ONTARIO LOTTERY WINS [For the CBC Fifth Estate episode of October 25, 2006.] by Jeffrey S. Rosenthal **UPDATE** (February 2014): An article about this analysis and the many subsequent developments, including links to numerous newspaper reports, is now available at: http://probability.ca/lotteryscandal/ (Dr. Rosenthal is a professor in the Department of Statistics at the University of Toronto, and the author of "Struck by Lightning: The Curious World of Probabilities". He maintains the web page probability.ca.) ** SUMMARY OF THE REPORT ** We consider the major ($50,000 plus) Ontario lottery wins, over the period 1999-2006. Our main findings are as follows. * Owners/employees of stores selling lottery tickets make up no more than one part in 247 [or, using an "alternative" baseline count as described below, one part in 148] of the adult Ontario population, and represent no more than one part in 165 [alternatively one part in 100] of the total amount spent on Ontario lottery tickets. * This means we would "expect" them to win no more than about 35 [alternatively 57] of the 5,713 major Ontario lottery prizes awarded in 1999-2006. In fact, they won about 200 of these prizes, or about 5.7 [alt. 3.5] times as many as expected, or about 165 [alt. 143] more than expected. This suggests that some other lottery customers missed out on major lottery prizes that they had won. * The probability that this would happen by pure luck is less than one chance in a trillion trillion trillion trillion, which is absolutely impossible. Hence, store owners/employees won significantly more major lottery prizes than they possibly could by pure luck. This conclusion remains valid even under very conservative assumptions about how many store owners/employees there really are, and how much they really spend on lottery tickets. * For the "Instant" lottery games alone, the situation is even more extreme. We would expect the store owners/employees to win about 4 [alt. 7] major Instant prizes. In fact, they won about 60 of them -- about 15 [alt. 9] times as many as expected. The probability of this happening by pure luck alone is, again, absolutely impossible. * Focusing solely on store owners (as opposed to owners and employees together) again leads to similar results, with about 83 observed major wins when we would expect less than 26, an excess of about 57. It is again absolutely inconceivable that they arose by pure chance alone. ** DETAILS OF THE ANALYSIS ** NUMBER OF RELEVANT INSIDERS: The number of lottery retail locations is 10,300 (averaged by year). The number of "insider" employees per retail location is unknown. However, a random survey of 200 locations conducted by Fifth Estate indicates that the average number is about 3.2. The survey figures have a standard deviation (variability) of 1.65, leading to a Monte Carlo error (sampling uncertainty) of about 1.65 / sqrt(200) = 0.12. This means that, even given the survey uncertainties, we can be fairly confident that there is an average of less than 3.2 + 0.12 = 3.32, or rounding up, about 3.5 employees per retail location or 36,050 employees total. Some have questioned this figure due to different types of stores being involved (convenience, grocery, drug, etc.). Such confusion could be avoided if we knew which insider wins correspond to which store type, but unfortunately OLG has not yet provided that information. As an "alternative" count, we note that an OLG employee (Alex Campbell, Director of Business Optimisation) testified under oath at a trial that the number of relevant insiders is "50,000 or 60,000", though it is not clear how this figure was obtained. So, to allow for this possibility, we also consider the "alternative" that the number of relevant store employees/owners is 60,000 (the upper figure from the trial testimony). AMOUNT THOSE RELEVANT INSIDERS SPEND ON LOTTERY TICKETS: In the Fifth Estate survey, 136 of 200 owners/employees (68%) said they played the lottery. Of those 136, five declined to specify an amount spent on lotteries. Of the remaining 131, the average claimed amount spent per year on lottery tickets is equal to $476.31, with a standard deviation of $602.50, leading to a Monte Carlo error of $602.50 / sqrt(131) = $52.64. This means that we can be fairly confident that the average amount spent by the 131 owners/employees is less than $476.31 + $52.64, or $528.95, or rounding up, $550 per year. (It is possible that those five owners/employees who refused to answer spend more than this, but that would only have a small effect on the results.) The average annual amount spent by all 195 owners/employees (omitting the five who refused to answer) is then $550 * (131/195) = $370. Combining this with the above figure of 36,050 total store owners/employees leads to the conclusion that the total amount spent annually on lottery tickets by lottery store owners/employees is less than 36,050 * $370 = $13,338,500. Or, using the "alternative" count of 60,000 owners/employees, we then compute that the total amount spent annually on lottery tickets by lottery store owners/employees is less than $370 * 60,000 = $22,200,000. As an aside, we note that annual lottery sales in Ontario for this period (averaged by year) are about $2.22 Billion. Given an adult Ontario population (averaged by year) of about 8,900,000, this works out to $249.44 per adult per year. Thus, on average, store employees spend at most $370 / $249.44 = 1.5 times as much as the average adult on lotteries. EXPECTED NUMBER OF INSIDER WINS: The above analysis says that, using our survey figures, the fraction of annual lottery spending that is done by lottery store employees is probably less than $13,338,500 / $2.22 Billion, or one part in 166. Now, there were 5,713 redemptions of major ($50,000 plus) prizes in this 1999-2005 period. Hence, on average, we would expect the number of such prizes won by store employees to be less than 5,713 / 166, i.e. less than 35. Or, using the alternative count of 60,000 owners/employees, the fraction of annual lottery spending that is done by lottery store employees would then be less than $22,200,000 / $2.22 Billion, or one part in 100. Hence, we would expect the number of such prizes won by store employees to be less than 5,713 / 100, or about 57.1. ACTUAL NUMBER OF INSIDER WINS: In point of fact, there were 214 major prizes claimed by "insiders" during this period. (This is likely an undercount, as doubtless some of the insiders e.g. denied that they were insiders, or got friends to claim the prizes for them.) For 160 of these 214 prizes, we have more detailed information. In two of those cases the claimant status is known. Of the other 158, 148 are listed as store owners or employees. So, extrapolating that figure to the unknown claims, we conclude that the number of major wins by lottery store owners/employees during this time period is about 214 * (148/158) = 200. P-VALUE, THE PROBABILITY THAT THIS HAPPENED PURELY BY CHANCE: The question then becomes, what is the "p-value", i.e. the probability that the store owners and employees would simply get lucky and win 200 times instead of the expected 35 [alt. 57.1] times? To compute this, we note that the number of wins should approximately follow a "Poisson distribution" (introduced by the French mathematician Siméon Denis Poisson in 1837). So the question is, what is the probability that a draw from a Poisson distribution with mean 35 [alt. 57.1] would be at least as large as 200? I wrote a computer program "poisson.c" (in the C programming language) to compute this probability (and also confirmed all such calculations using the Mathematica software package). The conclusion is that for the figure of 35 expected wins, this p-value is equal to six chances in a 1 followed by 82 zeroes, which is less than one chance in a billion trillion trillion trillion trillion trillion trillion -- absolutely inconceivable. Using the "alternative" count of 57.1 expected wins, the p-value becomes 6 x 10^(-49), or less than one chance in a trillion trillion trillion trillion -- still absolutely inconceivable. INSTANT GAMES ALONE: Next, we consider the Instant lottery games on their own, to test whether they alone are also unbalanced in favour of insider wins. The actual number of major insider Instant wins in this time period was 53+12=65. Thus, as above, the number of major wins by store owners/employees may be estimated as 65 * (148/158) = 61. I do not have clear figures for the total amount spent by Ontarians and by store owners/employees on Instant games specifically, so I cannot compare them directly. Instead, I assume that owners/employees spend the same factor compared to the average adult as for overall lottery spending, i.e. no more than 1.5 times as much. The total number of major Instant wins in this time period was 577+97=674. Thus, using our figure of 36,050 total owners/employees gives an expected number of wins of 674 * 36,050 * 1.5 / 8,900,000 = 4.10 (about 1/15 as much as actually observed). Or, using the "alternative" count of 60,000 total owners/employees, the expected number becomes 674 * 60,000 * 1.5 / 8,900,000 = 6.82 (about 1/9 as much as actually observed). The p-value, i.e. the probability of observing 61 or more Instant wins given an expected number of 4.10, is then less than one chance in a one followed by 48 zeroes, or one chance in a trillion trillion trillion trillion. Or, using the alternative expected value of 6.82, the p-value becomes 1.8 x 10^(-36), or about one chance in a trillion trillion trillion. Either way, this is still an unimaginably small figure, i.e. it is quite impossible that the employees would have gotten this lucky just by chance. NON-INSTANT GAMES ALONE: If we consider just the NON-Instant lotteries, then the number of insider wins for those is about 200 - 61 = 139. Meanwhile, the expected such number is no more than (5,713 - 674) / 166 = 30.4, or using the alternative count, (5,713 - 674) / 100 = 50.4. Using these figures, the corresponding p-value then becomes 1.1 x 10^(-46), or about one chance in ten billion trillion trillion trillion. Or, using the alternative count, 9.1 x 10^(-25), or about one chance in a trillion trillion. Either way, it is unimaginably unlikely that even the non-Instant insider wins can be explained by pure luck alone. ANALYSIS OF THE STORE OWNERS ALONE: We now concentrate on just the store owners (as opposed to store employees). Of the 158 insider wins for which the insider's status is known, 61 are listed as store owners. As above, we extrapolate that figure to the unknown 54 claims, leading to a conclusion of about 214 * (61/158) = 82.62, or about 83, insider major wins specifically by store owners. We know there are about 10,300 lottery ticket sales locations. Probably most of these locations (e.g. convenience stores) have exactly one individual owner. And although some special cases (e.g. Loblaws grocery stores) may have multiple owners, we have been informed that no Loblaws owner has won a major lottery prize, so this fact may be irrelevant. In any case, to be very conservative in our assumptions, we assume an average of 2 owners per ticket outlet, for a total of 20,600 owners. The next question is store owners' annual lottery ticket spending. Our survey of 200 convenience store owners/employees happened to include 75 owners. Of these, 61 reported buying lottery tickets to some extent (while the other 14 said they did not). Of those 61, three declined to specify how much they spend on tickets. The remaining 58 each specified an amount. The average of the 58 self-reported yearly lottery ticket spending figures was $516.53, with a standard deviation (variability) of $580.96, leading to a Monte Carlo error (sampling uncertainty) of about 580.96 / sqrt(58) = $76.28. Hence, being conservative, we can say that the average annual lottery ticket expenditures of owners who buy tickets is probably less than $516.53 + $76.28 = $592.81. Thus, the overall average annual lottery ticket expenditures of owners (whether they buy tickets or not) is probably less than $592.81 * (61/75) = $482.15, or about $482. (This is about 482/370 = 1.3 times as much as the average of all owners and employees, and about 482/249.44 = 1.9 time as much as the average of all Ontario adults.) Putting this together, the total annual amount spent on lottery tickets by store owners is probably less than $482 * 20,600 = $9,929,200. The fraction of annual lottery spending that is done by lottery store owners is then less than $9,929,200 / $2.22 Billion, or one part in 223.6. Hence, we would expect the number of such prizes won by store employees to be less than 5,713 / 223.6, i.e. less than 26. Given an expected number of wins of 26, the probability of observing 83 or more wins is then given by 5.2 x 10^(-19), or less than one chance in a trillion trillion. This is, again, absolutely inconceivable. That is, even analysing just the store owners alone, and even assuming a generous average figure of two owners per lottery outlet, it is still absolutely inconceivable that the number of insider store owner major lottery wins arose by pure chance alone. COMPARISON TO OLG EMPLOYEES: For comparative purposes, we also consider the employees of the Ontario Lottery Gaming Corporation (OLG) itself. There are about 7,150 such employees (averaged by year). So, the number of major lottery prizes we would expect them to win is about 5,713 * (7,150 / 8,900,000) = 4.6. In fact, they are confirmed as winning 4 such prizes (plus there are two claimants of "unknown" status) out of the 158 for which records are available. Extrapolating this to the remaining prizes means the OLG employees won about 214 * (4/158) = 5.42, i.e. about 5 prizes. This figure is perfectly consistent with their expected number, 4.6. It is not surprising or suspicious at all. That is, for the OLG employees, the number of major prizes won is perfectly consistent with what we would expect from the laws of chance, and there is absolutely no evidence of anything being wrong. THE PROBABILITY THAT AT LEAST ONE FIFTH ESTATE VIEWER MISSED OUT: The average number of Fifth Estate viewers is, apparently, about 500,000. With 500,000 viewers, the probability that any given Canadian adult is watching is about 500,000 / 25,297,520, or about 1/50. So, assuming that 165 people (as above) have missed out on major prizes during this time period, the probability that all 165 are NOT watching is: (49/50)^165 = 0.03566995. So, the probability that at least one of them IS watching is 1 - 0.03566995 = 0.96433, or over 96%. Or, using the "alternative" result that about 143 people have missed out on major prizes during this time period, the probability that at least one of them is watching is: 1 - (49/50)^143 = 0.9443675 or over 94%. EFFECTS OF EVEN MORE STORE EMPLOYEES: A last-minute e-mail from the OLG claimed there are really 140,217 relevant insiders -- much more than the 36,050 that we estimated ourselves, or the 50,000 to 60,000 claimed in the OLG trial testimony. We believe the 140,217 figure to be significantly inflated, however we now analyse it as well. Using this latest OLG count, we obtain a figure for total amount spent annually on lottery tickets by lottery store owners/employees of about 140,217 * $370 = $51,880,290. The fraction of annual lottery spending that is done by lottery store owners/employees would then be less than $51,880,290 / $2.22 Billion, or one part in 42.8. Hence, we would expect the number of such prizes won by store employees to be less than 5,713 / 42.8, i.e. no more than 133.5. Given an expected number of wins of 133.5, the probability of observing 200 or more wins is then given by 4.9 x 10^(-8), or less than one chance in 20 million, still EXTREMELY unlikely. That is, even the implausible-seeming figure of 140,217 store owners/employees STILL does not explain the large number of insider major lottery wins. For the Instant category alone, using the inflated population figure of 140,217 insiders, the expected number of major insider Instant wins would be 674 * 140,217 * 1.5 / 8,900,000 = 15.93. Given an expected number of wins of 15.93, the probability of observing 61 or more Instant wins is then given by 6.9 x 10^(-18), or about seven chances in a billion billion, again absolutely inconceivable. So, the implausible-seeming figure still does not explain the large number of insider major Instant lottery wins, either. Furthermore, of course this last-minute OLG employee count does not in any way affect the analysis (above) of the store owners alone, or in any way explain the excess number of store owner major lottery wins. MORE ABOUT THAT FIGURE OF 140,217: As noted above, the figure of 140,217 lottery store owners/employees seems unjustified given the available data. Furthermore, even using this figure still does not explain the large number of insider wins. However, the question remains as to whether the 140,217 figure is at all plausible. One way to assess this question is as follows. If there were in fact 140,217 store owners/employees, then with at most 20,600 owners, this leaves at least 129,400 employees. By the above analyses, the amount spent annually on lottery tickets by owners would be no more than $9,929,200 (and even that figure is very generous, assuming an average of two owners per location), while the amount spent by owners and employees together would be about $55,500,000. So, we would expect the fraction of major owner/employee wins which are in fact specifically owner wins to be no more than 9,929,200 / 51,880,290 = 0.1913867, i.e. about 19%. In fact, the fraction of major wins which were specifically from owners is about 61 / 200 = 0.305, i.e. about 31%, or about 1.6 times as large as expected. What is the probability of this arising purely by chance? Well, out of the 200 insider wins, the probability distribution for the number of insider wins specifically by owners should be Binomial(200, 0.1913867). So, the probability of there being at least 61 such wins, by pure chance alone, can be computed using the "R" computer software, with the command: pbinom(61, 200, 0.1913867, lower.tail=FALSE, log.p=FALSE) which works out to 4.1 x 10^(-5), or about one chance in 24,000. While not "absolutely inconceivable", this is still extremely unlikely. We conclude that even with very generous assumptions, the figure of 140,217 lottery store owners/employees is not consistent with the fraction of major lottery wins specifically by owners as opposed to employees. This appears to cast further significant doubt on the claimed figure of 140,217. REMOVING THE FORMER EMPLOYEE ("TURNOVER") COUNTS: Another issue concerning that last-minute OLG figure of 140,217 is that it includes not just current but also FORMER employees (or, "turnover" of employees) of lottery stores, even though former employees are apparently not counted as insiders when compiling insider win statistics and thus should not be counted when computing expected values either. If we remove the former employee category from the last-minute OLG counts, then the number of owners/employees remaining is equal to: 4877*4 + 1937*5 + 1196*8 + 229*5 + 123*10 + 214*10 + 18*10 + 397*6 + 998*25 + 173*6 + 18*6 + 731*40 = 101,174, i.e. 101,174 owners/employees. Using this count, the total amount spent annually on lottery tickets by lottery store owners/employees would be no more than 101,174 * $370 = $37,434,380. The fraction of annual lottery spending that is done by lottery store owners/employees would then be less than $37,434,380 / $2.22 Billion, or one part in 59.3. Hence, we would expect the number of such prizes won by store employees to be less than 5,713 / 42.8, i.e. no more than 96.33. Given an expected number of wins of 96.33, the probability of observing 200 or more wins is then given by 2.0 x 10^(-20), or one chance in 50 billion billion, again absolutely inconceivable. For the Instant games alone, the count of 101,174 owners/employees gives an expected number of major insider Instant wins of 674 * 101,174 * 1.5 / 8,900,000 = 11.49. Given this, the probability of observing 61 or more Instant wins is then given by 1.2 x 10^(-24), or about one chance in a trillion trillion, again absolutely inconceivable. POSSIBLE UNDERREPORTING OF LOTTERY EXPENDITURES: One concern with some of the above analysis is that it is based partially on store owners/employees self-reporting how much they spend on lottery tickets. It seems likely that those selling lottery tickets would have an accurate sense of how much they themselves spend on them, and would not feel any "shame" in admitting the precise figure, so that their self-reporting would be fairly accurate. However, the possibility of underreporting still remains. To investigate this, Fifth Estate conducted a small additional survey, this time of the general population through random phone calls. Since we know (see above) that the average Ontario adult spends $249.44 per year, this provides a check of how much the general population tends to underreport their lottery spending. The small survey indicates that the general population only reports spending about $141.03 on lottery tickets, for an underreporting factor of $249.44 / $141.03 = 1.77. Now, both logic and certain anecdotal evidence (obtained while conducting the survey) suggests that lottery owners/employees probably do not underreport as much as the general population. In other words, just because the general population underreports their lottery spending does not necessarily imply that lottery owners/employees -- a rather different group -- do as well. However, we now consider the effect if they do. Such underreporting would mean that the amount spent by owners/employees on lottery tickets would actually be about 1.77 times as much as the figure used in the above analysis. How does this affect the results? The expected number of total insider wins would now be 1.77 times as much as before, or less than 1.77 * 35 = 61.95 {alt. 1.77 * 57.1 = 101.1]. The probability of observing at least 200 such wins would then become 5.8 x 10^(-44) [alt. 2.8 x 10^(-18)], either way still absolutely inconceivable. For Instant wins, the expected number of insider Instant wins would now be less than 1.77 * 4.10 = 7.26 [alt. 1.77 * 6.82 = 12.07]. The probability of observing at least 61 such wins would then become 5.2 x 10^(-35) [alt. 1.4 x 10^(-23)], either way again absolutely inconceivable. Finally, we consider combining this underreporting factor of 1.77 with the OLG's last-minute, apparently inflated count of 101,174 owners/employees (i.e., with the former employees omitted). That count would lead to an expected number of total insider wins of 1.77 * 96.33 = 170.50. The probability of observing at least 200 such wins would then become 0.0148341, or just over one percent. This is no longer "astronomically small", but it is still very unlikely, and much less than the standard statistically significant cutoff of five percent. So, even combining the general population's underreporting factor with the OLG's last-minute count still does not properly explain the large number of major insider lottery wins. For Instant wins, the expected number of insider Instant wins would now be less than 1.77 * 11.49 = 20.34. The probability of observing at least 61 such wins would then become 2.8 x 10^(-13) another absolutely inconceivable event. So, even combining the general population's underreporting factor with the OLG's last-minute count still does come at all close to explaining the large number of major insider Instant lottery wins. =========================================================================