Basketball Model Methodology & Changes
How the forecast works
The forecast uses game results to create an ELO rating, and then the season is run 10,000 times do get an average result. The reason it is run so many times is to get as many possible seasons as possible. I will go further in-depth on that later. It then averages the results, to give us the final probabilities.
Elo was created by Arpad Elo, a Hungarian-American physics professor. The index was first created for chess, but has been adapted for multiple sports. I found the win percentage of each game by the difference of ELO What makes ELO a good rating is that ELO points never disappear or appear. The dispersion of points always averages to 0. The winner takes from the loser, and it is that simple.
I created my ELO rating by taking the win loss records or each team’s preseason schedule. The win percentage was then inverted into an ELO, by finding the difference of ELO based on that percentage. The other factor strength of schedule index from MaxPreps.com. The strength of schedule was multiplied by 10 to get an ELO difference. Those two factors are added together to get the total ELO difference. The ELO difference is added to 1300 to get a composite ELO. Since the I had very little data to go by, I knew the starting ELO’s were going to be very volatile and not truly represent the teams talents. So in order to get a little more stable ELO rating I mean reverted the each team by 1/3. In statistics noise and signal are often used terms, signal represents the true population or data, but signal is smaller movement that if read incorrectly, could mess up the calculations. I reduced the noise from the data given to me by doing this regression, and made the teams starting ELO closer to each other. This was incase I over or under-estimated each team’s true ELO or skill.
After each game ELO is recalculated based of the results and the margin of victory. Each game has a predicted win probability based off of the ELO difference. The predicted win equation is an exponential formula that I adjusted from the most used formula. The winner of the game takes x amount of points away from the loser. If a team is predicted to win they will take fewer points from the loser, but if it is an upset they take more points than they would lose. I used a K-value of 60 which is the maximum number of ELO points that can be traded in a game. For example Team a has a 60% chance of winning, and they win they would take 24 points. The formula is: 60*(1-win percentage). So the inserting of the results it is 60*(1-.6)=24. The next part of calculating the change in ELO is the margin of victory. The win percentage is converted into a spread first. Then the spread is matched versus the margin of victory. And the difference in the spread, is the amount of ELO points traded between teams. I capped the exchange amount from the margin of victory at 20, in case of a miscalculation of ELO or spread. It creates less volatility in the ELO rating.
The other adjustment to ELO is home court advantage. I put home court as a 25 point advantage. While in most games is might make a 1 or 2 percentage point difference in win probabilities, when the team’s ELOs are very similar, it could change by 5 or 6 percentage points. This was calculated from the previous seasons score differential at home and away versus the same opponent.
Simulating the Season
The season is the hardest part to calculate. I calculated the win percentage of the home team of each game, 56 games in total. Each game is simulated by a random number, and if the random number is greater than the win percentage, the away team is calculated as the winner, but if less than , the home team is the winner. This calculated for every game of the season. The season is simulated 10,000 times to fulfill the law of large numbers, which states the larger number of trials, the closer to the expected value you will get. Below is a visual of this and how the observed value becomes closer to the expected values.
The final step of simulating the season is the ranking. The model ranks each team by win percentage for each season. Tie breakers are random, as ties are broken by coin flips or a re-match. If the teams that are tied, did not beat each other once, the team that got the two wins in head-to-head will win the tie.
The model is probabilistic, not definitive.
The biggest misconception of statistical models, is how they are viewed. They are not definitive, or 100% confident. The media has had a big problem of this, ie. the 2016 Presidential Election. We assume something is going happen just because it is favored to happen. So a good example of this, and all probabilities, is a deck of cards. So you have a 1/13 chance of drawing an Ace, however, if you drew two in a row the probabilities would not be off, just a rare occurrence.
And if the model is correctly designed the favored team would not win every game. If it is correctly calibrated the underdog would win every once in a while. For example, if all matchups for a day were 75-25 (there are four games in this day.) If the model was correctly calibrated the favored would win around 3/4 (75%) of the games. The sample size is very small, it could easily be 100% of the games or 50% of the games. But if the underdog is not winning much, then the model might have a few flaws under the hood. Errors in the model are usually correlated, but ELO will iron them out throughout the season.
Changes to Model
I usually don’t like to change my model throughout the data collection period (the season). But there are ones that needed to be made.
The first one was city champions. At first I had it calculated for the best district record, and not the best city record. This change had no effect on the model, just the final data.
The next was the margin of victory index. I felt a complete ELO index would take into account the margin of victory. So how it was calculated was I took the win probability of the underdog and created a spread. This was calculated by an algorithm I regressed from previous data. Then I took the margin of victory and found the difference between the two. So if a team was favored by 8 points and won by 13, their margin of victory index for that game would be 5. That is then added to the ELO change based off of win percentage. I capped the margin of victory index at 20 points per game. So if a team beat or lost by more than 20 to their expected results, it would stop at 20. Below is a graph of the New calculated ELO v. old ELO. The biggest difference is Molina, their starting ELO was a bit over estimated, so the margin of victory index helped get their true ELO sooner, rather than later.
Thanks for all of the support, and reading.
This has been very fun, but stressing. If you could somehow get this out to more people, that would be great, but I am glad that you are reading this. If you have any queestion feel free to reach out to me on twitter, @jhkersting or email me at firstname.lastname@example.org