Writing a Sports Prediction Program
07-05-2013
, 06:57 PM
Set a very basic milestone like
1) Load all data into the DB
2) Write a Python script that lets you query the average minutes played by a given player (or whatever simple stat you want)
Then go from that. Some ideas for next steps:
- Automate the CSV->DB part+possibly automate further and automatically scrape+load as soon as new data is available
- Do whatever complex calculation you deem interesting
- Turn it into a web app with something like Django if you use Python (for your own internal use i.e. running on 127.0.0.1)
- Fancy graphing
- Think about how you'd add lines etc. eventually like my simplest vision would be finding a way to scrape the lines from one site, displaying all (maybe all point totals or some subset) and have your program highlight the line it thinks is juciest (or rank them whathaveu)
I'd say that would be a sweet milestone...use old data, run fancy-spancy math, get and display lines for you, fancy-spancy math shows you the line it likes most (just boldface vs normal or something simple) then you could place the bet manually.
Once you have that you can implement your own playmoney system to keep track of how you're doing if you don't want the hassle of betting
[Also you might consider dong NCAA-BB instead if you can get the data because I think there's some actual money to be made there alas it's capped due to market volume (kind of like higher stakes in poker not running enough etc.)
But if it's a fun project just stick with NBA if that's what you watch]
1) Load all data into the DB
2) Write a Python script that lets you query the average minutes played by a given player (or whatever simple stat you want)
Then go from that. Some ideas for next steps:
- Automate the CSV->DB part+possibly automate further and automatically scrape+load as soon as new data is available
- Do whatever complex calculation you deem interesting
- Turn it into a web app with something like Django if you use Python (for your own internal use i.e. running on 127.0.0.1)
- Fancy graphing
- Think about how you'd add lines etc. eventually like my simplest vision would be finding a way to scrape the lines from one site, displaying all (maybe all point totals or some subset) and have your program highlight the line it thinks is juciest (or rank them whathaveu)
I'd say that would be a sweet milestone...use old data, run fancy-spancy math, get and display lines for you, fancy-spancy math shows you the line it likes most (just boldface vs normal or something simple) then you could place the bet manually.
Once you have that you can implement your own playmoney system to keep track of how you're doing if you don't want the hassle of betting
[Also you might consider dong NCAA-BB instead if you can get the data because I think there's some actual money to be made there alas it's capped due to market volume (kind of like higher stakes in poker not running enough etc.)
But if it's a fun project just stick with NBA if that's what you watch]
07-09-2013
, 11:56 AM
07-09-2013
, 12:22 PM
yep.
perhaps you are just smarter than I, because my first inclination in all such circumstances is to write the script to parse the csv line by line and create an insert query that way. load data infile is much faster than that
perhaps you are just smarter than I, because my first inclination in all such circumstances is to write the script to parse the csv line by line and create an insert query that way. load data infile is much faster than that
07-14-2013
, 11:35 AM
Yesterday I spent a good chunk of time actual coding and getting things to work.
Basically, I wrote a script that scrapes all the season data for each team, as well as the Per Game data for every player. Right now it is just printing the info but the next step is to use sqlite3 to inject into a DB.
For anyone who cares, here's the script that grabs the info for Player Per Game. I still have to figure out how I'm going to do it across different seasons (i.e., handle players who switch teams)
And seasonal info:
The exception above is there to handle situations where, the Charlotte Bobcats did not exist pre-2005. I have a lot of tweaking to do, just coded it this way at the moment so that the entire program would run through 2000->2013.
Thoughts and suggestions are more than welcome
Basically, I wrote a script that scrapes all the season data for each team, as well as the Per Game data for every player. Right now it is just printing the info but the next step is to use sqlite3 to inject into a DB.
For anyone who cares, here's the script that grabs the info for Player Per Game. I still have to figure out how I'm going to do it across different seasons (i.e., handle players who switch teams)
Code:
from bs4 import BeautifulSoup
from urllib.request import urlopen
def grabTeam(url):
soup = BeautifulSoup(urlopen(url))
pergame = soup.find("table", id = "per_game")
mydata = pergame.find("tbody")
return mydata
def grabPlayers(per_game):
num = len(per_game.find_all('tr'))
count = 0
while (count<num):
i = 0
while (i<24):
print (per_game.td.string)
per_game.td.extract()
i+=1
count+=1
per_game.tr.extract()
print()
teams = ['ATL', 'BOS', 'BRK', 'CHA', 'CHI', 'CLE', 'DAL', 'DEN', 'DET', 'GSW', 'HOU', 'IND', 'LAC', 'LAL', 'MEM', 'MIA', 'MIL', 'MIN', 'NOH', 'NYK', 'OKC', 'ORL', 'PHI', 'PHO', 'POR', 'SAC', 'SAS', 'TOR', 'UTA', 'WAS']
team = ""
x = 0
while (x<30):
team_url = "http://www.basketball-reference.com/teams/"+teams[x]+"/2013.html"
grabPlayers(grabTeam(team_url))
x+=1
Code:
from bs4 import BeautifulSoup
from urllib.request import urlopen
def grabTeam(url):
soup = BeautifulSoup(urlopen(url))
pergame = soup.find("table", id = "team_stats")
mydata = pergame.find("tbody")
return mydata
def grabStats(team_stats):
count = 0
while (count<2):
team_stats.td.extract()
team_stats.td.extract()
team_stats.td.extract()
i = 0
while (i<19):
print (team_stats.td.string)
team_stats.td.extract()
i+=1
count+=1
team_stats.tr.extract()
team_stats.tr.extract()
print()
teams = ['ATL', 'BOS', 'NJN', 'CHA', 'CHI', 'CLE', 'DAL', 'DEN', 'DET', 'GSW', 'HOU', 'IND', 'LAC', 'LAL', 'MEM', 'MIA', 'MIL', 'MIN', 'NOH', 'NYK', 'OKC', 'ORL', 'PHI', 'PHO', 'POR', 'SAC', 'SAS', 'TOR', 'UTA', 'WAS']
team = ""
year = 2000
while (year<2013):
x = 0
while (x<30):
try:
team_url = "http://www.basketball-reference.com/teams/"+teams[x]+"/"+str(year)+".html"
print(teams[x])
grabStats(grabTeam(team_url))
x+=1
except(AttributeError):
x+=1
pass
year+=1
Thoughts and suggestions are more than welcome
07-14-2013
, 06:51 PM
