** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD **

[Shoe Lace]

It really depends on what you want to do with the data; in this case the hand histories themselves. Picking the right DB is all about knowing what type of data you're storing, what you want to do with it and what type of traffic/load you're expecting.

[gaming_mouse]

jason,

what are your favorite books on programming? and how about specifically low-level stuff?

[sng_jason]

Originally Posted by clowntable

Does not compute :P

Touche. No comment on Logo/turtle graphics?

[sng_jason]

gaming_mouse,

hmm... its been a long time since I've opened a programming book... lol

Oh, but I know... I can definitely recommend this book written by a guy I worked with, like 13 years ago (he's gone on to be a lead programmer on the God of War games):

http://www.amazon.com/exec/obidos/tg...altimecolli-20

While the book focuses on 3d collision detection (a pretty specific subject), it also has a bunch of *excellent* chapters on low-level optimizations for modern hardware. In particular is a *double excellent* bit on the gory details of how floating point math works and all of the pitfalls that await the unaware.

This book is *mandatory* for any would be 3d-game programmers.

[Ryanb9]

Originally Posted by clowntable

Maybe not for pathfinding but the next progression would be local search algorithms and genetic algorithms. Maybe solve N-Queens with both or something
Might be a nice idea to get "Artificial Intelligence, a Modern Approach" and read through the search chapters.

That is a very expensive book, $91 for a used copy on amazon but it looks so good >.<

[_dave_]

Originally Posted by clowntable

Talking about databases (a topic I really should improve on given I have only used MySQL,Postgres and ZODB). If you would build some sort of poker software that has to handle boatloads of hand histories (smth like HEM or maybe the pokerserver itself) today, what kind of DB would you use for the HHs? HEM uses postgres but it feels like some other type of DB could be better for this. But then you'd probably have to interface with postgres et. anywys for statsdata and so forth.

Interesting topic for sure, I've thought about it a lot and I can't imagine much better than postgres. Hard to know what you mean, since the database servers in HEM or PT don't do anything with the HHs, other than store a copy for backup purposes. When you say "for stats data" - that's almost the entirety of the product!

So I don't know, maybe you mean something else, and maybe a completely different approach would call for something else entirely.

But if you're wondering if something could be done better than currently using an alternative database platform, it would be interesting to hear where you think postgres is failing these trackers. I believe HEM2 uses some form of hybrid strategy, but I haven't studied it yet. PT database I have studied quite extensively, not so much with 4 but it seems mostly unchanged.

Long way down the road I hope to find time to learn python, so I can conduct such experiments on FPDB and perhaps make it my dream tracker. but obv need to learn python first!

[sorrow]

Originally Posted by _dave_

Interesting topic for sure, I've thought about it a lot and I can't imagine much better than postgres. Hard to know what you mean, since the database servers in HEM or PT don't do anything with the HHs, other than store a copy for backup purposes. When you say "for stats data" - that's almost the entirety of the product!

So I don't know, maybe you mean something else, and maybe a completely different approach would call for something else entirely.

But if you're wondering if something could be done better than currently using an alternative database platform, it would be interesting to hear where you think postgres is failing these trackers. I believe HEM2 uses some form of hybrid strategy, but I haven't studied it yet. PT database I have studied quite extensively, not so much with 4 but it seems mostly unchanged.

Long way down the road I hope to find time to learn python, so I can conduct such experiments on FPDB and perhaps make it my dream tracker. but obv need to learn python first!

Schema design is much more important than the database being used.

An RDBMS is the only conceptual way I know how to deal with the number of individual records required for poker. You need to be able to produce individual hands and aggregate data

Join us _dave_ - you won't regret learning python

[Shoe Lace]

I'm not convinced that a relational db is the best choice. I kind of think the decision to use postgres for PT or HEM was because viable alternatives didn't exist.

For starters, if he were just storing and querying hand histories then a document based DB is definitely the best fit. That will allow you to store everything in a non-rigid format.

With postgres you would need to do all sorts of multiple table grossness to try and normalize a poker hand because every hand is so different even if you limit it to 1 game. Your queries would also be monstrous and probably not very performant once your DB has grown to a huge size.

With a document based db you would just parse the HH and format the hand with json. Some hands will have a turn/river, others will not. Some hands will have 5 people to a pot, others will not. Awesome and super simple.

Then querying hands would be really easy too because you have everything broken up.

[tyler_cracker]

Originally Posted by Shoe Lace

With a document based db you would just parse the HH and format the hand with json. Some hands will have a turn/river, others will not. Some hands will have 5 people to a pot, others will not. Awesome and super simple.

how many times have i had A9? by position? facing a raise?

seems like those kinds of questions are going to be tough and/or expensive to pull out of json blobs.

[Shoe Lace]

Originally Posted by tyler_cracker

how many times have i had A9? by position? facing a raise?

seems like those kinds of questions are going to be tough and/or expensive to pull out of json blobs.

You can set indexes on certain keys with some document dbs. It wouldn't be that bad at all. I should also add that writing a query (or map/reduce job) to do that would be awesomely easy to write.

It would take literally 10 seconds to write a mongodb query to pull out what you want. I'm rusty with SQL and it would depend on how you have your tables broken up but would it be easy to do the same in postgres?

[sorrow]

Originally Posted by Shoe Lace

I'm not convinced that a relational db is the best choice. I kind of think the decision to use postgres for PT or HEM was because viable alternatives didn't exist.

For starters, if he were just storing and querying hand histories then a document based DB is definitely the best fit. That will allow you to store everything in a non-rigid format.

With postgres you would need to do all sorts of multiple table grossness to try and normalize a poker hand because every hand is so different even if you limit it to 1 game. Your queries would also be monstrous and probably not very performant once your DB has grown to a huge size.

With a document based db you would just parse the HH and format the hand with json. Some hands will have a turn/river, others will not. Some hands will have 5 people to a pot, others will not. Awesome and super simple.

Then querying hands would be really easy too because you have everything broken up.

Hand history data is already highly structured, and most queries don't care about individual hands only the aggregated data.

Most common query: Total profit. Requires the total of all monies committed v all monies won for all hands. Even with a couple of million hands this isn't a particularly costly query on and rdbms. I cant comment on how expensive it would be nosql.
Next most common queries: Total profit for a subset of hands by opponent, time, session, gametype, limit and position in any combination. Depending on options you would still need to hit all hands.

You could do all of this from a document based database - I have no doubt about this. You can also do this with an rdbms, and you could argue that is what rdbms is best at.

A document based approach is better for some types of hh data, but your example above is not one of them imho.

[kerowo]

Originally Posted by Shoe Lace

You can set indexes on certain keys with some document dbs. It wouldn't be that bad at all. I should also add that writing a query (or map/reduce job) to do that would be awesomely easy to write.

It would take literally 10 seconds to write a mongodb query to pull out what you want. I'm rusty with SQL and it would depend on how you have your tables broken up but would it be easy to do the same in postgres?

I've heard that mongodb has web scale too, you'll probably need some of that.

[daveT]

PostgreSQL is an amazingly powerful RDMS. There's not many queries that can't be done with it. I'm running a database with 15 tables that range from a few dozen tuples to 500,000 tuples. The database isn't well-normalized -- thanks to the database I extracted from any my dis-interest in doing so -- and there hasn't been many queries I haven't been able to create.

I've had to check things like sales by territory, product conversion, average size of orders, sample-to-sales conversions, product and customer profit analysis, and all sort of things that you'd think would be difficult at first glance, but really end up being fairly simple to do. All of this stuff is basically the same as one would face when doing poker queries. It's similar because there are many tuples with null sales, variable price-results, and tons of odd or missing information.

I'm not going to claim that noSQL is worse than PostreSQL, but I will claim that using noSQL because you think queries on PostgreSQL is hard or too slow doesn't seem like a good excuse: the difficulty would arise because you don't understand how queries work in regular SQL, and there isn't a good reason to assume that noSQL will make things better. Saying that there is a concern because you don't have a game with 9 players at a full table is like saying you can't do full-scale sales queries because one territory didn't make any sales on Monday.

I also question, since noSQL doesn't use SQL as the query language, why you'd want to deal with noSQL for smaller projects outside of academic interest. I mean, noSQL of course makes perfect sense for Twitter or Facebook, but to do data comparisons, you are forced to create your own queries anyways, and most of those queries will revolve around join, natural join, and querying sub-tables and temporary tables, which are the very foundations of SQL. I'd rather have all of this built-in, and I'm pretty sure HEM and PT would rather have this built-in as well.

[Gullanian]

I think almost all the performance is going to come down to having a well designed database. I would work with whichever dbms you are most comfortable with, and then once it's up and running experiment with supporting other dmbs which shouldn't be too difficult assuming you're not doing anything fancy.

The question is if you want to go relational or non relational I suppose, I don't have any experience with non relational databases but I would assume that relational databases would be preferable as they would allow you to build complex reports easier?

[Gullanian]

Designing a database to store HH's would be quite a fun exercise! If we had a set of data, and an objective to fetch 3 different reports. I'm in if anyone else wants to!