Steffen and Eratosthenes will be pleased to hear that. I think Eratosthenes was considering ways to make overlapping HUD work in a maintainable fashion - would you care to elaborate on your strategy?

There hasn't been a lot of performance tuning as yet, correctness is still the priority.

I'm not sure that the auto import is even its own thread yet.

With the current import code this would be difficult as it has no record of previous states.

Has the file been modified - yes. Import it. The handId lookup is pretty quick, so the code diesn't spend a lot of time on stored hands. (Had some timing code in there a while ago)

I think the biggest gains to be made are in the HUD cache generation code, there are a couple of inefficient loops and a few places where stats can be collected in a less CPU intensive way.

Please feel free to join the mailing list and discuss https://****************/mail/?group_id=226872

This is a quick reply; I’ll address your other points a little later.

I added a method to the class HUD which is called whenever the HUD main_window (the one you close to get rid of that particular table’s HUD) receives a “window-state-event”. The method checks whether the main_window has just been iconified and if so hides all the stat_windows. Thus when one wants the statistics for a given table one looks in the taskbar for the HUD with the same name and calls it up, minimizing (iconifying) it again when it is no longer needed. An ugly solution, but it works for me.

A “real” solution along these lines (which would also call for less frantic clicking on the part of the user) would involve the program’s keeping track of which table window is currently at the top of the stack (or the cascade, which is functionally equivalent).

Ignore - deleted double posting.

That's possible, and since the HUD is a separate program anyways it doesn't matter at all in which language the importer is written. However, an easier approach would probably be to write a converter from the history format of site X to PS format.

There isn't any significant difference between alpha5 and alpha6 for people who already got it working, so I'd skip alpha6 if I were you. The current git (and future alpha7) will however contain data-fixes ie. it fixes places where alpha6 and lower were storing wrong info so you probably wanna be taking that one.

Haven't really looked into performance at all since it's fine for the needs of myself and the other devs. For keeping the position inside a file, sorrow is reworking fpdb_import right now so it's probably best to wait for that to come to the main tree before you start implementing this. But as he noted, the biggest gains can probably be made in HudCache generation (fpdb_simple.py). Since the vast majority of CPU is spent in MySQL it might also help to play with the MySQL options, or try PGSQL as backend. Another thing that I suspect will bring a notable benefit is if there were a way to get the ID of a new row without having to run a SELECT.

Cheers, Steffen

Thanks for the update on your use--I am excited that people are using fpdb and I'm sure the other team members are too.
I have been thinking about stacked/cascading tables, but I have not come up with a way to handle them that I would be happy distributing. I want to keep the HUD pretty simple so that it remains reliable and speedy. wow wow wow Do you mind feeding your code back to us. If it is reliable enough for you to 14-table, we can probably add it to the tree and you won't have to worry about reintegrating your code. (you can PM me here, or go to IRC channel #fpdb on freenode) If your changes are only to HUD.py, you can probably just drop your HUD.py into the new release and have it work fine. (there is at least one small bug fix in the latest HUD.py that relates to saving the stat window layout) Another option for you would be to set up your own git repo and then let git merge your changes with the latest in one of our git repos.

thanks

I wrote an importer in perl a while back that remembered its location in the file and only parsed new stuff. It should pretty easy to implement the same thing in python. I have been planning to do this, and I will bump it up the to do list.

Yeah, this is the part of the cascading/stacked tables support that worries me. The code that detects when a new table is on top has to be fast and reliable to prevent the stats belonging to the previous table from staying on the screen too long. Hiding one set of stats and showing another will be quick and easy if we get the new window detection bit right.

Stranger--What operating system are you using?

Yes I would also very much like to give this a go - I use a 12" laptop for poker so even two tabling is impossible for me at the moment (well, unless I use a different workspace, but then I tend to miss my turn...).

That would be fine, as well, but if that's the way to go, I'd like to have a way to tag my sites with a different name .. with PokerTracker, I appear to be stuck calling Everleaf "Pokerstars" .. so that kinda sucks.

Yes, any potential such converter should insert the correct site name where in an original PS history would say PokerStars. Adapting the importer for that scenario is a 5 minute job, if anyone wants to give this a go just send me a PM or email. Then your Everleaf hands will be called Everleaf.

I understand this completely, of course. Since you have been very ambitious about the variety of games the software is to support, correctness is obviously the first consideration.

But since I happen to multi-table full-ring NLHE, I decided to see if I could get it to work (with surprising results).

Obviously performance starts to become a consideration once one has more than, say, ten HUDs running simultaneously.

I ended up stripping the code down quite drastically in the attempt to make it run as smoothly as possible. I removed the tool-tips and the pop-up window from the statistics windows, as I can happily cope without them. This, among other things, makes my code unsuitable for general use.

I also noticed that MySQL accounted for a lot of the load on the processor, and tried tampering with the database code as well. Whether this ultimately had a beneficial effect, I am not sure. As I have mentioned, I make the automatic importer wait two minutes between updates: this keeps the workload under control.

I’m using Linux.

At one point I wondered how automatic detection of the currently active table could be achieved, and a cursory investigation suggested that it should be possible in X-Windows using python-xlib. Unfortunately, the variety of situations to be accounted for involving the possibility of partially overlapping tables (which might or might not obscure each others’ HUDs) is daunting. We want stack-like behaviour when the tables obscure one another, and the existing behaviour (all HUDs visible) otherwise. I have nothing useful to suggest on this topic at the moment.

I was thinking that I would add a "stacked" parameter to the config file. If stacked is false, then the HUD handles things the way it does now. If true, the HUD uses the stacked routines and any window that is overlapped is considered to be underneath and its stat windows (+ etc.) would be hidden. This will not be ideal for users who want to play with overlapped windows, but it will be a start.

I've thought for some time that, for any poker tracking database program, the best approach to the problem of different history formats would be to design your own site-neutral format and convert all sites' histories to it. Support for new sites could easily and independently be added simply by writing a converter (cake2fpdb.py, for example). fpdb would only need to understand and parse its own format after that.

HUD for a new site might be a problem using this approach, though. I guess the converter would have to have the ability to monitor its site's home directory and convert new hand histories as they occur. That wouldn't be a difficult thing, but it doesn't seem like real-time monitoring is a function a converter should have to implement. Anyway, that's a fairly minor detail.

Using the PS format as your default format would be sort of okay, I guess, but designing your own format would be cleaner and more in keeping with the whole philosophy of open source. I've downloaded fpdb and got it mostly running on Ubuntu, but I haven't really looked at the source at all and I've never written anything in python, so I don't know how big of a problem implementing this idea would be at this stage of fpdb's life. I do think it's worth considering, though. I believe it could make many future decisions simpler.

Don't really see the difference between designing our own format and using a random pre-existing one tbh.

I'm with Steffen on this one. Stars format is stable, readable, software parsable, supports pretty much every online poker game and format - with dozens of existing converters already in existance for other tracking software.

Why on earth would you wan't to re-invent the wheel for no benefit?

Converting hand histories is not a sensible measure, converting is parsing and rewriting so you might as well ditch the rewriting and just put the work into fpdb's parser (which is clearly an unstructured hack that badly needs a rewrite).

You are exactly right about the site-neutral internal format Weevil. There are only 3 of us working on this and there is a huge amount of work to do. Therefore, I am perfectly happy to use the existing converters to convert from ObvRiggedPoker.com hh format to PokerStars format and importing from there.

You're oversimplyfying, a hand converter will generally only need to do a minimal amount of parsing since the differences are generally small. For example, it doesn't need to be able to understand that 3betting is not just a raise, it doesn't need to able to know in which street one is, etc.

A parser should be acting like the kind of converter you are talking about internally. The obvious model of a hand history parser is to use site-specific parsers to parse lines individually to a sequence of actions, cards and results then use shared code to derive statistics and insert the information into the database.

That's basically how fpdb does it. See fpdb_parse_logic.py, though it is a bit messy in places. But since PS and FTP have about 99% identical format all the code is shared, except where necessary it splits up. However, this is of course not viable for supporting 20 sites. But for the time being I'd be happy to accept patches that use any of the methods described, whether converting to PS or a whole independent importer or an independent parser that uses the existing analysis code (namely generateHudCacheData).

Well, it's a minor nit, but it just seems more in keeping with the philosophy of open source to use a home-grown, site-neutral format. It just seems cleaner to me, but I'm having trouble articulating why, beyond the fact that designing your own means you're not constrained by someone else's design decisions. Anyway, it's a minor thing.

The main point is to standardize on a format so that your importer only has to understand one. Using the current PokerStars format accomplishes that just as well as designing a new one.

Of course, at some point you'll be writing a converter to convert newPS to fpdbPS since even PokerStars changes their HH format occasionally and breaks poker programs. Even if you use their current format exactly as it exists, I think it makes sense to at least refer to it as fpdb format in order to avoid any confusion in the future.

If you are satisfied with the format as it exists, I can't think of a strong reason to roll your own beyond what I said in my reply to Steffen. I definitely think it should be thought of and referred to as the fpdb format, though, and not the PS format, even though it's copied directly, without change, from their current format.

I can see your point, but I'm not quite strong enough a free software fanatic to invest the required work into it
And by using the approach that we laid out it will be possible for other poker tracking software to benefit directly from fpdb's hand converters (they have to support the old PS format anyways...).
Though you are right, eventually we will probably no longer be able to make up for PS' breakages by using regexps instead of string search (that's what we did with the recent format change, the new regexp works with both the old and the new format). I think we will then simply refer to them as "fpdb-PS" and "newPS-year-month" or sth along those lines. But this semantics only anyways
For the record, I want to emphasize that these converters can be written in ANY programming or scripting language. If anyone has a converter from anything to PS or FTP (or even into another target format) that they would be willing to release under a free software license please let me know and hopefully we'll find an arrangement to make fpdb use it

Using a standard format would make it trivial to add support for new sites or update support for existing sites whose HH format changed overnight, for one thing. A converter is a simple thing to write and needn't be touched once it works and for as long the site it supports doesn't change; a parser/importer isn't so simple and must change constantly as new ideas are implemented and as the database design itself evolves. It would be much simpler if the parser/importer only had to worry about a single, unchanging format.

If one of the goals of the project is to support as many different sites as possible, then a single, standard format will save many future headaches. The simplest way to accomplish this, in my opinion, is by using a different converter program for each site and having them write into fpdb's HH directory. fpdb's parser/importer wouldn't have to know anything about any other site's directory structure or HH format. It would only need to understand its own format and where to find its own hand histories.