Suppose you want to measure the difference between two players' strategies (e.g. you think someone is multiaccounting and not varying their play much, or someone's playstyle is so unusual compared to the average reg that it's suspicious). You could often do this by eye via HEM or similar if you had enough data, but I'm curious how you would model it formally.

If you're using this measure for more detailed analysis you also want it to observe other properties, for instance:

- Distance between parts of the distribution matters as well as the total distance: if two strategies are mostly similar but one player takes a certain action much more often, that's still strange

- We might want to assign extra 'weight' to certain actions where a discrepancy is a good indication of what you're trying to prove

- Even where there's an obvious way of 'ordering' parts of a distribution, that might not apply in all cases (e.g. 1/4 pot and 1/3 pot are 'closer' than 1/4 pot and 2/3 pot but that's misleading when building a range of bet sizes; or maybe the site you're using has a 1/2 pot button so, for most human players, bet sizes in the vicinity of 1/2 would be collapsed into 1/2 and a ~52% pot bet would 'differ' more from 1/2 than a 40% pot bet).


In some of the academic work on poker, earth mover's distance is used to group hands based on hand strength for the purpose of abstraction; someone suggested I should use it here, but it doesn't conform to the properties above (afaik?).

Some of the independent exposes on bot use have used the squared Euclidean distance between common stats. This works well enough in those cases - mostly where the conclusion is clear from looking at the stats and they are just trying to add some rigour - but it has some apparent issues. It's sensitive to the choice of stats: adding or removing certain stats can shift the result, so you need some way of deciding which stats are useful. Also, many of those stats aren't independent of each other; a small change to the strategy will affect a bunch of stats at the same time, so the change in distance could be too large.

Please let me know if you have corrections to the above or thoughts on the problem.

For all the reasons you describe, this is a very challenging problem to tackle.

One of the important facets you touch upon is deciding what stats to track. Of course, everything stems from this. You have to utilize knowledge of poker and knowledge of complexity of poker strategies (both human and bot) to delve into the problem at the correct "level".

In reality, the number of factors than can enter into a poker decision is nearly endless. In addition to your cards, your position, position of each player who has acted before you, stack size of each player who has acted before you and still to act, tendencies of each player, etc., etc.

For tractability assume that there are 10 key stats that compose a strategy. Then you observe two "people" playing poker for awhile and compile these 10 key stats for each player. You want to determine whether these two players are utilizing the same underlying strategies.

Bayesian analysis can be used for this analysis. You need to posit how the stats are typically generated. Then compare the two sets of stats and determine the probability that they came from the same strategy vs some sufficiently-different strategy.

Of course, each link in that chain of Bayesian reasoning is its own challenge. But the Bayesian approach often sidesteps the pitfalls of the standard traditional statistical approach (some of which you mention above).