Comparing traditional and 2 new "ICM" systems against deep future game simulation - Poker Theory

Two Plus Two Forums Poker Strategy Poker Theory & GTO

Comparing traditional and 2 new "ICM" systems against deep future game simulation

Post Reply Subscribe

...

12-23-2018 , 04:36 PM

LektorAJ

Carpal \'Tunnel

Join Date: May 2014 Posts: 7,826

There are different ways to evaluate the monetary value of the stacks various players have in a poker tournament. In this post/thread I will look at the values of stacks when a short-stacked tournament is simulated to a depth of 9 hands ahead, using the FGS8 setting in Holdem Resources calculator, and compare the values found to the values given by stack evaluation algorithms.

The tournament structure used is based on the 5-handed SNG which runs on Unibet (an untracked euro site). 1st place pays double the prize for second place. For simplicity, below we are going to use 200-100 as the payouts. There are a total of 10K chips in play. The first round with antes (often the last round altogether) is the 120/240/25 round.

Below is an earlier result I got from the same process (IIRC it was using 50/100 blinds, and the numbers shown are the totals of the Equity Post % column in HRC when the calculation is run 3 times, once each for possible starting position):

Code:

1000	500	500 
137.37	79.65	82.99

Note the above shows the players ordered as they sit clockwise. The player with 500 on the right of the big stack has an advantage over the player to the left of the big stack.

The difference between these numbers and the results given by other algorithms led me to try to come up with something better.

Here are the values of the stacks given by various algorithms:

Chip chop - 1: 150.0000 2: 75.0000 3: 75.0000
Landrum Burns - 1: 125.0000 2: 87.5000 3: 87.5000
Combined CC/LB - 1: 137.5000 2: 81.2500 3: 81.2500
Declining Chip Count - 1: 140.0000 2: 80.0000 3: 80.0000
Ben Roberts - 1: 135.7143 2: 82.1429 3: 82.1429
Malmuth-Weitzman - 1: 130.0000 2: 85.0000 3: 85.0000
Malmuth-Harville - 1: 133.3333 2: 83.3333 3: 83.3333
Swales, Depth 7 (0.002 seconds) 1: 138.8889 2: 80.5556 3: 80.5556
Swales, Depth 19 (8.944 seconds) 1: 138.8889 2: 80.5556 3: 80.5556

Combined CC/LB is just the average of the values given by the first two algorithms.

These numbers look pretty close, but the differences can be crucial. For example if the Malmuth-Weitzman figures are correct, the two short stacks have a "bubble factor" of 1.307 against each other (see "Kill Everyone", this means that if they flip against each other they are risking an 85 euro stack for a benefit of 65 euros - as the winner of the flip goes heads up with an equal stack worth 150 - and 85/65 is 1.307 and this is a measure of the "odds against" the payout system is giving them), the short stack against the big stack has a bubble factor of 1.889 and the big stack has a bubble factor of 1 against the short stacks (going up to a stack value of 175 if he wins is a gain of 45, or he loses 45 euros of equity if he swaps places with the short stacks). This means the game is to a large extent about the big stack pushing the short stacks around and they have to keep out of his way.

On the other hand, if the combined CC/LB figures are accurate (and those are the closest ones to the simulated values) the bubble factors are 1.182 (short v short) 1.444 (short v big) and 1.5 (big vs short) and therefore the big and short stacks have to both be equally wary of each other. In fact the big stack advantage mainly comes from the likelihood of the two small stacks colliding as they can still go at each other.

I will explain and give Pascal code for "Declining Chip Count" and "Swales" now:

Declining Chip Count is intended to be an improvement on ordinary chip count. In the example above, the player with 1000 chips is held to have a 50% chance of winning, and a 50% chance of finishing 2nd, (and also a 50% chance of finishing 3rd if there are payouts for it), which means absurd results often occur valuing the stack at more than the first place prize.

Using declining chip count, the first place percentages (50-25-25) are calculated based on the number of chips each player has. Then, before the second prize is calculated we reduce the number of chips in each stack by the percentage already accounted for, so 2nd place is calculated based on fictional stacks of 500-375-375 resulting in probabilities of 40-30-30 for second place.

There is a further hard condition that the total probabilities for a particular player are capped at 100%.

Thee code for this calculation system follows:

Code:

procedure TSituation.DecliningChipCount;
var
  i,j: integer;
  totalstacks:double ;
  numberofstacks: integer;
  spareprob: double;
  unblocked: double;
  problemfound: boolean;
  probdone:array[1..10] of double;
  probwant:array[1..10] of double;
  probmax:array[1..10] of boolean;
begin
  ClearValues;
  numberofstacks := 0;
  j:= 1;
  for i := 1 to 10 do
  begin
    probdone[i] := 0;
    probmax[i] := False;
      if (stack[i] > 0) then
      begin
        inc(numberofstacks);
      end;
  end;
  repeat
    totalstacks := 0;
    for i := 1 to 10 do
    begin
      if ((1-probdone[i])*stack[i] > 0) then
      begin
        totalstacks := totalstacks + ((1-probdone[i])*stack[i]);
      end;
    end;

    for i := 1 to 10 do
    begin
      probwant[i] := ((1-probdone[i])*stack[i]) / totalstacks
    end;

    repeat
      spareprob:=0;
      unblocked := 0;
      problemfound := False;
      for i := 1 to 10 do
      begin
        if probwant[i] > (1-probdone[i]) then
        begin
          problemfound := true;
          spareprob := spareprob+ probwant[i] - (1-probdone[i]);
          probwant[i] := (1-probdone[i]);
          probmax[i] := True;
        end;
        if probmax[i] = False then
        begin
          unblocked := unblocked +probwant[i];
        end;
      end;

      if problemfound and (unblocked>0) then
        begin
        for i := 1 to 10 do
        begin
          if probmax[i] = False then
          begin
            probwant[i] := probwant[i] + (probwant[i] * spareprob / unblocked)
          end;
        end;
      end;
    until (problemfound = false) or (unblocked = 0);
    for i := 1 to 10 do
    begin
      value[i] :=  value[i] + (prize[j] * probwant[i]);
      probdone[i] := probdone[i] + probwant[i];
    end;
    inc(j);
  until (j = numberofstacks+1) or (prize[j] = 0);
end;

The "stack", "prize" and "value" arrays are in the TSituation object and are hopefully self explanatory. The "problemfound" boolean refers to when we have found a situation where someone's probabilities would add up to more than 100% and we have to redistribute their surplus elsewhere.

The Swales (named after myself) algorithm for evaluating stacks works by simulating collisions between stacks. There is a given probability of a particular pair of stacks colliding and in a collision each stack has a 50-50 chance of winning the all-in. Each of the possible combinations (so three handed there are six of them A>B, A>B, B>A, B>C, C>A or C>B) leads to another position to evaluate and there is a "tree" of positions. When there are only two people left the position is evaluated by chip chop. When the tree reaches a certain depth then Declining Chip Count is used to evaluate the position. The probability for a particular pair of stacks to collide is proportional to: the inverses of stack A, of stack B and of the effective (i.e. lower) stack between them. For 3 handed positions a depth of 5 is sufficient, we are going to use the figure for 7 in the test but it's usually completely converged even by depth 5 (as this corresponds to 5 "all-in/calls" it's unusual for us still to be three handed by that time as the short stack has to keep winning).

Here is the computer code for it:

Code:

procedure TSituation.Swales;
var
  i,j, k:integer;
  fSituation:TSituation;
  numberofstacks: integer;
  totalstacks: double;
  totalstackinverses: double;
begin
  If depthleft = 0 then
  begin
    DecliningChipCount;
  end
  else
  begin
    ClearValues;
    numberofstacks := 0;
    totalstackinverses := 0;
    for i := 1 to 10 do
    begin
      if stack[i] > 0 then numberofstacks := numberofstacks + 1;
    end;
    if numberofstacks = 2 then
    begin
      totalstacks := 0;
      for i := 1 to 10 do
      begin
        if stack[i] > 0 then totalstacks := totalstacks + stack[i];
      end;
      for i := 1 to 10 do
      begin
        if stack[i] > 0 then
        begin
          value[i] := (prize[1]*stack[i] / totalstacks) + (prize[2]*(totalstacks-stack[i]) / totalstacks)  ;
        end
        else
        begin
          value[i] := 0;
        end;
      end;
    end
    else
    begin
      for i := 1 to 10 do
      begin
        if stack[i] > 0 then
        begin
          for j := 1 to 10 do
          begin
            if (i<>j) and (stack[j]>0) then
            begin
              if stack[i]>stack[j] then
              begin
                totalstackinverses := totalstackinverses + (1/(stack[i]*stack[j]*stack[j]));
              end
              else
              begin
                totalstackinverses := totalstackinverses + (1/(stack[i]*stack[i]*stack[j]));
              end;
            end;
          end;
        end;
      end;
      for i := 1 to 10 do
      begin
        if stack[i] > 0 then
        begin
          for j := 1 to 10 do
          begin
            if (i<>j) and (stack[j]>0) then
            begin
              fSituation := TSituation.Create;
              for k := 1 to 10 do
              begin
                fSituation.prize[k] := prize[k];
              end;
              if stack[i]>=stack[j] then
              begin
                fSituation.depthleft := depthleft-1;
                for k := 1 to 10 do
                begin
                  if k = i then
                    fSituation.stack[k] := stack[i]+ stack[j]
                  else if k = j then
                    fSituation.stack[k] :=0
                  else
                    fSituation.stack[k] := stack[k];
                end;
                fSituation.Swales;
                for k := 1 to 10 do
                begin
                  if k = j then
                  begin
                    value[k] := value[k] + (prize[numberofstacks] * (1/(stack[i]*stack[j]*stack[j]))/totalstackinverses);
                  end
                  else
                  begin
                    value[k] := value[k] + (fSituation.value[k] * (1/(stack[i]*stack[j]*stack[j]))/totalstackinverses);
                  end;
                end;
              end
              else
              begin
                fSituation.depthleft := depthleft-1;
                for k := 1 to 10 do
                begin
                  if k = i then
                    fSituation.stack[k] := stack[i]*2
                  else if k = j then
                    fSituation.stack[k] := stack[j] - stack[i]
                  else
                    fSituation.stack[k] := stack[k];
                end;
                fSituation.Swales;
                for k := 1 to 10 do
                begin
                  value[k] := value[k] + (fSituation.value[k] * (1/(stack[i]*stack[i]*stack[j]))/totalstackinverses);
                end;
              end;
              fSituation.Free;
            end;
          end;
        end;
      end;
    end;
  end;
end;

These are the values found using deep FGS based on the unibet 5-handed SNG for every possible combination of 3 stacks with multiples of 1000 chips totaling 10000.

fgs8 120/240/25

Code:

		ABC	CAB	BCA	tot
A	8	60.75	61.06	60.13	181.94
B	1	20.89	17.2	19.95	58.04
C	1	18.28	21.74	19.91	59.93

What the above means, is that in a setup where the players have 8000, 1000, 1000, stack A has 60.75% of the equity when on the button, 61.06% when on the small blind, 60.13% when in the big blind, for a total of 181.94 also corresponding to 181.94 euros with payouts of 200-100

Here are the values given by the different algorithms:

Chip chop - 1: 240.0000 2: 30.0000 3: 30.0000
Landrum Burns - 1: 170.0000 2: 65.0000 3: 65.0000
Combined CC/LB - 1: 205.0000 2: 47.5000 3: 47.5000
Declining Chip Count - 1: 180.0000 2: 60.0000 3: 60.0000
Ben Roberts - 1: 178.6301 2: 60.6849 3: 60.6849
Malmuth-Weitzman - 1: 174.1176 2: 62.9412 3: 62.9412
Malmuth-Harville - 1: 177.7778 2: 61.1111 3: 61.1111
Swales, Depth 7 (0.005 seconds) 1: 179.8881 2: 60.0559 3: 60.0559

Code:

				
		ABC	CAB	BCA	tot
A	7	56.44	56.87	56.18	169.49
B	2	29.59	28.43	29.2	87.22
C	1	13.97	14.73	14.62	43.32

Chip chop - 1: 210.0000 2: 60.0000 3: 30.0000
Landrum Burns - 1: 155.0000 2: 80.0000 3: 65.0000
Combined CC/LB - 1: 182.5000 2: 70.0000 3: 47.5000
Declining Chip Count - 1: 170.0000 2: 84.8000 3: 45.2000
Ben Roberts - 1: 166.8085 2: 90.2128 3: 42.9787
Malmuth-Weitzman - 1: 161.3043 2: 89.5652 3: 49.1304
Malmuth-Harville - 1: 165.2778 2: 88.8889 3: 45.8333
Swales, Depth 7 (0.006 seconds) 1: 168.8767 2: 87.4890 3: 43.6343

Code:

		ABC	CAB	BCA	tot
A	7	56.63	56.47	56.2	169.3
B	1	14.08	13.51	14.09	41.68
C	2	29.32	30.01	29.71	89.04

Chip chop - 1: 210.0000 2: 30.0000 3: 60.0000
Landrum Burns - 1: 155.0000 2: 65.0000 3: 80.0000
Combined CC/LB - 1: 182.5000 2: 47.5000 3: 70.0000
Declining Chip Count - 1: 170.0000 2: 45.2000 3: 84.8000
Ben Roberts - 1: 166.8085 2: 42.9787 3: 90.2128
Malmuth-Weitzman - 1: 161.3043 2: 49.1304 3: 89.5652
Malmuth-Harville - 1: 165.2778 2: 45.8333 3: 88.8889
Swales, Depth 7 (0.006 seconds) 1: 168.8767 2: 43.6343 3: 87.4890

Code:

				
		ABC	CAB	BCA	tot
A	6	52.04	52.34	51.73	156.11
B	3	35.93	35.06	35.73	106.72
C	1	12.03	12.6	12.54	37.17

Chip chop - 1: 180.0000 2: 90.0000 3: 30.0000
Landrum Burns - 1: 140.0000 2: 95.0000 3: 65.0000
Combined CC/LB - 1: 160.0000 2: 92.5000 3: 47.5000
Declining Chip Count - 1: 160.0000 2: 102.0000 3: 38.0000
Ben Roberts - 1: 154.4444 2: 110.5556 3: 35.0000
Malmuth-Weitzman - 1: 148.8889 2: 107.7778 3: 43.3333
Malmuth-Harville - 1: 152.3810 2: 108.3333 3: 39.2857
Swales, Depth 7 (0.006 seconds) 1: 156.1994 2: 107.3399 3: 36.4607

Code:

			
		ABC	CAB	BCA	tot
A	6	51.97	51.81	51.62	155.4
B	1	12.03	11.6	12.23	35.86
C	3	35.99	36.59	36.15	108.73

Chip chop - 1: 180.0000 2: 30.0000 3: 90.0000
Landrum Burns - 1: 140.0000 2: 65.0000 3: 95.0000
Combined CC/LB - 1: 160.0000 2: 47.5000 3: 92.5000
Declining Chip Count - 1: 160.0000 2: 38.0000 3: 102.0000
Ben Roberts - 1: 154.4444 2: 35.0000 3: 110.5556
Malmuth-Weitzman - 1: 148.8889 2: 43.3333 3: 107.7778
Malmuth-Harville - 1: 152.3810 2: 39.2857 3: 108.3333
Swales, Depth 7 (0.007 seconds) 1: 156.1994 2: 36.4607 3: 107.3399

Code:

		
		ABC	CAB	BCA	tot
A	6	51.82	51.65	51.2	154.67
B	2	23.91	23.04	24.04	70.99
C	2	24.28	25.31	24.76	74.35

Chip chop - 1: 180.0000 2: 60.0000 3: 60.0000
Landrum Burns - 1: 140.0000 2: 80.0000 3: 80.0000
Combined CC/LB - 1: 160.0000 2: 70.0000 3: 70.0000
Declining Chip Count - 1: 160.0000 2: 70.0000 3: 70.0000
Ben Roberts - 1: 152.3077 2: 73.8462 3: 73.8462
Malmuth-Weitzman - 1: 145.7143 2: 77.1429 3: 77.1429
Malmuth-Harville - 1: 150.0000 2: 75.0000 3: 75.0000
Swales, Depth 7 (0.002 seconds) 1: 155.4546 2: 72.2727 3: 72.2727

Code:

	
		ABC	CAB	BCA	tot
A	5	47.27	47.49	46.76	141.52
B	4	41.37	40.63	41.18	123.18
C	1	11.36	11.88	12.06	35.3

Chip chop - 1: 150.0000 2: 120.0000 3: 30.0000
Landrum Burns - 1: 125.0000 2: 110.0000 3: 65.0000
Combined CC/LB - 1: 137.5000 2: 115.0000 3: 47.5000
Declining Chip Count - 1: 143.1034 2: 121.3793 3: 35.5172
Ben Roberts - 1: 141.3043 2: 126.9565 3: 31.7391
Malmuth-Weitzman - 1: 136.2069 2: 122.7586 3: 41.0345
Malmuth-Harville - 1: 138.8889 2: 124.4444 3: 36.6667
Swales, Depth 7 (0.005 seconds) 1: 141.1571 2: 124.5418 3: 34.3011

Code:

	
		ABC	CAB	BCA	tot
A	5	46.93	46.65	46.48	140.06
B	1	11.71	11.46	12.13	35.3
C	4	41.36	41.89	41.39	124.64

Chip chop - 1: 150.0000 2: 30.0000 3: 120.0000
Landrum Burns - 1: 125.0000 2: 65.0000 3: 110.0000
Combined CC/LB - 1: 137.5000 2: 47.5000 3: 115.0000
Declining Chip Count - 1: 143.1034 2: 35.5172 3: 121.3793
Ben Roberts - 1: 141.3043 2: 31.7391 3: 126.9565
Malmuth-Weitzman - 1: 136.2069 2: 41.0345 3: 122.7586
Malmuth-Harville - 1: 138.8889 2: 36.6667 3: 124.4444
Swales, Depth 7 (0.006 seconds) 1: 141.1571 2: 34.3011 3: 124.5418

Code:

		ABC	CAB	BCA	tot
A	5	45.99	45.8	45.35	137.14
B	3	31.95	31.18	32.02	95.15
C	2	22.06	23.02	22.64	67.72

Chip chop - 1: 150.0000 2: 90.0000 3: 60.0000
Landrum Burns - 1: 125.0000 2: 95.0000 3: 80.0000
Combined CC/LB - 1: 137.5000 2: 92.5000 3: 70.0000
Declining Chip Count - 1: 140.3226 2: 93.8710 3: 65.8065
Ben Roberts - 1: 136.3636 2: 98.1818 3: 65.4545
Malmuth-Weitzman - 1: 130.6452 2: 97.7419 3: 71.6129
Malmuth-Harville - 1: 133.9286 2: 97.5000 3: 68.5714
Swales, Depth 7 (0.003 seconds) 1: 137.9596 2: 95.9927 3: 66.0476

Code:

			
		ABC	CAB	BCA	tot
A	5	45.96	45.52	45.31	136.79
B	2	21.76	21.38	22.13	65.27
C	3	32.28	33.1	32.56	97.94

Chip chop - 1: 150.0000 2: 60.0000 3: 90.0000
Landrum Burns - 1: 125.0000 2: 80.0000 3: 95.0000
Combined CC/LB - 1: 137.5000 2: 70.0000 3: 92.5000
Declining Chip Count - 1: 140.3226 2: 65.8065 3: 93.8710
Ben Roberts - 1: 136.3636 2: 65.4545 3: 98.1818
Malmuth-Weitzman - 1: 130.6452 2: 71.6129 3: 97.7419
Malmuth-Harville - 1: 133.9286 2: 68.5714 3: 97.5000
Swales, Depth 7 (0.005 seconds) 1: 137.9596 2: 66.0476 3: 95.9927

Code:

		
		ABC	CAB	BCA	tot
A	4	39.71	39.31	39	118.02
B	4	38.72	37.96	38.82	115.5
C	2	21.57	22.73	22.18	66.48

Chip chop - 1: 120.0000 2: 120.0000 3: 60.0000
Landrum Burns - 1: 110.0000 2: 110.0000 3: 80.0000
Combined CC/LB - 1: 115.0000 2: 115.0000 3: 70.0000
Declining Chip Count - 1: 117.5000 2: 117.5000 3: 65.0000
Ben Roberts - 1: 118.5714 2: 118.5714 3: 62.8571
Malmuth-Weitzman - 1: 115.0000 2: 115.0000 3: 70.0000
Malmuth-Harville - 1: 116.6667 2: 116.6667 3: 66.6667
Swales, Depth 7 (0.002 seconds) 1: 117.2727 2: 117.2727 3: 65.4546

Code:

		
		ABC	CAB	BCA	tot
A	4	38.9	38.2	38.1	115.2
B	3	30.41	30.29	30.91	91.61
C	3	30.69	31.51	31	93.2

Chip chop - 1: 120.0000 2: 90.0000 3: 90.0000
Landrum Burns - 1: 110.0000 2: 95.0000 3: 95.0000
Combined CC/LB - 1: 115.0000 2: 92.5000 3: 92.5000
Declining Chip Count - 1: 116.3636 2: 91.8182 3: 91.8182
Ben Roberts - 1: 115.6757 2: 92.1622 3: 92.1622
Malmuth-Weitzman - 1: 112.7273 2: 93.6364 3: 93.6364
Malmuth-Harville - 1: 114.2857 2: 92.8571 3: 92.8571
Swales, Depth 7 (0.003 seconds) 1: 117.9379 2: 91.0310 3: 91.0310

The sums of the arithmetic differences between the 36 deep FGS numbers calculated and the numbers calculated using the algorithms are:
Combined: 250.15
Malmuth-Weitzman 155.3066
Malmuth-Harville 74.987
Declining Chip Count 88.8958
Swales 38.85029
Ben Roberts 65.6356

Conclusion.

These are two interesting alternatives for calculating the values of stacks in tournaments. Further research is needed, but the more computationally intensive Swales method may prove to be a more accurate tool for people negotiating chops at final tables etc.

The Declining Chip Count has an accuracy level approaching Malmuth-Harville, but it has lower computational intensity; the time for a calculation is proportional to the number of competitors multiplied by the number of places paid. This may give it a niche as a way to evaluate stacks in situations when too many competitors remain to use other methods.

Quote

02-22-2019 , 02:03 PM

LektorAJ

Carpal \'Tunnel

Join Date: May 2014 Posts: 7,826

I received a PM pointing out an objection to using the Declining Chip Count at the leaf nodes of the Swales method. Basically the issue is that in a setup like 8000-1000-1000 chips, the declining chip count assigns the big stack's finishing probabilities as 80%-20%-0%. While this gives a very accurate representation of the value of the stack in a 2-1-0 SNG, on a satellite bubble the problem would be that the player would be indifferent between having 10000, 9000 and 8000 chips (as with all of those numbers they are seen as being guaranteed to get a seat) and would therefore play randomly.

Using FGS8, the real value of the position is:

Code:

	8000	1000	1000
8, 1, 1	49.93	24.95	25.12
1, 8, 1	49.88	20.96	29.16
1, 1, 8	49.91	23.96	26.13
Avg	49.907	23.29	26.803

so the big stack really is almost certain to go through.

The different stack evaluation systems assign the following values to the stacks:

Chip chop - 1: 80.0000 2: 10.0000 3: 10.0000
Landrum Burns - 1: 45.0000 2: 27.5000 3: 27.5000
Combined CC/LB - 1: 62.5000 2: 18.7500 3: 18.7500
Declining Chip Count - 1: 50.0000 2: 25.0000 3: 25.0000
Ben Roberts - 1: 49.3151 2: 25.3425 3: 25.3425
Malmuth-Weitzman - 1: 47.0588 2: 26.4706 3: 26.4706
Malmuth-Harville - 1: 48.8889 2: 25.5556 3: 25.5556
Swales, Depth 7 (0.003 seconds) 1: 49.9441 2: 25.0280 3: 25.0280

so we can see that the Swales and DCC systems give by far the closest values to FGS8 and any system that is underestimating the value of getting into this position will make strategic errors elsewhere.

The other point I want to make is "horses for courses". While it might not be good to calculate strategy using the DCC at the leaf node of an FGS tree for this reason, it is fine for the final node of the Swales tree as we are now at least seven steps (potentially 7 lost all-ins) removed from where any strategy is calculated and therefore in any normal setups the Swales method will still consider itself to have a decision to make. Also there are multiple uses of evaluation systems besides strategy calculations. For example, evaluating stacks for the purposes of final table deals is another important situation and it's important not to undervalue the big stack in that situation (in fact top players don't trust the values given by Malmuth-Harville as accurately reflecting the value of the big stack).

However, the system for evaluating the leaf nodes isn't essential to the Swales system as with enough depth it tends towards a limit value. Note that in the 8-1-1 satellite bubble setup even with depth 7 it is primarily the collisions that determine the values of the stacks, not the evaluation of the leaf node for the rare paths when 7 stack collisions don't settle it. See below:
Swales, Depth 7 (0.002 seconds) 1: 49.9441 2: 25.0280 3: 25.0280
Swales, Depth 14 (0.743 seconds) 1: 49.9437 2: 25.0282 3: 25.0282
Swales, Depth 15 (1.598 seconds) 1: 49.9437 2: 25.0282 3: 25.0282
Swales, Depth 16 (3.461 seconds) 1: 49.9437 2: 25.0282 3: 25.0282
Swales, Depth 17 (7.491 seconds) 1: 49.9437 2: 25.0282 3: 25.0282
Swales, Depth 18 (15.909 seconds) 1: 49.9437 2: 25.0282 3: 25.0282

There is no problem with people using another system for the leaf nodes if they have a philosophical objection to DCC (for example they could use Malmuth-Harville or to preserve the relative speed of evaluation, they could also evaluate the nodes using a 99-1 mix of DCC and Landrum-Burns)

This effect of DCC considering the big stack to be unable to get 3rd persists even down to a 5800-2100-2100 setup

Chip chop - 1: 58.0000 2: 21.0000 3: 21.0000
Landrum Burns - 1: 39.5000 2: 30.2500 3: 30.2500
Combined CC/LB - 1: 48.7500 2: 25.6250 3: 25.6250
Declining Chip Count - 1: 50.0000 2: 25.0000 3: 25.0000
Ben Roberts - 1: 45.6102 2: 27.1949 3: 27.1949
Malmuth-Weitzman - 1: 42.3358 2: 28.8321 3: 28.8321
Malmuth-Harville - 1: 44.4177 2: 27.7911 3: 27.7911
Swales, Depth 7 (0.004 seconds) 1: 47.2180 2: 26.3910 3: 26.3910
Swales, Depth 12 (0.980 seconds) 1: 47.2172 2: 26.3914 3: 26.3914
Swales, Depth 13 (2.893 seconds) 1: 47.2172 2: 26.3914 3: 26.3914
Swales, Depth 14 (8.461 seconds) 1: 47.2172 2: 26.3914 3: 26.3914

The figures calculated for FGS8 for this position are

Code:

		5800	2100	2100
58,21,21	48.09	25.27	26.63
21,58,21	47.56	24.47	27.97
21,21,58	47.72	24.7	27.58
Avg		47.79	24.813	27.393

In this case too, the Swales algorithm gets results closer to the FGS8 than any of the rival methods and the DCC algorithm is tied for second place with the Ben Roberts algorithm.

Quote

02-25-2019 , 06:04 AM

plexiq

veteran

Join Date: Apr 2007 Posts: 2,554

The Swales algorithm looks interesting, I think there's definitely potential there. But as I mentioned via PM I'd be very hesitant to use DCC as the leaf evaluator in a general setting, at least without a ton of testing for various payout structures and edge cases.

Testing against the averaged FGS-8 values is a decent starting point and the results look promising, but these examples don't really cover the space of possible stack setups very well. Unless I'm mistaken, you've only tested for stacks in the form of x-y-y / y-x-y / y-y-x so far. Will be interesting to see results for more general stack setups and additional players.

I'm a bit swamped w/ other work right now, but I'll run the Swales algorithm against a larger set of true push/fold tournament equities and get back here with the results. (Alternatively LektorAJ, if you have spare time and want to run these tests yourself I'm happy to share the equity data with you.)

Quote

02-25-2019 , 08:59 AM

LektorAJ

Carpal \'Tunnel

Join Date: May 2014 Posts: 7,826

Thanks. The OP has all unique combinations of x-y-z where x+y+z =10000 and x,y and z are multiples of 1000. The second column (with no header sorry) in each group of data is the stack sizes running clockwise round the table divided by 1000.

I would appreciate it if you could send me the data to test this against true equities.

Quote

02-25-2019 , 09:57 AM

plexiq

veteran

Join Date: Apr 2007 Posts: 2,554

The test set is for 2-5 players for all stack combinations adding up to 45 small blinds, with stacks being multiples of a sb. That's about 150k spots total, or roughly 2.6k if you average over the position permutations like you did. Payout structure is 50-30-20. PM'ed you details.

Quote

06-10-2019 , 08:59 AM

LektorAJ

Carpal \'Tunnel

Join Date: May 2014 Posts: 7,826

I tested some of the algorithms over plexiq's test data

The first table shows the sums of the (positive) differences (or errors) between the percentage equity values in plexiq's file and the values calculated for the different algorithms. The first number is for 3-handed spots only, the second number is 4-handed spots only, the third number is 5-handed spots only and the final number is the total (including 1- and 2-handed, there were some very small differences with 2-handed totaling about 0.15 for most algorithms).

ICM 120.8015-1840.853-5942.605-7904.41
Weitzman 197.219-3417.785-11292.4-14907.55
Roberts Fast 148.3483-2061.452-5931.835-8141.785
Roberts Full 148.3483-2067.453-6080.699-8296.65
DCC 99.66301-1218.277-3176.3-4494.388
Swales-DCC1 90.10367-1157.171-3575.669-4823.092
Swales-DCC2 90.04-1246.661-3915.969-5152.819
Swales-DCC3 91.12691-1296.524-4121.038-5508.837
Swales-DCC4 91.73381-1317.516-4210.207-5619.605
Swales-DCC5 91.90543-1329.081-4258.188-5679.322
Swales-DCC6 91.95047-1334.345-4287.84-5714.284
Swales-DCC7 91.98967-1336.889-4304.74-5733.766
Swales-ICM1 103.4742-1583.531-5427.994-7115.147
Swales-ICM2 96.62289-1472.268-4783.856-6352.894
Swales-ICM3 93.87573-1411.755-4544.385-6050.164
Swales-ICM4 92.82415-1375.931-4447.133-5916.036
Swales-ICM5 92.34753-1357.295-4396.615-5846.405
FGS1 71.59267-1222.412-4340.563-5634.405
FGS2 41.84507-871.6478-3371.109-4284.679

So the best here was FGS2>DCC>Swales-DCC1 (i.e. doing one step of the Swales algorithm then using DCC). If your main focus is three hand spots then read the left column. That says FGS2>FGS1>Swales-DCC2

I think it's interesting that doing a couple of steps of the Swales algorithm then just using ICM as before gives about half or 2/3 of the benefits of doing a step of FGS, and presumably the extra computation time required is going to be far less.

Plexiq suggested that we should weight the differences by the actual equity, so getting 22 instead of 20 percent equity is a worse error than getting 42 percent instead of 40 percent (it should be considered as bad as getting 44 percent instead of 40 percent). I disagree as money is money whatever, I also think that this compounds the flaw in the test data, that it contains a lot of spots when someone has a very short stack, but here are the results for the main systems we're looking at anyway:

ICM 3.584041-90.36386-382.5564-476.5081
Roberts-Fast 4.779825-129.3711-489.0189-623.1736
DCC 3.009964-66.39031-278.1449-347.5489
Swales-DCC1 2.907339-77.41879-329.7363-410.0662
Swales-DCC2 2.935635-83.61083-377.8611-464.4113
Swales-DCC7 2.983448-85.79849-405.6566-494.6566
Swales-ICM1 3.16685-80.51255-369.1277-452.8109
Swales-ICM2 3.061852-83-83213-361.1987-448.0964
Swales-ICM5 2.989855-85.91537-399.1025-488.0115
FGS2 1.21979-39.01612-175.6158-215.8537

On this chart FGS2>DCC>Swales-DCC1

One IMHO flaw with the test data is, that as it contains every possible combination of stacks, it is heavily weighted towards situations where someone has a very short stack. Of the 2611 situations covered, in a full 816 of them the short stack has 1SB, and there are another 595 situations where the short stack has 1BB (so together that's more than half the spots examined).

If we sum the differences again, just using the 1200 situations where all the stacks have at least 3 SB, we get the following results

ICM 86.03214-991.0948-2047.573-3124.846
Roberts-Fast 101.8268-835.6983-1293.635-2231.305
DCC 76.24312-573.3028-620.0772-1269.768
Swales-DCC1 62.58628-429.1582-728.9045-1220.794
Swales-DCC2 57.36966-440.0271-802.4176-1299.959
Swales-DCC7 57.66106-468.1184-825.1817-1351.106
Swales-ICM1 68.37142-642.5322-1373.405-2084.453
Swales-ICM2 61.37494-541.0036-931.7481-1534.271
Swales-ICM5 57.92516-476.8388-825.3843-1360.293
FGS2 31.21832-535.2613-1342.828-1909.382
FGS3 17.929636-322.8691-933.7961-1284.647
FGS5 7.00276-195.1627-377.6569-579.8487

Overall, we have
FGS5 > Swales-DCC1 > DCC > FGS3 > ... > Swales-ICM5 > Swales-ICM2 > FGS2 > ... > ICM

Focusing on 3-handed spots we have
Any FGS > Swales-DCC2 > ... > Swales-ICM5 > DCC > ICM

I think part of the reason DCC is getting strong results, particularly for 4 and 5-handed spots is it tends to give a high value to the big stack, and in 9-man SNGs stack swinging is of great importance - when we just look at the data for 3-handed spots it is less accurate.

Quote

06-11-2019 , 05:45 AM

plexiq

veteran

Join Date: Apr 2007 Posts: 2,554

I think you are definitely on to something w/ the Swales model.

Quote:

One IMHO flaw with the test data is, that as it contains every possible combination of stacks, it is heavily weighted towards situations where someone has a very short stack. Of the 2611 situations covered, in a full 816 of them the short stack has 1SB, and there are another 595 situations where the short stack has 1BB (so together that's more than half the spots examined).

About the flaw in the test data: I guess I can calculate relatively easily how often each of the spots is actually reached when playing out the tournament in GTO, and we could then weight the deviations in each spot according to that frequency.

But ultimately, comparing equity estimates is only good for a quick first evaluation anyway. In practice we usually aren't that concerned about the deviation of equity estimates from the true value for a given model, unless we use it specifically for final table deals or something like that. Our main concern is usually how strong the resulting strategies are when calculating with a model.

I tried to evaluate this in two ways:
1) What's the maximal ROI an exploitative player can achieve against a tournament filled with players using the model based strategies.
2) How well do the model based strategies perform in a tournament filled with "true" GTO players.

I'll run the various variants you suggested through these tests, but this requires calculation of the model based Nash ranges for all spots, so that takes a while. But I'll hopefully have a bit more time for side projects soon, I'll post back with results then. I'll pm/email for details if anything is unclear re: implementation.

Last edited by plexiq; 06-11-2019 at 06:00 AM.

Quote

07-02-2019 , 06:25 AM

LektorAJ

Carpal \'Tunnel

Join Date: May 2014 Posts: 7,826

Thinking about this a bit more, the valuations of the very short stacks is going to be a difficult problem to solve. If i understand it correctly, the "test data" file (and for that matter the fgs files) have the flaw that they assume that in a set up like:

BTN 0.5 BB
SB 11 BB
BB 11 BB

the BTN can jam, and SB is forced to choose between risking it's whole stack and potentially coming 3rd or folding, leaving the short stack with 2:1 odds (i.e. 3.00 in euro odds) against the BB.

In fact what happens in real play in these types of spots is that the SB and BB "soft collude" here with the SB completing and the BB checking and they go 3-way to the flop, and only bet if they actually hit something. It's true that they can end up betting out a hand that was going to river something but it would be really more realistic to model these situations as a 3-way flip for a pot of 3BB.

If the simulations assume that 0.5 BB = chance to flip 2-way with 2:1 odds then they are going to severely overestimate the values of those short stacks because the games don't play like that.

TL;DR conclusion is I'm not too bothered if a formula is underestimates the values of those short stacks relative to the simulations, because I think the simulations are wrong here.

Quote

07-02-2019 , 07:01 AM

plexiq

veteran

Join Date: Apr 2007 Posts: 2,554

The calculations in the test data assume strict push-or-fold, so SB would indeed have to fold or risk their whole stack here.

Quote

10-07-2019 , 09:19 AM

#10

LektorAJ

Carpal \'Tunnel

Join Date: May 2014 Posts: 7,826

I've put a calculator for this up on the web as a subpage on a home brew social media group I'm starting.

http://richardferments.com/calculator.asp

So feel free to experiment.

Quote

10-10-2019 , 01:30 PM

#11

coon74

Carpal \'Tunnel

Join Date: Apr 2011 Posts: 10,684

You've slated 'Getting Women' to come soon after 'Getting Yeast' on that group, hmm

Thanks for the calculator! <3

Quote

Post Reply Subscribe

...