Quote:
Originally Posted by getmeoffcompletely
Seems like you don't quite understand how exploitability works. Just because a bot is highly exploitable doesn't mean it's bad. Humans are also extremely exploitable. Imagine you'd have to lay out your entire strategy before playing, by how much do you think someone would be able to exploit you? A metric ton. Humans would do no better than open folding if they were to play their nemesis.
Thanks for stating the obvious. You basically explained (once again) why bot vs human poker competitions are a very noisy measure at best; if a bot loses vs humans over a decent sample one might conclude that the bot is far off from equilibirum, if it wins however, it can mean anything since all that has been achieved is beating an opponent who is likely very far off from equilibrium as well. And how close to equlibirum the bot is is the only thing that matters when you develop algorithms in order to solve large imperfect-information games -- esp as a university that has clearly stated that they're trying to use these techniques in other fields as well (if successful).
Also, "Humans would do no better than open folding if they were to play their nemesis." -- proof?
Quote:
Originally Posted by getmeoffcompletely
Also the other younger guy stated on stream that the main goal of this bot is to beat humans, low exploitability will come later. Actually the same thing happened in limit holdem. The bots first beat humans while still being highly exploitable, low exploitability came quite a few years after that.
It's fine if you got sold in their approach, but that doesn't mean it's scientifically sound. Do you really buy the first statement? If that was the case, CMU would have developed an exploitative bot based on the first 80k hands competition in order to beat humans -- did they do that? No, ofc not, they developed a new bot that they believe is better, ie closer to equilibirum. So beating humans and lower exploitability is exactly the same goal, given the approach. The only point is that you don't need to be super close to Nash to beat humans, but obv there is still value in computing lower bounds of exploitability in oder to compare bots, to test whether algorithms converge (esp if they don't have a theoretical guarantee of converging) etc.
Look at the other thread (Deepstack) -- why do you think the authors went through the pain of measuring a lower bound of exploitability even though they got a decent sample vs humans?
I'm usually a fan of your posts; hopefully you can reflect a little more on this. Even if I were wrong on all of the above, I still have every right to call someone out who does things like setting up a competition where it's extremely likely -- ex ante -- that most results will be statistically insignificant, or says things like HUNL is more complicated than Go.