Two players are dealt a number ranging from 1-10 (only integers). If P1 is dealt a 5, P2 still has a 1/10 chance to be dealt a 5. There is $100 in the pot.

P1 has the option to check or bet $100.
Given that P1 has bet, P2 has the option to call or fold.
Given that P1 has checked, P2 has the option to bet $100 or check behind.

What would be the solution to this game? Or more importantly, how would I go about solving for the solution?

Thanks in advance to anyone who can provide any insight.

Whoever figured out some sort of numbers or math will back up the well known concept of not betting a middle hand.

I'm sure the numbers will reflect that, so all the help I can give is you should fold a 5 everytime, and in this range It seems that 4-5-6 are all in the same boat, while 3 and 7 are slightly better and playable, it would be with caution. 8-9-10 being premiums.

Of coarse I'm using 1 -2 and 3 as bluff cards to make anything but an 8-9-10 fold.

I'd be interested to see some math on this, as I'm just giving a very genral overview of basic basic basic strategy.

The continuous version of this is examined in MOP = "The
Mathematics of Poker" by Chen and Ankenman ( specifically,
example 17.1, pp.198-203 ). It may be instructive to those
that haven't studied much game theory to look at the
indifference equations for that example.

It turns out for the continous version, the first player only
bets the top 1/6 of his hands and the bottom 1/12 of his
hands as bluffs. Then, after the first player checks, the
second player can value bet the top 13/24 of his hands and
bluff with the worst 13/48 of his hands as bluffs. [ The
general solution for the one bet continuous "toy game" is in
the box of p.202 of MOP. ] Of course, the above sheds some
light on the OP's game in that it is unlikely that P1 can value
bet with anything less than a 9. I am not implying that there
is an immediate "correspondence" from the continous toy
game to the discrete one [ this I'm not sure of ].

In the OP's "toy game", P=1 or this is a "pot-limit" discrete
version. An important idea to note is that the GTCF (game
theoretic calling frequency) for pot-sized bets is 1/2, i.e.,
both players must call with the top 1/2 of their remaining
range.

Just some notation: let x1= P1's minimum legitimate betting
hand, and y1= P2's minimum legitimate betting hand after P1
cheks. Let x be P1's hand and y be P2's hand.

Thus, after P1 bets, P2 must call with the top half of his
range or when y>=6. Thus, you can see right away that
for x=8, this is a borderline betting hand and if P1 were to
always bet with x=8,9,10 he would have to check and call
with 5 exactly 3/4 of the time so as not to be exploited
(P1 would be bluffing with 1 and 2 half of the time so his
range after checking is 3 to 7 and half of the 2's).

It's better to reserve x=8 as a "bluff-catcher". If P1 bets
with x=9, 10 or 1, then he would have to check and call
with 6 to 8 and half of his 5's, but from the solution to the
continuous "toy game", P1 always betting x=9 may be
suboptimal. If P1 is betting "too often", his bluff catching
range will be weaker and thus, P2's value betting range is
wider thereby allowing P2 to bluff slightly more frequently
compared to the case if P1's betting range were narrower.

You would simply want to check how frequently P1 should
bet a 9 as opposed to checking and calling with it. It could
be that x1=9 but P1 might be betting this only 2/3 of the
time (if there is a correspondence with the continous case).
Just remember that the bluffing frequency is half of the
legitimate betting frequency for pot-limit.

After finding how often P1 value bets, all the other variables
can be readily found.

In the continuous version, player 1 would bluff with [0,1/12], bet with [5/6,1] Check folds with [ 1/12,1/2] and check calls with [1/2,5/6]

If player 1 bets, then player 2 folds with [0,1/2] and calls with [1/2,1]

If player 1 checks, then player 2 bets from [0,1/6], checks from [1/6,2/3] and bets from [2/3,1]

This page gives you the details on how to solve similar problems like this.

http://www.math.ucla.edu/~tom/papers/poker2.pdf

We know for sure that a solution would involve a mix strategy. In fact, it appears as if no pure strategy exists.

The answer for the continuous version of the game is fine as well. The solution looks different than I expected. Thanks for the help guys.

Does the author mention how he goes about solving the game?