Suppose instead of Rock-Paper-Scissors, you have the following game:

Fork-Spoon-Chopsticks-Bowl-Plate

Fork > Spoon
Fork > Chopsticks
Spoon > Chopsticks
Chopsticks > Bowl
Bowl > Plate
Plate > Fork

So, Fork is extra powerful because it beats two things, and Chopsticks more vulnerable because it loses to two things.

Any matchup that isn't addressed above (e.g. Spoon vs. Bowl) is a draw.

How do you solve this? What's the optimal strategy?

The optimal strategy depends on the game.

If you are told “you get to play this once against a random player and if you win you get 10 dollars and you lose you die” then spoon, bowl, plate are equal best choices.

If you are told “you get to play this once against an slightly experienced player with the same outcomes as above” then you could assume they will pick bowl, plate or spoon with random likelihood therefore bowl is now the best choice (2/3 draw, 1/3 win) and spoon the second best (always draw).

If you are told “you are playing a random bot 10 times for a small amount of money” then pick anything except chopstick for equal EV.

The optimal play is therefore dependant on the stakes, the number of games, the motivation, and your perception of your opponent.

@David I think they're asking about the Nash equilibrium, which doesn't depend on any of those things. Also, it sounds like you missed some of the rules.

Yeah, this.

I think I have it: never pick Spoon, and pick each other utensil 25% of the time.

Not expecting the solution to be so nice, I did math first and asked questions later. However, one can arrive at the same numbers via reasoning.

Spoon only beats chopsticks. Chopsticks are also beaten by Fork, and since Fork also beats Spoon, we might as well do all of our chopstick-breaking with Fork.

Once we rule out Spoon from our range, we can just look at the non-spoon matchups:

Fork > Chopsticks
Chopsticks > Bowl
Bowl > Plate
Plate > Fork

Ties are inconsequential, so it's clear that we can be safe picking each utensil 25%. No matter what Villain does, they'll have a 25% chance of winning a round and a 25% chance of losing, including if they pick Spoon. However, they'll be exploitable if they pick any spoons.

If hand-wavy explanations aren't your thing then here's the math:

We wanna make Villain indifferent to picking each utensil. Each player wants to maximize the net outcome.

If we pretend they're betting one unit, the EV of a decision is P(win) - P(lose).

Indifference means that P(win w/ Fork) - P(lose w/ Fork) = P(win w/ spoon) - P(lose w/ Spoon) = ...

We know that P(V wins w/ fork) = P(Hero chooses spoon) + P(Hero chooses chopsticks)
and so on.

Let f,s,c,b,p = the frequencies at which we choose each utensil.

Expanding the indifference equation: s+c-p = c-f = b-f-s = p-c = f-b

We also know that f+s+c+b+p = 1, which we can incorporate into the above as follows:

1-f-b-2p = 1-f-p-b-2f = 1-c-p-2f-2s = 1-f-b-s-2c = 1-s-c-p-2b

The 1's cancel and we can flip the signs, so we have:

f+b+2p =
b+p+s+2f =
c+p+2f+2s =
f+b+s+2c =
s+c+p+2b

And now via substitution or however you want, there's enough to solve for each variable.

Awesome, thank you!