Pre-flop Player Model

Introduction

I’ve been thinking about Texas Hold’em strategies for a long time, but I’d like to think about the behavior model of NPCs in the game. However, Texas Hold’em is more complicated than I thought at first, and it’s difficult to think of a behavior model for NPCs that play like humans. The purpose of this article is to identify the components that need to be considered in a preflop strategy, and to provide some idea of ​​what it would be like to develop a preflop player model.

Components of Pre-Flop Strategy

I believe the most important factors in pre-flop strategy are:

1. Hand Preference Order
2. VPIP (loose or tight)
3. Bet Sizing and Raising Strategy
4. Stack-Based Betting Strategy
5. Whether or not ante and rake are taken into consideration
6. Whether or not to consider position, whether or not the raiser is OOP/IP
7. Betting strategy according to the number of participants (passive or aggressive in multi-way)
8. Whether or not they bluff and how often

Of these, the most basic are 1. to 3., and in this article we will consider a model that reflects these components. Of particular importance is how to express the preference ranking of cards in 1., and we will consider in detail how to model this.

Hand Preference Order

I mentioned earlier that the probability of winning in suited or connected poker changes relatively between heads-up and multi-way, but since the number of people participating after the flop is uncertain, in reality players will have different preferences for hands. Depending on post-flop strategy, the preferred hands may not necessarily be in order of winning probability. For example, a real player might prioritize entering hands according to their preferences, such as:

1. Evaluation of pocket pairs
2. Evaluation of suited
3. Evaluation of connectivity (Connected, 1-Gapper, 2-Gapper, 3-piece straight possible, No Connectivity)
4. Evaluation of the rank of the stronger card (importance of the higher rank of a pair)
5. Evaluation of the rank of the weaker card (importance of kicker)

For example, players may have different opinions on which they prefer: 77 or JTs, A9o or 65s, or K2s or Q7s. Players who have high ratings for each tend to have the following characteristics:

1. They like pocket pairs and are bullish on them.
2. They like flushes, and suited hands want to see the flop.
3. They like straights and care more about how easy it is to make a straight than the strength of their rank.
4. They like Aces and Ks, and become bullish when they have an A or K in their hand.
5. Even if they have an A or K, they become anxious and cannot play aggressively if their kicker is weak.

 By creating a scoring model for these evaluation items, and giving players a higher score, we can prioritize their hand based on their preferences. For example, we could consider the following model.

1. Dummy variable (pocket pair) \(\small x_1\): 1 if there is a pocket pair, 0 otherwise
2. Dummy variable (suited) \(\small x_2\): 1 if Suited, 0 otherwise
3. Connectivity \(\small x_3\): Connected is 4, 1-Gapper is 3, 2-Gapper is 2, 3-card straight possible is 1, otherwise 0
4. Ranking of stronger cards \(\small x_4\): A is 12, K is 11, … 2 is 0
5. Ranking of weaker cards \(\small x_5\): A is 12, K is 11, … 2 is 0

Then, We could simply build a model like

\[ \small s = \beta_1 x_1+\beta_2 x_2+\beta_3 x_3+\beta_4 x_4+\beta_5 x_4^2+\beta_6 x_5+\beta_7 x_5^2 \]

by giving parameters \(\small \beta_1,\cdots, \beta_7\). For the Connectivity item, we will assign the following parameters to each hand based on the probability of a straight being made.

• \(\small x_3=4\): JT, T9, 98, 87, 65, 54
• \(\small x_3=3\): QJ, QT, J9, T8, 97, 86, 75, 64, 53, 43
• \(\small x_3=2\): KQ, KJ, KT, Q9, J8, T7, 96, 85, 74, 63, 52, 42, 32
• \(\small x_3=1\): AK, AQ, AJ, AT, A5, A4, A3, A2, K9, Q8, J7, T6, 95, 84, 73, 62
• \(\small x_3=0\): 上記以外

The rank evaluation is a quadratic function taking into account the curvature. For example, if the stronger card is an A or K, a special evaluation tends to be given, so a linear model may not be able to properly reflect the strength of the hand.

 Using the above model, we can construct a model that represents the preferences of various players, but there is a preference ranking among the cards that must be true no matter who thinks about it. For example, with pocket pairs, AA > KK > … > 22 is always true. This is a matter of rationality, not preference. Since the probability of each hand being made is the same, and the probability of winning increases with the rank, a player who prefers 77 to AA must be considered irrational. Similarly, there is no reason to prefer the off-suit over the suited, even if they are of the same rank. It is best to assume that players will not generally choose between 87o and 86o, 87o and 76o, A8o and K8o, or K8o and K7o. Such a hand of strength, where the ranking does not change depending on the player’s preference, is often called dominate. For example, AA would dominate KK (note that dominating does not necessarily mean winning). If you find yourself all-in preflop and think, “Oh no,” it’s easier to understand if you think that in most cases your opponent’s hand dominates your own.

 If we think about it this way, even if it reflects the players’ preferences, preferences that upset the dominance relationship are not appropriate. Of the first five evaluation criteria, except for whether or not it is a pocket pair, the higher the value, the more positive the evaluation that should be given. In other words, it must be \(\small \beta_2,\beta_3 \geq 0\), and this means that the parameters need to be determined within a range that satisfies

\[ \small \begin{align*} \beta_4 x_4 + \beta_5 x_4^2 \geq 0 \quad x_4 \in\{0,1,\cdots,12\} \\ \beta_6 x_5 + \beta_7 x_5^2 \geq 0 \quad x_5 \in\{0,1,\cdots,12\} \end{align*} \]

and

\[ \small \begin{align*} \beta_4 + 2\beta_5 x_4 \geq 0 \quad x_4 \in\{0,1,\cdots,12\} \\ \beta_6 + 2\beta_7 x_5 \geq 0 \quad x_5 \in\{0,1,\cdots,12\}. \end{align*} \]

For now, a simple solution would be to always ensure that the coefficients of the score model are non-negative.

Parameter Setting Example

Let’s assign the following numerical values ​​to the specific parameters. Each parameter will be evaluated on an 11-point scale from 0 to 10, and the score will be calculated by multiplying the evaluation by the following coefficients (the coefficients are small because they are balanced by applying regression analysis to the winning probability of each hand, but the results will not change even if all the coefficients are multiplied by 100).

\[ \small (\beta_1, \beta_2, \beta_3, \beta_4, \beta_5, \beta_6, \beta_7) = (0.05, 0.008, 0.002, 0.0028, 0.0002, 0.0004, 0.0002) \]

For example, if you set the value of a pocket pair to 5, you add up the scores in the format \(\small s= s + 0.05 \times 5 \) and determine the hand to enter in order of the highest total.

 The combination of VPIP and hand preference parameters for each strategy can represent the behavior of various players. For example, we can consider the following examples.

	3bet/3bet	2bet/3bet	2bet/2bet	1bet/3bet	1bet/2bet
NIT	5%	—	—	—	—
3better	10%	—	—	—	—
Loose 3better	15%	—	—	—	—
Tight Aggressive	5%	10%	20%	—	—
Tight Passive	2%	7%	20%	—	—
Loose Aggressive	7%	15%	50%	—	—
Loose Passive	2%	7%	50%	—	—
Limper	2%	7%	20%	—	60%
Limp Caller	2%	10%	—	60%	—
Trap Limper	5%	10%	20%	2%	60%

Prioritize their hand by prioritizing the betting strategy with the lowest VPIP. For Trap Limper, 1bet/3bet is prioritized first, and then 3bet/3bet is assigned to their hand. It is relatively easy to imagine the preference order of hands for the following combinations of parameters.

	Pocket Pair	Suited	Connected	Rank (High)	Rank (Low)
Pocket Pair Freak	8	3	3	5, 5	5, 5
Flush Freak	4	10	6	3, 3	3, 3
Straight Freak	2	4	10	1, 1	1, 1
AK Freak	3	3	3	8,8	3,3
Dislike Weak Kicker	3	3	3	3,3	8,8
Speculator	6	8	10	2,2	2,2
Rank Prioritized	3	1	1	8,8	8,8
A-High Flush Freak	3	8	3	8,8	3,3

 In fact, I implemented a program that can express what hand range will result when you specify VPIP and manipulate the parameters \(\small \beta_1,\cdots,\beta_7\). By varying the VPIP and \(\small \beta_1,\cdots,\beta_7\) settings, we will be able to represent players with a variety of preflop strategies without compromising rationality.

When we think of hand ranges, we often think about VPIP and how to counter loose or tight players. However, it might be interesting to think about how to play when you take into account various characteristics, such as a hand range that is biased towards suited, or when you have A or K, or you only want to play pocket pairs.

Conclusion

This article is getting long, so I will write a sequel next time. Real players do not take such definite actions for each hand (depending on their mood, they may become timid even with strong cards, or aggressive even with weak cards. There are also cases where they are influenced by the actions of their opponents.), and there may be cases where they intentionally use a mixed strategy to prevent their opponents from knowing the strength of their hand.

 Another thing is that I want to incorporate cases where players suddenly go all-in into their behavior. Generally, cases where players suddenly go all-in are as follows:

1. They usually go all-in on pocket pairs (AA, KK, QQ) (players who like pocket pairs may also go all-in on such as 55 or 33).
2. They are not confident in their ability to play well after the flop when holding strong cards (AK, AQ, AJ, KQ, etc.), so they go all-in and bet on a high probability of winning.
3. Since there were many players in front of them who were making weak moves such as limping in, they went all-in to steal the bet.
4. If all the players in front of them have folded, go all-in as a bluff from the closest position (BTN or SB).
5. Since their stack is only 20-30bb, go all-in instead of 3-betting. If they have 10bb or less, go all-in instead of open-raising. (In either case, they will likely end up going all-in after the flop.)
6. Having recently lost a lot of chips, they lose their cool and go all-in regardless of what they have in hand.

1. and 2. could be categorized as All-in/All-in in the betting strategy category, but I feel that the strategy is a bit complicated. There are cases where they go all-in after an open-raise, and cases where they go all-in with a re-raise after being 3-bet. It seems that in many cases this also doubles as a bluff or steal like 3. or 4. We should think of it this way: there are hands where they can go all-in only when they are in a certain position or situation. We’ll look at how to express these next time.