Overview
Although I’ve described myself as an expert in financial engineering, I’ve hardly touched on financial theory in this blog. While I’d normally want to write about niche, specialized topics, this time I’d like to take a general approach and explain in simple terms the often-cited argument that “you can’t beat the market average in the stock market.” Of course, this claim isn’t meant to be true under all circumstances; rather, it holds true under the somewhat idealized market models used in economics. By understanding the assumptions underlying this claim, we can discuss what actually happens in real-world markets. Also, since this site has been covering poker for about the past year and a half, I’d like to explain this by tying it to poker theory.
*Because I’ve been spending so much time looking at Kioxia’s message boards, I ended up using a Kioxia chart as the featured image—which, in a sense, is the exact opposite of index investing…
Index Fund Investing as a GTO Strategy
To prove the claim that “it is impossible to outperform the market average in the stock market,” we need to assume an “idealized market model, in a sense.” This assumption is central to financial economics and financial engineering; in fact, it can be said that this assumption serves as the fundamental principle of the discipline of financial engineering. That assumption is the “Efficient Market Hypothesis” (EMH). I wrote “hypothesis” because this hypothesis generally does not hold true in real financial markets, and there are quite a few economists who are eager to present evidence against it. However, it is considered a hypothesis that approximates the essence of financial markets, and financial products such as derivatives are priced and traded based on this hypothesis.
The Efficient Market Hypothesis posits that financial markets determine market prices by reflecting all conceivable information, and that fundamental information is reflected immediately. As a result, it becomes impossible to achieve positive returns even by utilizing information or knowledge, and market prices—such as stock prices—follow a random walk. In other words, the economic activity of investing is no different from a game of chance.
Modern portfolio theory has proven that, under these conditions, a market portfolio weighted by market capitalization is the most efficient portfolio—one that yields the highest return relative to risk (it is included in the set of portfolios with the lowest risk relative to return, known as the “efficient frontier”). This means that even if you construct a portfolio of individual stocks that deviates from this market portfolio, the expected return remains unchanged while risk alone increases.
Ultimately, under this hypothesis, it is rational for all investors to hold exactly the same portfolio; creating a portfolio that deviates from it would amount to nothing more than a negative-expected-value play (a strategy that can be exploited). Furthermore, if all market participants held the same portfolio, it could be nothing other than the market average portfolio. In other words, under the efficient market hypothesis, investing in the market-average portfolio is equivalent to what is known as a GTO strategy in poker.
Of course, in reality, information asymmetry (such as insider information or differences in information-gathering capabilities) does exist, and there are likely quite a few people who bet their entire fortune on investments despite lacking knowledge of financial markets (irrational decision-making). In poker terms, if there are many players who cannot correctly assess the strength of their hands or who frequently make mistakes, it becomes possible to profit by employing exploitative strategies. In markets where the Efficient Market Hypothesis does not hold,
- the presence of irrational players can cause prices to deviate significantly from their fair value due to psychological factors such as panic selling (fear) or herd investing (greed).
- If your opponents are making mistakes (prices are distorted), adopting a strategy that exploits those distortions (such as value investing, momentum investing, or quantitative investing) will yield a higher expected value than maintaining a market-average portfolio (GTO).
That is the conclusion.
In summary, the essence of index fund investing (market-average portfolio investing) is that
- you cannot identify market distortions (cannot exploit them)
- In that case, one should stick strictly to a 100% defensive GTO strategy that cannot be exploited by anyone.
Because it is this kind of strategy, index fund investing is often referred to as a passive investment strategy. If you were a poker player, you might be considered a boring guy who’s normally strong but plays an extremely tight, defensive game. You’d be a player who can’t win big but follows a strategy that makes it hard to lose. Under the Efficient Market Hypothesis, where everyone adopts a GTO strategy, index fund investing becomes the one and only unbeatable strategy.
Trend-Following and Contrarian Strategies as Exploitation Strategies
Well, that said, if the real world were a market that satisfied the Efficient Market Hypothesis, the stock market would be nothing more than a gambling den, and thinking about investment strategies would be a waste of time—which would end the discussion. Therefore, in the following, we’ll present arguments that refute the Efficient Market Hypothesis. Investment strategies that aim to achieve returns exceeding the market average—whether by holding a portfolio with a different composition than the market average or by timing market trades—are referred to as active investment strategies.
Under the Efficient Market Hypothesis, strategies such as trend-following and contrarian investing that use fundamental or technical analysis are all ineffective; adopting an investment strategy based on these analyses implies a rejection of the Efficient Market Hypothesis. Both fundamental investing and strategies based on technical analysis—such as momentum investing and contrarian investing—can be classified as exploitative strategies that capitalize on market inefficiencies or irrationalities.
In fundamental analysis, the basic strategies are value investing—which focuses on undervalued stocks—and growth investing—which focuses on growth potential. While stocks that are neither undervalued nor have growth potential may sometimes surge in price simply because they are unpopular, this is known as “speculative stock” (or “junk stock”) investing. While large institutional investors are unlikely to engage in this, it has become a fairly well-recognized investment strategy among retail investors. Both strategies are based on the premise of market inefficiencies, where undervaluation or growth potential is not reflected in stock prices.
In technical analysis, which focuses on chart analysis to predict stock price trends, trading strategies are classified based on whether trades follow the trend (trend-following, momentum investing) or go against it (contrarian trading). Trading styles such as day trading and swing trading—which involve repeated short-term trades based on pattern analysis, regardless of market trends—are also commonly used. It is not uncommon for retail investors to trade without fully understanding whether they are following the trend or taking a contrarian approach. Both of these trading methods are based on the assumption that the autocorrelation coefficient of stock prices is not zero; as such, they are incompatible with the Efficient Market Hypothesis, which posits that the autocorrelation coefficient is zero (random walk).
For retail investors managing small amounts of capital, active investment strategies may still offer a chance of success. However, empirical analysis has shown that for institutional investors managing assets exceeding several trillion yen, the average return on active investment strategies falls short of that of index funds. For this reason, it is believed that institutional investors managing massive assets—such as pension funds—allocate the vast majority of their equity investment capital to index funds.
How Do Hedge Funds Detect Trends and Reversals?
As explained in the previous section, institutional investors such as pension funds, insurance companies, and mutual funds tend to buy and sell based on fundamentals, and it is believed that most of them currently adopt passive investment strategies (index investing). Consequently, they often do not exert significant influence on the short-term trading of individual stocks. The key players in short-term price movements are hedge funds that engage in algorithmic system trading—often referred to as CTAs or quant funds—and it is highly likely that short-term trends and reversals are driven by the algorithms of these institutional investors. While these firms are generally not well-known by name, it is believed that they often participate in the market by utilizing margin trading services provided by foreign investment banks such as Goldman Sachs and Morgan Stanley. These services for institutional investors are known as prime brokerage services.
Incidentally, to the best of the author’s knowledge, foreign investment banks such as Goldman Sachs and Morgan Stanley currently engage in very little proprietary trading to influence the market. The positions of these financial institutions that appear in large shareholding reports and short selling disclosures stem from margin trading by clients such as hedge funds; it is therefore presumed that it is these clients—including hedge funds—held by foreign investment banks that are engaging in the types of trades that dominate the market and move stock prices. Consequently, it can be inferred that the focus of research should not be on the trading tactics of securities firms, but rather on the trading strategies of the hedge funds (CTAs and quant funds focused on short-term trading) operating behind the scenes.
So, how do speculative players—such as hedge funds focused on short-term trading—predict stock price trends and identify turning points? While some may use chart analysis like retail investors or employ more sophisticated quantitative models, in reality, it appears they base their decisions on what is known as “alternative data”—information generally inaccessible to retail investors.
Alternative data includes things like social media posts and messages on Yahoo message boards (though “reading the board” typically refers to analyzing order books, some people apparently use the term to describe reading message board posts as well); however, in reality, this data is not considered particularly significant. Among alternative data sources, PFOF (Payment for Order Flow) is particularly significant. Some online brokerages and discount brokers sell aggregated data—comprising customer order and position data—to hedge funds for a fee. This means hedge funds trade while having insight into data such as the prices at which the general public is taking positions and the extent of their losses.
Of course, it is not possible to obtain data from every securities firm, and you might think that since the clients of online brokerages and discount brokers are mostly small-scale retail investors, it would be impossible to capture the trading trends of the entire retail investor population. However, statistical data often allows us to infer the big picture by observing only a very small portion of the data. During elections, results such as “certain to win” are announced even when less than 1% of the votes have been counted; similarly, in stock trading, it is well known that even with just 1% of the data, the overall picture tends to be roughly the same. As a result, hedge fund algorithms can track retail investors’ trading trends and profit/loss situations, and by executing trades designed to maximize those investors’ losses, they can manipulate stock prices.
Retail investors desperately try to anticipate these situations through chart analysis, but hedge funds utilize data that allows them to make even more precise predictions. Furthermore, they seek to profit by exerting influence on stock prices through their substantial financial resources. Therefore, it is important to note that trading based on technical analysis often struggles to maintain an advantage against hedge fund strategies. If you use technical analysis, you’ll need more than just memorizing simple chart patterns; you’ll also need the imagination to infer information such as the prices at which the general public has taken positions and the extent of their losses.
How Hedge Funds “Gamble” on Earnings Reports
While investment based on fundamental analysis may bring to mind long-term investing, it is also possible to engage in short-term trading using fundamental analysis. This can be achieved by taking positions immediately before or after earnings announcements and closing those positions within a short timeframe. If executed successfully, this can yield very high returns in the short term. Often referred to as “earnings gambling,” hedge funds that employ this strategy are known as event-driven hedge funds.
Of course, there are cases where human analysts predict earnings and execute trades, but even here, algorithmic trading plays the leading role. The sharp spikes or plunges in stock prices immediately following earnings announcements are caused by algorithmic trading. When earnings are announced during the trading day, prices often fluctuate wildly or surge and plummet immediately afterward, which might lead you to wonder, “How on earth are they making these decisions?”
Of course, this isn’t based on human judgment; orders are placed based on algorithmic analysis. You might wonder if AI is reading PDF files in an instant, but in addition to PDF files, financial results are published in XBRL format—which is designed for easy system processing—and by reading this data, numerical figures from the financial results can be processed instantly. The algorithm compares the announced data with its own forecast; if the actual results exceed the forecast, it automatically places a buy order, and if they fall short, it places a sell order.
While the financial results predicted by the algorithm in this case may sometimes be generated internally, it generally appears that the system relies on analyst estimates released by various securities firms. Consequently, if the actual financial results exceed analyst estimates, buy orders are more likely to be triggered, and if they fall short, sell orders are more likely to be triggered. Of course, as explained in the previous section, the system also monitors order flow data in addition to earnings data; therefore, even if earnings are strong, the stock price may still fall if many retail investors have already bought the stock in advance. This is often referred to as “good news already priced in” (the opposite being “bad news already priced out”). It is important to note that focusing solely on earnings data can often lead to disappointment.
Summary: The Modern Stock Market’s “YouTube-ization”
I have outlined the hypothesis that the stock market has become a platform dominated by algorithms provided by hedge funds. The essence of these mechanisms shares an identical structure to Google’s search algorithms and YouTube’s recommendation system. In the financial world, such mechanisms are apparently referred to as “attention-based trading algorithms.” Just as YouTube directs viewers by recommending videos that are experiencing a surge in click-through rates and view counts, hedge funds are believed to monitor news, earnings reports, and retail investor order data to shape overall market trends in a way that maximizes their own profits, thereby influencing the behavior of retail investors.
In such a market, analyzing investor sentiment through fundamental analysis of information that shapes stock prices or through technical analysis is not particularly effective (though, of course, it cannot be said to be completely ineffective); rather, the ability to optimize one’s actions in line with the algorithms provided by hedge funds becomes a key factor in generating profits in the market. Furthermore, for many retail investors, these algorithms remain a “black box” and are difficult to understand.
Based on the above analysis, it can be inferred that whether one engages in momentum investing (or contrarian investing) based on technical analysis or “earnings gambling” based on fundamental analysis, without highly sophisticated knowledge and analytical skills, one will often end up as prey for hedge funds. It is well known that many of these investments are nothing more than gambles with negative expected returns (much like pachinko), and that while investors may occasionally win, they tend to lose their capital over the long term and are forced out of the market.
Of course, that doesn’t mean you should only invest in index funds—that’s hardly an appealing proposition either. While the average return on index funds is said to be around 7% per year, the reality is that even if a poor person achieves a 7% annual return, they’ll likely die before they become rich. It’s not uncommon for someone who has been lucky enough to acquire a massive fortune to lose their entire wealth through investing about ten years later—which shows that in the gamble of investing, a streak of good luck can indeed continue. How to distinguish whether profits from investing are due to luck or to investment skill is also a significant issue in financial theory.
Comments