Quantitative Trading:How to Build Your Own Algorithmic Trading Business(2)

"Quantitative Trading: How to Build Your Own Algorithmic Trading Business" by Ernie Chan is a comprehensive guide that explores the world of quantitative trading and provides practical advice for building a algorithmic trading business, especially for individuals interested in quantitative trading.

It covers essential concepts, methodologies, and practical tips to help readers develop and implement their own algorithmic trading strategies while effectively managing risk and building a sustainable trading business.

Due to the significant benefits I received from reading this book, I want to write down some of the most important takeways I get from this book, with the hope that it can also be useful to you.

This passage is about the last four chapters, which introduces execution system in actual trading(automated and semi-automated), how to minimize transaction cost and determine the optimal leverage using the Kelly Criterion. It also talks about some special topics or common sense in trading. Finally, it lists some advantages of individuals investors over institutional investors.

Execution Systems

This chapter is about building an automated trading system and ways to minimize trading costs and divergence with your expected performance based on your backtests.

Automated Trading System

An automated trading system is a piece of software that automatically generates and transmits orders to your brokerage account based on your trading strategy.

• It eliminates manual operation so that you can simultaneously run multiple strategies.
• Most importantly, it allows speedy transmissions of orders,which is essential to high-frequency trading strategies.

There are two kinds of trading systems: fully automated and semiautomated, and the choice depends on the frequency of your trading.

Fully Automated Trading System

A fully automated trading system can run the trading algorithm in a loop again and again, constantly scanning the latest prices and generating new waves of orders throughout the trading day. The submission of orders through an API to your brokerage account is automatic, so you would not need to load the trades to a basket trader or spread trader

A fully automated system has the advantage that it minimizes human errors and delays. For certain high-frequency systems, a fully automated system is indispensable, because any human intervention will cause enough delay to seriously derail the performance.

However, a fully automated system is also complicated and costly to build, often requiring professional programmers with knowledge of high-performance programming languages such as Java, C, or C++ in order to connect to your brokerage’s application program- ming interface (API).

Semiautomated Trading System

In a semiautomated trading system, the trader still needs to manually upload a text file containing order details to a basket trader or spread trader, and manually press a button to transmit the orders at the appropriate time. However, the order text file can be automatically generated by a program such as Python.

Whether you have built a semiautomated or a fully automated trading system, there is often a need for input data beyond the prices that your brokerage or data vendor can readily provide you. For example, earnings estimates or dividends data are often not provided as part of the real-time data stream.

These non-price data are typically available free of charge from many web sites, but are usually embedded in an HTML format and not readily usable. Hence, an automated system also must be able to retrieve such web pages, parse them, and reformat them into a tabular format that your trading strategy can utilize

Minimizing Transaction Costs

In the previous chapter, we had talked about how to reduce transaction costs. Besides changing your brokerage or proprietary trading firm to one that charges a lower commission, there are a few things you can do in your execution method to minimize the transaction costs.

And the book regards this mainly as a matter of not allowing your order size to be too big relative to its average trading volume and relative to its market capitalization

Cut Down on Commissions

You should refrain from trading lowprice stocks. Typically, institutional traders do not trade any stocks with prices lower than $5

Because low-price stocks increase your total commissions costs (since you need to buy or sell more shares for a fixed amount of capital)（Why？？）

Also it has a wider bid-ask spread and therefore increase your total liquidity costs

The definition of bid-ask spread from ChatGPT is as follow

The bid-ask spread is a concept used in financial markets, particularly in securities trading such as stocks, bonds, commodities, and foreign exchange. It refers to the difference between the highest price a buyer is willing to pay (the bid price) and the lowest price a seller is willing to accept (the ask price) for a particular asset or security.

A wider spread can indicate lower liquidity or higher transaction costs. Market conditions, supply and demand dynamics, trading volume, and the nature of the asset being traded can all influence the bid-ask spread.

Minimize Market Impact Cost

We should limit the size (number of shares) of your orders based on the liquidity of the stock, on which we can do two things:

• limit the size of order according to the certain measurement(like average daily volume)
• allocate assets effectively between large-cap and small-cap stocks

One common measure of liquidity is the average daily volume (it is your choice what lookback period you want to average over). As a rule of thumb, each order should not exceed 1 percent of the average daily volume. As an independent trader, you may think that it is not easy to reach this 1 percent threshold, and you would be right when the stock in question is a large-cap stock belonging to the S&P 500. However, you may be surprised by the low liquidity of some small-cap stocks out there

Another way to reduce market impact is to scale the size of your orders based on the market capitalization of a stock.

A linear scale (i.e., scaling the capital of a stock to be linearly proportional to its market capitalization) would result in practically zero weights for most small- and micro- cap stocks in your portfolio, and this will take away any benefits of diversification

If we were to use linear scale, the capital weight of the largest large-cap stock will be about 10,000 of the smallest small-cap stock. To reap the benefits of diversification, we should not allow that ratio to be more than 10 or so, provided that the liquidity (volume) constraint described above is also satisfied.

Slippage

Many institutional traders who desire to execute a large order will break it down into many smaller orders and execute them over time. This method of trading will certainly reduce market impact; however, it engenders another kind of transactions costs, namely, slippage，

Reducing market impact in this way may increase slippage, it is not really suitable for retail traders whose order size is usually not big enough to require this remedy

Slippage is usually outside of your control, perhaps your brokerage’s execution speed is simply too slow, due to either software issues (their software processes your orders too slowly), risk control issues (your order has to be checked against your account’s buying power and pass various risk control criteria before it can be routed to the exchange), or pipeline issues (the brokerage’s speed of access to the exchanges). Or perhaps your brokerage does not have access to deep enough “dark-pool” liquidity. These execution costs and issues should affect your choice of brokerages

Paper Trading

Paper trading refers to the practice of simulating trades in a risk-free environment using virtual money instead of real capital.

When performing paper trading, there are several key aspects to pay attention to in order to make the experience as valuable and realistic as possible

You should be able run your ATS, execute paper trades, and then compare the paper trades and profit and loss (P&L) with the theoretical ones generated by your backtest program using the latest data. If the difference is not due to transaction costs (including an expected delay in execution for the paper trades), then your software likely has bugs.

It gives you better intuitive understanding of your strategy, including the volatility of its P&L, the typical amount of capital utilized, the number of trades per day, and the various operational difficulties including data issues.

There are some common problems in paper trading, some of them are listed in the following

The moment you start paper trading you will realize that there is a glaring look-ahead bias in your strategy—there may just be no way you could have obtained some crucial piece of data before you enter an order

If you are able to run a paper trading system for a month or longer, you may even be able to discover data-snooping bias, since paper trading is a true out-of-sample test.

Backtesting also won’t reveal the operational difficulties, such as how fast you can download all the needed data before the market opens each day and how you can optimize your operational procedures in actual execution, but paper trading will make you feel all these.

In summary, paper trading allows you to:

• Discover software bugs in your trading strategy and execution programs.
• Discover look-ahead or even data-snooping bias.
• Discover operating difficulties and plan for operating schedules.
• Estimate transaction costs more realistically.
• Gain important intuition about P&L volatility, capital usage, portfolio size, and trade frequency.

Actual Performance V.S Expectations

What do you do in the situation when your live trading underperforms your backtest?

The list of what possibly may have caused this divergence from expectation is as follows

• Do you have bugs in your ATS software?
• Do the trades generated by your ATS match the ones generatedby your backtest program?
• Are the execution costs much higher than what you expected?
• Are you trading illiquid stocks that caused a lot of market impact?

You can start by addressing the usual problems: Eliminate bugs in the strategy or execution software; reduce transaction costs; and simplify the strategy by eliminating parameters. But, fundamentally, your strategy still may have suffered from data-snooping bias or regime shift.

• data-snooping bias

You can verify this cause by trying to eliminate as many rules and as many parameters in your strategy as possible. If the backtest performance completely fell apart after this exercise, chances are you do have this bias and it is time to look for a new strategy.

• regime shifts

When the financial market structure or the macroeconomic environment undergoes a drastic change so much so that trading strategies that were profitable before may not be profitable

One regime shift is relevant if your strategy shorts stocks.

Many stocks, especially the small-cap ones or the ones with low liquidity, are “hard to borrow.” For you to be able to short a stock, your broker has to be able to borrow it from someone else (usually a large mutual fund or other brokerage clients) and lend it to you for selling.

If no one is able or willing to lend you their stock, it is deemed hard to borrow and you would not be able to short it. Hence, again, a very profitable historical short position may not actually have been possible due to the difficulty of borrowing the stock.

Money and Risk Management

Kelly Criterion

This chapter provides an important tool for risk management: the determination of the optimal leverage using the Kelly Criterion.

It also determines the optimal allocation of capital among different strategies, based on the covariance of their returns.

The general form of the Kelly Criterion is as follows:

\[f^{*} = \frac{p}{a} - \frac{q}{b}\]

where - \(f^{*}\) is the fraction of the assets to apply to the security. - \(p\) is the probability that the investment increases in value. - \(q\) is the probability that the investment decreases in value(\(q=1-p\)) - \(a\) is the fraction that is lost in a negative outcome. If the security price falls 10%, then \(a=0.1\) - \(b\) is is the fraction that is gained in a positive outcome. If the security price rises 10%, then \(b=0.1\)

In an intuitive understanding of the Kelly Criterion, it involves taking the probability of winning divided by the multiple of the wagered capital when you lose (typically 1 in the case of gambling), and then subtracting the probability of losing divided by the multiple of the capital gained when you win.

Since this formula can result in Kelly fractions higher than 1. In this case, it is theoretically advantageous to use leverage to purchase additional securities on margin.

The proof of Kelly Formula is also easy to understand, the details in wiki is as follows

But it's important to note that the Kelly Criterion is not suitable for all situations. It assumes that you know the odds or returns and expected probabilities, while ignoring market uncertainties and other factors. Therefore, in practical applications, the Kelly Criterion is often combined with real-world considerations and risk tolerance to formulate more reasonable and feasible money management strategies.

Also, the Kelly criterion is valid only for known outcome probabilities, which is not the case with investments. Risk averse investors should not invest the full Kelly fraction.

Mind-Set

The ultimate risk management mind-set is very simple: Do not succumb to either despair or greed. To gain practice in this psychological discipline, one must proceed slowly with small position size, and thoroughly test various aspects of the trading business (model, software, operational procedure, money and risk management) before scaling up according to the Kelly formula.

In order to proceed slowly and cautiously, it is helpful to have other sources of income or other businesses to help sustain yourself either financially or emotionally (to avoid the boredom associated with slow progress). It is indeed possible that finding a diversion, whether income producing or not, may actually help improve the long-term growth of your wealth.

Special topics

The special topics in this book includes

• mean reversion and momentum
• regime switching
• stationarity and cointegration
• factor model
• seasonal trading models
• high-frequency trading.

Mean Reversion and Momentum

Trading strategies can be profitable only if securities prices are either mean-reverting or trending. Otherwise, they are random-walking, and trading will be futile.

At any given time, stock prices can be both mean reverting and trending depending on the time horizon you are interested in. Constructing a trading strategy is essentially a matter of determining if the prices under certain conditions and for a certain time horizon will be mean reverting or trending, and what the initial reference price should be at any given time.(When the prices are trending, they are also said to have “momentum,” and thus the corresponding trading strategy is often called a momentum strategy.)

There are multiple reasons for momentum, slow diffusion of information, private liquidity needs and herdlike behavior

• slow diffusion of information

Momentum can be generated by the slow diffusion of information—as more people become aware of certain news, more people decide to buy or sell a stock, thereby driving the price in the same direction.

Stock prices may exhibit momentum when the expected earnings have changed. This can happen when a company announces its quarterly earnings, and investors either gradually become aware of this announcement or they react to this change by incrementally executing a large order .

• private liquidity needs

Besides the slow diffusion of information, momentum can be caused by the incremental execution of a large order due to the liquidity needs or private investment decisions of a large investor.

With the advent of increasingly sophisticated execution algorithms adopted by the large brokerages, it is, however, increasingly difficult to ascertain whether a large order is behind the observed momentum.

• herdlike behavior

Momentum can also be generated by the herdlike behavior of investors: investors interpret the (possibly random and meaningless) buying or selling decisions of others as the sole justifications of their own trading decisions.

But momentum regimes generated by private liquidity needs and herdlike behavior have highly unpredictable time horizons.

The most important things we need to pay attention to may be the effects of increasing competition from traders with the same strategies

• For mean-reverting strategies, the effect typically is the gradual elimination of any arbitrage opportunity, and thus gradually diminishing returns down to zero. When the number of arbitrage opportunities has been reduced to almost zero, the mean-reverting strategy is subject to the risk that an increasing percentage of trading signals are actually due to fundamental changes in stocks’ valuation and thus is not going to mean revert.

• For momentum strategies, the effect of competition is often the diminishing of the time horizon over which the trend will continue. As news disseminates at a faster rate and as more traders take advantage of this trend earlier on, the equilibrium price will be reached sooner. Any trade entered after this equilibrium price is reached will be unprofitable.

Regime Switching

The desire to predict regime switches, which are also commonly known as turning points, is also as old as financial markets themselves

Academic attempts to model regime switches in stock prices generally proceed along these lines（Markov regime switching or hidden Markov models）

1. Propose that the two (or more) regimes are characterized by different probability distributions of the prices. In the simplest cases, the log of the prices of both regimes may be represented by normal distributions, except that they have different means and/or standard deviations.

2. Assume that there is some kind of transition probability among the regimes.

3. Determine the exact parameters that specify the regime probability distributions and the transition probabilities by fitting the model to past prices, using standard statistical methods such as maximum likelihood estimation.

4. Based on the fitted model above, find out the expected regime of the next time step and, more importantly, the expected stock price.

The method above assumes constant transition probabilities among regimes at all times. In practice, this means that at any time, there is always a very small probability for the stock to transition from a normal, quiescent regime to a volatile regime. But this is useless to traders who want to know when—and under what precise conditions—the transition probability will suddenly peak. This question is tackled by the turning points models.

Stationarity AND Cointegration

A time series is “stationary” if it never drifts farther and farther away from its initial value. In technical terms, stationary time series are “integrated of order zero"

It is obvious that if the price series of a security is stationary, it would be a great candidate for a mean-reversion strategy. Unfortunately, most stock price series are not stationary—they exhibit a geometric random walk that gets them farther and farther away from their starting

If you find** a pair of stocks such that if you long one and short the other, the market value of the pair is stationary.** If this is the case, then the two individual time series are said to be cointegrated. because a linear combination of them is integrated of order zero.

Usually, two stocks that form a cointegrating pair are from the same industry group. Traders have long been familiar with this so-called pair-trading strategy. They buy the pair portfolio when the spread of the stock prices formed by these pairs is low, and sell/short the pair when the spread is high—in other words, a classic mean-reverting strategy

If a price series (of a stock, a pair of stocks, or, in general, a portfolio of stocks) is stationary, then a mean-reverting strategy is guaranteed to be profitable, as long as the stationarity persists into the future (which is by no means guaranteed).

However, the converse is not true. You don’t necessarily need a stationary price series in order to have a successful mean-reverting strategy. Even a nonstationary price series can have many short-term reversal opportunities that one can exploit, as many traders have discovered.

Factor Models

Assume excess returns \(R\) can be obtained in the formula

\[R = Xb+u\]

There are multiple components in the formula

• Factor returns, \(X\) are the common drivers of stock returns, and are therefore independent of a particular stock.

• Factor exposures, \(b\) are the sensitivities to each of these common drivers.

• Specific return, \(u\), any part of a stock’s return that cannot be explained by these common factor returns is deemed a specific return (i.e., specific to a stock and essentially regarded as just random noise within the APT framework). Each stock’s specific return is assumed to be uncorrelated to another stock’s.

A famous factor model is Three-Factor model, that is excess return of a stock depends linearly on only three factor exposures: beta, market capitalization and book-to-price ratio

These factor exposures are obviously different for each stock and for each time period. (Factor exposures are often normalized such that the average of the factor exposures within a universe of stocks is zero, and the standard deviation is 1.)

What about the factor returns and specific returns? We cannot directly compute the factor returns and specific returns—we have to infer their values by running a multivariate linear regression of the excess returns of stocks against the factor exposures.

Note that each stock represents one data point in this linear regression, and we have to either run a separate linear regression for each time period or, if we want an average value over many time periods, aggregate the values from all these time periods into one training set and run one regression against them all.

But the limitation of factor models are also obvious, one can make a general observation that factor models that are dominated by fundamental and macroeconomic factors have one major drawback—they depend on the fact that investors persist in using the same metric to value companies

For example, even though the value (book-to-price ratio) factor returns are usually positive, there are periods of time when investors prefer growth stocks such as during the Internet bubble in the late 1990s

Therefore, it is not uncommon for factor models to experience steep drawdown during the times when investors’ valuation method shifts, even if only for a short duration. But then, this problem is common to practically any trading model that holds stocks overnight.

Exit Strategy

There isn’t usually much variety in the way exit signals are generated. They are based on one of these

• A fixed holding period
• A target price or profit cap
• The latest entry signals
• A stop price

Fixed Holding Period

For momentum model, the optimal period typically decreases due to the increasing speed of the diffusion of information and the increasing number of traders who catch on to this trading opportunity. Hence a momentum model that has worked well with a holding period equal to a week in the backtest period may work only with a one-day holding period now

For mean-reverting model, there is a more statistically robust way to determine the optimal holding period that does not depend on the limited number of actual trades, that is Ornstein-Uhlenbeck formula, which describes the behavior of a continuous-time stochastic (random) process

The Ornstein-Uhlenbeck process is a stationary Gaussian process that exhibits mean-reverting behavior. In simpler terms, it models a system that tends to return to a central equilibrium or mean value over time, similar to a particle undergoing Brownian motion but with a restoring force pulling it back towards a mean position.

The definition of the Ornstein-Uhlenbeck

Mathematicians tell us that the average value of \(X(t)\) follows an exponential decay to its mean \(\mu\), and the half-life of this exponential decay is equal to \(ln(2)/θ\)

Target Price

If you believe that your security is mean reverting, then you also have a ready-made target price—the mean value of the historical prices of the security, or \(\mu\) in the Ornstein-Uhlenbeck formula.

But target prices are not as easily justified in momentum models as in mean-reverting models.

Latest entry signals

Suppose you are running a trading model, and you entered into a position based on its signal. Some time later, you run this model again. If you find that the sign of this latest signal is opposite to your original position (e.g., the latest signal is “buy” when you have an existing short position), then you have two choices. Either you simply use the latest signal to exit the existing position and become flat or you can exit the existing position and then enter into an opposite position

Seasonal Trading

These strategies recommend that you buy or sell certain securities at a fixed date of every year

However, from my own experience, much of the seasonality in equity markets has weakened or even disappeared in recent years, perhaps due to the wide spread knowledge of this trading opportunity, whereas some seasonal trades in commodity futures are still profitable

The most famous seasonal trade in equities is called the January effect.

One version states that small-cap stocks that had the worst returns in the previous calendar year will have higher returns in January than small-cap stocks that had the best returns. The rationale for this is that investors like to sell their losers in December to benefit from tax losses, which creates additional downward pressure on their prices.

High-Frequency Trading

In general, if a high Sharpe ratio is the goal of your trading strategy (as it should be), then you should be trading at high frequencies, rather than holding stocks overnight.

The reason why these strategies have Sharpe ratio is simple: Based on the “law of large numbers,” the more bets you can place, the smaller the percent deviation from the mean return you will experience. With high-frequency trading, one can potentially place hundreds if not thousands of bets all in one day.

Therefore, provided the strategy is sound and generates positive mean return, you can expect the day-to-day deviation from this return to be minimal. With this high Sharpe ratio, one can increase the leverage to a much higher level than longer-term strategies can

But no strategy can actually guarantee positive mean return, besides, high-frequency trading also requires harder backtesting and more powerful infrastructure

Though successful high-frequency strategies have such numerous merits, it is not easy to backtest such strategies when the average holding period decreases to minutes or even seconds. Just having high-frequency data with last prices is not sufficient—data with bid, ask, and last quotes is needed to find out the profitability of executing on the bid versus the ask

High-speed execution may account for a large part of the actual profits or losses. Professional high-frequency trading firms have been writing their strategies in C instead of other, more user-friendly languages, and locating their servers next to the exchange or a ma- jor Internet backbone to reduce the microsecond delays.

Even though the Sharpe ratio is appealing and the returns astronomical, truly high-frequency trading is not by any means easy for an independent trader to achieve in the beginning. But there is no reason not to work toward this goal gradually as expertise and resources accrue

Can Independent Traders Succeed?

This section primarily outlines some advantages that individual investors have in quantitative trading compared to institutional investors.

capacity

A significant difference between individual and institutional quant trading lies in the size of capital (capacity).

The book states that it is far, far easier to generate a high Sharpe ratio trading a 100,000 account than a 100 million account

Different capital sizes lead to different strategies, and it's generally easier to profit in small-scale quant trading compared to managing large amounts of capital. And there are many simple and profitable strategies that can work at the low capacity end that would be totally unsuitable to hedge funds.

More details about this can be obtained from this question on zhihu

The specific reason for such phenomenon, as explained in the book, starts with liquidity: small capital provides liquidity, while large capital demands liquidity.

Most profitable strategies that have low capacities are acting as market makers: providing short-term liquidity when it is needed and taking quick profits when the liquidity need disappears.

However, you have billions of dollars to manage, you now become the party in need of liquidity, and you have to pay for it. To minimize the cost of this liquidity demand, you necessarily have to hold your positions over long periods of time. When you hold for long periods, your portfolio will be subject to macroeconomic changes (i.e., regime shifts) that can cause great damage to your portfolio.

Another issue with large-scale funds is that they engage in competition with different hedge funds, ultimately resulting in similar holdings and similar returns

The intense competition among hedge funds means the strategies become less profitable. The lowered returns in turn pressure the fund manager to overleverage.

The fundamental market inefficiency that they are trying to exploit may remain the same, and thus their portfolios may still end up holding very similar positions. When market environment changes, a stampede out of similar losing positions can (and did) cause a complete meltdown of the market.

constraints

Another difference between individual and institutional quant trading is the constraints during trading.

For example, as a trader in a quantitative fund, you may be prohibited from trading a long-only strategy, but long-only strategies are often easier to find, simpler, more profitable, and if traded in small sizes, no more risky than market-neutral strategies. Or you may be prohibited from trading futures. You may be required to be not only market neutral but also sector neutral. You may be asked to find a momentum strategy when you know that a mean-reverting strategy would work.And on and on. Many of these constraints are imposed for risk management reasons, but many others may be just whims, quirks, and prejudices of the management.

When your strategy shows initial profits, these managers may impose enormous pressure for you to scale up quickly, and when your strategy starts to lose, they may force you to liquidate the portfolios and abandon the strategy immediately.

As every student of mathematical optimization knows, any constraint imposed on an optimization problem decreases the optimal objective value. Similarly, every institutional constraint imposed on a trading strategy tends to decrease its returns

Summary

This passage firstly introduces two kinds of execution system, automated and semi-automated, and the choice depends on the frequency of your trading. But no matter which one to choose, minimizing transaction cost is very necessary, we should cut down on commissions, minimize market impact cost and try to avoid slippage as much as possible.

In actual trading, risk management is very necessary. One important point is to determine the optimal leverage using the Kelly Criterion.And the mind-set is also very important, do not succumb to either despair or greed, proceed slowly with small position size, and thoroughly test various aspects of the trading business.

It also talks about some special topics or common sense in trading, including mean reversion strategy and momentum startegy, regime switching, stationarity and cointegration, factor model, seasonal trading models, high-frequency trading. These can be also regarded as common sense in quantative trading

Finally, it lists some advantages of individual investors over institutional investors, capacity and constraints. Individual investors can use many more strategies than institutional investors with smaller capacity, and also face fewer constraints.

Once you have automated everything and your equity is growing exponentially, can you just sit back, relax, and enjoy your wealth? Unfortunately, experience tells us that strategies do lose their potency over time as more traders catch on to them. It takes ongoing research to supply you with new strategies. So keep studying and thinking, this will be an endless game.