Author: jz

Oil futures markets are less than perfectly elastic. At the time of the oil future expiration, a bunch of real barrels of oil change hands, so a pretty good model of the market is to just think about all trades as predicting that single big final transaction between “real” economic actors. It’s pretty clear that the price “ought to be” somewhat inelastic, because if I really need to buy oil, someone has to drill more / drain a reserve and sell it to me.

In the case of the stock market, there is no physical cost to being long/short the stock, just risk/capital costs that can be diversified away. Consequently, it’s easier to have the intuition that the price really should be quite elastic around some consensus “fundamental value”, and this intuition is reflected in much of the literature. But I think it’s a fairly bogus assumption, unless the stock is really easy to model there is no strong consensus price around which people can be elastic, and supply and demand are important.

Highly traded speculative assets that nobody knows how to price (e.g. GME, TSLA) are the least efficient products in this flow shock elasticity sense; they ought to mostly move on flows, because there’s a lack of other information to form a consensus price around. The most efficient products are arbitrage spreads.

I recent had an interesting conversation with Zach Mazlish. We were trying to make sense of the following two papers:

HHL: “The rise in passive
investing over the last 20 years has made the demand for individual stocks 11% more inelastic.”

FIM: AQR Capital’s trading costs (which are dominated by the short-term elasticity of market makers) have generally decreased over the last two decades.

These seem to have opposing implications for the change over time in the elasticity of the stock market with respect to a random flow shock.

One way to reconcile this discrepancy is to suppose that the shape of the shock matters. Maybe the market is more willing to provide to short sharp flows, and less willing to provide to slow steady flows. The decline in active management and rise in systematic trend following behavior may have contributed to such a shift. This theory is not very predictive, because it’s practically difficult to measure exogenous long-term impact.

My personal trading hero is Jim Simons, a differential geometer who founded the world-famous hedge fund Renaissance Technologies — more on him in another post. But from a social welfare perspective, I’d have to award the MVP of the financial markets over the last 50 years to Warren Buffet.

Unlike the single-expiration oil futures market we’ve been discussing thus far, equity markets don’t have a definite expiration date, and it’s harder to draw clean lines between long-term suppliers, long-term demanders, and traders. However, no matter how you define these terms, Warren Buffet is a long-term demander — he loves buying great companies at attractive prices and holding them forever.

Buffet’s activity reduces the noise in the price level on a macroscopic scale. In addition, he’s so big that his decisions alone can cause companies to literally become worth more — markets are not just about predicting the future, they’re also about allocating capital, and his investments into good companies during times of crisis helped make them better. Finally, his price sensitivity reduces the pie — when long-term investors are smart about where they get in, there’s less surplus remaining for shorter-term actors. The influence of his approach has extended well beyond Berkshire’s portfolio; the man is truly a giant.

How did Buffet get so big? Aside from time in the game, one major factor is that his signals have a lot of capacity, because they are long-term in nature. 

Moving back to oil: suppose the price of oil is currently at $60, and consider the following two speculation signals that our trading firm might have:

S1: oil will be worth $61 in a minute;

S2: oil will be worth $61 in an hour.

To simplify issues of counterfactual price with vs. without our actions, let’s assume that these signals are known only to us, and are driven by advance knowledge of fundamental news events (e.g. OPEC) that are released publicly and atomically; everybody will see these events and immediately agree that the new price is $61, allowing us to close our position for free.

Which of the two signals is worth more to us? S1 gives us a higher “return per unit time”; the information advantage it provides us is sharper relative to the background noise in the price. But S2 is more valuable for all but the most extreme levels of low risk tolerance and high opportunity cost of capital, because we will be able to get more size off before saturating the signal. This is because the price impact of exogenous flows tends to decay over time.

The markets are not mathematical; there is no proof of the above statement, and in certain situations it won’t hold. But here is an intuitive argument for why we might expect it to be true:

1. Frequently, participants need to trade significantly more than the size posted on the orderbook. Therefore, they split up their orders into many small pieces.
2. As a result of 1), everybody expects “flows” to exhibit serial correlation over time.
3. That implies that during a flow, the price in part reflects the market’s expectations of how long the current flow will continue.
4. Immediately after a flow ends, those expectations ought to decrease, and we should observe reversion in the price.

Those with physics backgrounds may find the implications of this equilibrium interesting — Brownian motion does not behave this way.

Given impact decay, it’s clear that S2 can be sized bigger than S1: we can copy the same trading plan we used to trade S1, wait a few minutes for some impact to decay, then trade more size. The market may wise up eventually and stop providing liquidity depending on how detectable our activity is.

So: slower signals are worth more than faster signals of the same size. What other considerations shape trading firms’ decisions about what timescales to prioritize during signal search?

1. More data is available on shorter timescales, so “dumb exhaustive” ML architectures can be used to search a large space of ideas automatically without running into the noise wall.
2. It’s more fun to make $1000 in a minute than in a month.
3. Slower signals are worth more because you can trade more size in their direction; but once you saturate your risk-taking capacity, the fraction of the value of the signal you are able to capture starts to go down. If you knew that the S&P 500 was going to go up 1% tomorrow, conservatively you’d need to trade billions of dollars before saturating the signal.
4. Very long (>1 year) holding periods can take advantage of the long-term capital gains tax.
5. Signals that are smaller than the bid-ask spread may be difficult to execute, and it’s harder to find greater-than-spread 1 hour signals than 1 day signals.

With the rise of cheap compute and public data analysis tooling over the last few decades, consideration 1) has become more important. Simultaneously, the bid-ask spread has decreased due to the rise of algorithms, so consideration 5) has become less binding. Consequently, we should expect that the marketplace is doing much, much more research into short-term signals than long-term signals, relative to before.

I conjecture that this shift in focus has played a significant role in the macroscopic efficiency decline observed by Cliff Asness.

[NOTE: this post needs some work. In particular, I think my framework for talking about social welfare is very incomplete, and doesn’t touch enough on the positive benefits of the existence of and social belief the market in the first place + which participants contribute most to that state]

Mechanically, how do trading firms implement spread collection strategies?

Back in the 1970s, the answer was to hire ex-football players to yell at each other in a “pit”. Institutional investors delivered buy/sell orders to brokers on the floor of the pit. The brokers would go to the market makers, ask for a bid/offer market, and trade the order against it.

Nowadays, the vast majority of trading is done electronically, and algorithms provide continuous two-way markets for humans and other algorithms to trade against. This is much more efficient: algorithms are cheap to run, easy to tweak, and can react very quickly. Barriers to entry have been lowered. The resulting competition has driven down bid-offer spreads dramatically over the last 30 years.

A first-order algorithm might look like: Have some fair value for oil (let’s say $60/barrel). Always be willing to buy one contract for 0.25 below fair, and sell for 0.25 above fair. If you get filled on the offer, increase your fair value, and vice versa. At the end of the day / week, close out of your positions, and hope that the spreads you earned exceed the “got run over” costs you incurred.

This fully automated strategy, as simple as it sounds, is probably profitable today in the right market conditions. I haven’t looked at the data, but I would guess that it was profitable 20 years ago even on normal days. Profitable or not, here are four ideas for improving it:

1. Sometimes, you can predict ahead of time that a lot of information is coming out about the underlying supply and demand forces driving the oil price (OPEC meetings, oil company earnings reports, etc.) Other times, you can infer that “something is interesting is happening” even without knowing anything about the fundamentals, because you can detect patterns in the market data itself: e.g. if prices are moving a lot today on heavy volume. In these situations, it’s probably better to ask for more edge on your trades.
2. If oil stock prices spike up, maybe you should be wary of selling oil futures for a bit.
3. You look at historical data, and notice that your sales lose money when the weather is unusually cold. You’re not entirely sure why, but the data is highly significant, so you back off your algorithm.
4. You fit a big machine learning model using all the data you can think of to predict the PNL (profit “n” loss) of your trades, and turn your algorithm off when it predicts you’ll lose money.

Once you have something that makes at least a little money on average, you can hire a bunch of smart people to come up with and implement ideas for making it better. Now you’ve got a fully automated trading business. Every idea you implement makes you more money-per-day, as opposed to a one-time gain.

Of all the ideas you might try to pursue to improve your spread collection algorithm, which contribute more or less to overall market efficiency, as measured by reduction of noise in the price level and the amount of trading surplus captured by trading firms?

The noise question is kind of ambiguous — your actions serve to tamp down very short-term swings in the price, but also make it move a bit more slowly when information comes out. Overall, I think the second term is probably slightly dominant here, but neither matter much — your actions are having an almost imperceptible first-order effect on any kind of macroscopic timescale because you’re going back-and-forth so frequently. Real economy oil demanders and producers don’t care about the magnitude of short-term autocorrelations in the timeseries of price returns, they care about the average levels over longer time periods.

As for surplus, running this kind of algorithm in the first place reduces the trading firm value capture, since you are directly competing with other trading firms to reduce their spread collection profits. Any time you are willing to buy or sell against random people that might be end producers / demanders, you are effectively giving them more of the surplus in expectation. Ideas that get you to do more good providing trades will generally cut into other trading firms’ profits, while ideas that get you to do fewer bad providing trades will generally “increase the pie”.

Now recall that thus far in this narrative, I’ve assumed that spread collectors only trade against real producers / demanders. But once the spread collectors (electronic or otherwise) show up and start providing liquidity, a new class of trader will be incentivized to trade: the speculator. Speculators take liquidity as opposed to providing it, but they aren’t producers / demanders, just independent players with opinions about where the price is going to go. Marginal speculators tend to remove noise from the price level, especially longer-term speculators with large opinions and bankrolls. Their signals generally increase the pie, especially signals directly anticipating the activity of demanders / producers. There are probably also “noise traders” who are literally gambling, but I think the term noise trader is somewhat overused, and most people in the market are making money even though their behavior may not appear perfectly rational under a particular lens.¹

As we can see, there are plenty of ways for trading firms to make money through increasing the pie, many of which don’t have a socially material effect on the macroscopic noise in the price level. This is somewhat concerning, because work in this direction doesn’t improve social welfare, so we should be worried that it’s over-incentivized. In particular, it’s not at all obvious that the system should get more efficient over time in the socially relevant sense.

The easiest way for trading firms to decrease the pie — namely, tighten spreads — incentivizes speculators to find ever-shorter horizon signals and increase the pie again², potentially leading to a wasteful arms race that does not reduce macro-scale price noise or aggregate trading firm revenues. This has had significant implications for the frontier of trading firm competition and innovation over the past few decades, and perhaps the years to come. In the next post, I’ll flesh out this timescale issue a bit more carefully. You might want to check out this post in the meantime.

1. Obviously there are exceptions, e.g. GameStop ↩︎
2. Often the speculators and spread collectors are the same firms, maybe the same systems; it doesn’t change the basic argument, although you have to imagine some sort of decomposition into a spread collection vs. liquidity taking basis. ↩︎

There are many oil producers \(P\) and demanders \(D\). They need to decide how much oil to produce and/or consume, and find a fair price for the transaction. They’ve somehow agreed to coordinate their trading around a single “expiration”: on January 21, lots of suppliers will send barrels of oil to lots of demanders in exchange for money. How should they determine the price and allocation?

For the sake of illustrating just how hard this is, let’s start by making some powerful assumptions. Suppose nobody cares about knowing their allocation before January 21, and that nobody’s preferences change over time. Does this make the problem easy to solve?

An egghead mathematician like me might naively suggest some sort of souped-up Vickrey auction. Design the mechanism so that it’s incentive-compatible: it’s impossible to gain an advantage through lying about one’s preferences, so nobody needs to think strategically. Then just get everybody to submit their “true” preferences to some centralized mechanism, which crunches everything together and spits out a price and an allocation.

More formally, we’d want the mechanism to have these properties:

1. Everybody submits a function \(f: Q \to P\) specifying their indifference price \(P\) for buying/selling each possible quantity \(Q\);
2. The mechanism comes up with a single price and an allocation that maximizes the sum of total profits (“trading surplus”) relative to the submitted indifference prices;
3. The amount people get paid beyond their indifference point is linked to their counterfactual contribution to the sum of total trading surplus;
4. No single participant has an incentive to lie about their preferences to gain a better allocation / move the price favorably.

I have no idea if such a mechanism exists. Maybe if you make a bunch of assumptions, you could come up with something, although I could also see there being an impossibility result given possibly different assumptions. At minimum, it’s expensive to even compute the optimal allocation, so some discretization and approximation would have to be made; every participant would have to be on board with those decisions. And once you start accounting for dynamically changing preferences, this quickly becomes a really impractical plan, akin to launching a rocket to the Moon without an on-board computer system for control.

So how do we solve this in practice? The collective genius of humanity came up with the central limit order book (CLOB). Just let everybody place discrete orders to buy/sell oil at specified prices, and encourage a third type of market participant — traders \(T\)– to sit in the middle and be willing to both buy and sell oil futures contracts at any given time. This lets us replace the one-shot allocation computation with a continuous “descent” process, where we let participants slowly express their preferences via making trades at prices they like, and pay traders to help us discover the price at which supply equals demand. Needless to say, trying to formalize this mathematically would be incredibly complex, require tons of unrealistic assumptions, or both. Below, I’ll refer to the world I’m defining here as a “narrative”, to emphasize that I’m working within an implicit set of assumptions, even though I am unable to formalize them into a model and make explicit deductions.

Some observations:

1. It’s hard to say much about social welfare in this narrative. However, we can be pretty sure that most participants, most of the time, will be able to trade roughly to their indifference point. That means the closing price + size distribution will be a locally efficient allocation, in the sense that everybody is pretty happy with the size they have on at that price. If there is a unique price with this local optimality property, this implies that the outcome of the “descent” is not path-dependent.
2. Participants aren’t required to write down their preferences to interact with the CLOB. They don’t need to reason about what they want in all possible worlds — just this one. Depending on their rationality, this may cause problems.
3. The traders in this narrative get paid through spread collection; they are happiest if a buyer comes in right after a seller, giving them a quick roundtrip profit. The spreads need to be wide enough to compensate them for the fact that the price will on average move against them: if the price starts at $60, but ends at $80, the spread collectors will probably end up getting “run over” by the excess buyers at the lower prices. The larger this net imbalance is relative to the overall trading volume, the wider they need to be.
4. The traders are ultimately getting paid out of the “trading surplus”: producers and demanders are doing a bunch of trades they’re really happy about, so they don’t mind giving up a bit of margin to the traders in the middle.
5. The perfect system would immediately converge on the correct closing price and allow producers and consumers to quickly do all their trading for no spread. The costs to imperfection are the trading surplus siphoned off by the traders (rather than encouraging the “real” economic activity of the producers and demanders), plus noise in the price level that causes misallocation of resources / planning difficulties.
6. The EMH implies that the price at all times should reflect its expected value on January 21. Because the price on January 21 is inherently trying to balance net supply and net demand, in equilibrium we should expect some inelasticity in this market, because some of the net supply/demand is based upon the decisions of individual participants that are not perfectly knowable until they have been released as trades. Flows are fundamental to this market price!

How can traders make more money? Within this narrative, there are two basic ways to do it: Compete with other traders for market share of the surplus already being captured by traders, or try to “grow the pie” by finding ways to take surplus from the producers and demanders. In my next post, I’ll talk about some more concrete things traders do along each of these dimensions.

The classic economic argument for why business is good for the world rests on the notion of consumer surplus.

Acme Corp. sells widgets for $10 each. It pays workers $3 per widget, spends $4 on materials, and $2 on advertising, leaving it with $1 of profit per widget.

All of the people involved in this system are happy according to economic theory. The workers are happy with their pay, or they would stop working; same for the material suppliers and the advertisers. The consumers are paying $10 for something, and they wouldn’t buy it if they didn’t want it. So the total value Acme has contributed to the world is equal to its profit, which gets shared with the rest of society via taxes, plus the difficult-to-measure but definitely non-negative value that its activity is providing to the people involved.

Here are some common objections to this model:

1. It’s hard for workers to change jobs, so they might end up stuck working at jobs that make them unhappy.
2. Acme might be causing negative externalities, like polluting the environment.
3. Consumers might buy products that are bad for them, like cigarettes.
4. Welfare isn’t linear over money, both because money isn’t everything and because rich people need money less than the poor.

These are all important depending on the context, and for some industries it’s hard to determine the sign of their overall impact on the world, while others seem (to my eyes) clearly negative. Nonetheless, it appears to me that in aggregate business is clearly good for the world — the profit motive has coordinated (if not necessarily inspired) the development of many important medical and agricultural advances, more prosaic businesses both small and big play important roles within communities, and taxes fund the government which does lots of good things. If you agree with the position held by Industrial Society and Its Future, find a more exciting blog.

However, this doesn’t imply that more business activity is always better for the world, on the margin. Here are some things you might try to earn a living through business:

1. Invent a cure for cancer.
2. Create a fun and addicting iPhone game.
3. Open the first Thai restaurant in a 20,000 person town.
4. Open the seventh pizza restaurant in a 20,000 person town.
5. Join a big law firm and work your way to the top.
6. Work as a barista at a local coffee shop.
7. Develop a viral marketing campaign to sell beer.
8. Develop a robot that automates welding for car manufacturers.
9. Buy a local dentist’s office and cut costs through aggressive negotiation with suppliers.
10. Mine gold.

My personal ordering, going purely on gut feel, is

\(1 > 3 > 8 > 6 > 5 > 7 > 2 > 4 > 9 > 10\)

Your opinion will vary, due to different perception of the cost-benefit analysis but also aesthetic preference. Ultimately, there is no objective social welfare function. That said, here are some heuristics about welfare on the margin that I used to make the ranking above and feel comfortable defending in most situations:

1. Innovative deep tech is by far the highest in magnitude impact thing you can be working on, although the impact is occasionally negative.
2. If you are working at a job, and you’re happy, it’s good for the world. If you’re running a business, and your workers are happy, it’s very good for the world.
3. Providing better service or quality is good for the world.
4. Using resources more efficiently is good for the world.
5. Charging more for providing the same quality of service, or paying less for the same inputs, is not good for the world.
6. Barely-profitable businesses in highly competitive industries are only as good on the margin as the jobs they create, even if the industry as a whole is very beneficial to society. Unique businesses are often much more valuable on the margin.
7. Competition for eyeballs is bad for the world, unless you are really that much better than the others.
8. Automation is great when the machine is much better / cheaper than the human, and is bad if the machine is about as good / expensive as the human.
9. Price improvement matters most when consumers are elastic and/or poor.
10. Most jobs will get filled by someone else if you don’t take them, so choosing a job you aren’t well suited for is generally bad.

What else?

Here are ten ways to make money in the financial markets:

1. Buy and hold index funds for 20 years
2. Buy and hold big stakes in undervalued companies like Warren Buffet
3. Buy a stock you like after doing 20 minutes of internet research
4. Momentum investing
5. Cross-asset arbitrage
6. Market making (i.e. collecting the bid-offer spread)
7. Trading around flows
8. Exploiting loopholes in bond covenants
9. Buy dogecoin right after Elon Musk tweets about it
10. Buy microwave towers and speed race photons from Chicago to New York

On the margin, I think some of these are probably socially valuable on net, and some of these are probably not, and some of these I’m unsure about. The goal of this blog will be to develop a framework for cost-benefit analysis.

The financial market plays many distinct roles in society: among other things, it’s how we allocate capital to businesses, decide which resources to supply, share investment gains with the population, and store our wealth. If the financial market was only trying to fill a single purpose, maybe we could hope for an easy-to-describe efficient equilibrium, but we don’t live in that world.

This problem is very difficult, but progress matters. Between investor returns and fees, hedge funds and trading firms earn annual excess profits measured in the hundreds of billions. If you think what they’re doing is useless, then moatlessly automating and/or re-incentivizing the industry will add lots of value to the world in the form of talent reallocation. If you think some of those profits are earned in proportion to their social value, then getting better at solving the underlying problems might be worth even more.

My next post will focus on a more concrete market — the oil commodity future market — to build some intuition for trading concepts.

I’m Joseph Zurier, trader and resident of Cambridge MA. Nothing on this blog is investment, trading, or legal advice. I am 26 years old with no economics background, so take everything you read here with a grain of salt. These concepts are confusing, but instead of hedging everything I say, just assume that anything not backed by math proof or physical law is subject to revision. I am here to analyze the system, not to critique its participants.