Sample Selection and Summary Statistics

When you first look at a dataset of thousands of stocks, it’s tempting to include every last one. But a few very tiny or hard-to-trade stocks can make the whole picture look strange. This chapter shows how two simple screens—on share price and on company size—help us see the real patterns in stock returns, and why many odd-looking results vanish once we clean up the sample.

The Big Picture#

We want to understand why some stocks earn higher returns than others over the same period. But the raw data includes stocks that are almost impossible to trade in real life, or whose prices jump around for technical reasons that have nothing to do with the company’s health. If we leave them in, our statistics can get distorted and we might “discover” patterns that don’t exist for the stocks a typical investor can actually buy. Applying a price screen and a size screen is like clearing away the fog so we can see the landscape clearly. It’s not about cherry‑picking; it’s about focusing on the investable universe where economic forces, not market frictions, dominate.

Why We Need to Clean the Data: The Illiquidity Problem#

Imagine a farmer’s market. At a busy stall with lots of buyers and sellers, a single tomato’s price reflects what most people think it’s worth. But at a quiet stall with only one buyer and a seller who rarely shows up, the price can swing wildly just because someone had to sell quickly. In financial markets, illiquidity is that quiet stall: a stock that trades very few shares each day, often with a big gap between the price you can buy at and the price you can sell at. Prices of illiquid stocks are noisy and can give a misleading impression of risk and return.

When researchers study the cross‑section of stock returns, they usually need thousands of stocks over many years. If they include extremely illiquid names, a handful of weird price moves can pull the averages and make it look like some strategy earns 50% a year—when in reality nobody could trade enough shares to capture that return without moving the price against themselves.

So the first step in any serious analysis is to apply a price screen and a size screen. These rules remove stocks that are too cheap or too small, leaving a sample that reflects the opportunities a real‑world investor can actually access. This chapter explores exactly how those screens work, what they do to the statistical shape of the data, and why many famous “anomalies” weaken or disappear once you apply them.

📝 Section Recap: Raw stock data includes many illiquid stocks whose prices are noisy and unrepresentative. Cleaning the sample with price and size screens helps us focus on the investable universe where economic patterns, not trading frictions, dominate.

The Price Screen: Cutting Out the Cents and the Extremes#

A price screen is a rule that keeps only stocks whose share price falls within a certain range. The most common version in studies of stock returns excludes stocks with a price below 1,000.

Why 5—often called “penny stocks” in the US—tend to be extremely illiquid. Their low price often reflects a company in distress, a very small number of shares outstanding, or both. Even a tiny order can push the price up or down by several percent. The bid–ask spread (the difference between what you can buy and sell for) can be enormous as a fraction of the price. Including these stocks would inject a huge amount of microstructure noise—random wiggles in recorded prices that come from the trading process itself, not from genuine news about the company.

Why 1,000 is unusual, but it can happen when a company never splits its shares (think of a famous example like a certain high‑priced conglomerate). Such stocks are often very large in market value, but their high nominal price can cause strange behaviour in statistical calculations. For instance, a return computed from a price of 1,000 is a tiny percentage change, but if the price is reported with a rounding error of a few cents, the measured return can be misleading. More importantly, a single very high‑priced stock can dominate value‑weighted portfolio returns just because of its price, not its economic importance. The $1,000 cap removes these outliers without losing much information, because there are very few of them.

So the price screen is really a liquidity screen in disguise: it throws out stocks where trading costs are huge and price data are unreliable. After applying it, the remaining sample consists of stocks that an institutional investor could actually trade in reasonable size.

📝 Section Recap: The price screen keeps stocks between 1,000, removing penny stocks (illiquid and noisy) and extremely high‑priced outliers that can distort calculations. This leaves a more liquid and reliable set of stocks.

The Size Screen: Only the Big Enough Fish#

Even after removing very cheap stocks, we still have many tiny companies. A stock can have a price of $20 but only a few million dollars in total market value, meaning it trades very little. That’s where the size screen comes in.

The standard size screen restricts the sample to stocks whose market capitalisation (share price times number of shares outstanding) is at or above the 10th percentile of stocks listed on the New York Stock Exchange (NYSE). The NYSE is used as a benchmark because it historically lists larger, more established companies. By anchoring the cutoff to the NYSE, we avoid the trap of letting the smallest stocks on other exchanges (like NASDAQ) pull the threshold down over time.

Why the 10th percentile? The smallest 10% of NYSE stocks are still very small—often micro‑caps with limited trading. But by using the NYSE distribution, we get a stable cutoff that doesn’t drift with the flood of tiny new listings on other markets. Stocks below this cutoff are excluded. The resulting sample contains the vast majority of the market’s total dollar value while dropping the stocks that are hardest to trade.

Think of it as fishing with a net that has a certain mesh size. You let the smallest fish go because they aren’t meaningful for the catch you’re studying, and they would just add noise to your measurements. The size screen ensures we are looking at companies where prices are set by many active buyers and sellers, not by a few sporadic trades.

Together, the price and size screens are often applied jointly. A stock must pass both to be included. This double filter is the standard approach in serious research on stock returns.

📝 Section Recap: The size screen keeps only stocks with market cap at or above the NYSE 10th percentile, removing very small, illiquid companies. Combined with the price screen, it creates a sample of economically meaningful, tradable stocks.

How Screens Change the Look of the Data: Skewness and Kurtosis#

When you apply these screens, the statistical shape of the data changes in predictable ways. Two key properties are skewness (how lopsided the distribution is) and kurtosis (how fat the tails are).

In the full, unscreened universe, the distribution of stock returns is often highly skewed to the right. That means there are many small negative returns but also a few enormous positive returns—sometimes from penny stocks that double or triple in a day. These extreme winners pull the average return upward, even though most stocks do poorly. Kurtosis is also high: the tails are fatter than a normal distribution, again driven by those tiny, volatile names.

After applying the price and size screens, the sample loses most of the extreme micro‑cap stocks. The right tail shrinks, so skewness drops. The distribution becomes more symmetric, closer to a bell shape. Similarly, kurtosis falls because the wild outliers are gone. The returns now look more like what you’d expect from mature, regularly traded companies.

To see this, imagine a simple example. Suppose we have 1,000 stocks. Before screens, the monthly return distribution has a skewness of 2.5 and a kurtosis of 15 (a normal distribution has skewness 0 and kurtosis 3). After keeping only stocks above $5 and above the NYSE 10th percentile, the skewness might drop to 0.8 and kurtosis to 5. The numbers are illustrative, but the pattern is robust: screens tame the wildness.

This matters because many statistical tools assume data are roughly symmetric and don’t have super‑fat tails. By reducing skewness and kurtosis, the screens make regression results and portfolio sorts more reliable. We are less likely to be fooled by a handful of extreme observations.

📝 Section Recap: Price and size screens reduce the skewness and kurtosis of return distributions by removing tiny, volatile stocks that produce extreme outliers. The screened data are more symmetric and have thinner tails, making statistical analysis more trustworthy.

Why Anomalies Fade When You Remove Small and Illiquid Stocks#

One of the most important consequences of these screens is that many well‑known anomalies—patterns in average returns that seem to contradict simple risk‑based models—weaken or even disappear.

Take, for example, the “idiosyncratic volatility puzzle”: the finding that stocks with high firm‑specific risk tend to have lower future returns. In the full sample, this effect looks strong. But when you remove stocks below $5 and the smallest 10%, the puzzle shrinks dramatically or vanishes. Why? Because the original result was driven largely by tiny, illiquid stocks that are expensive to trade and whose high volatility is partly a statistical artefact of noisy prices. Once those stocks are gone, the remaining sample shows little or no reliable relationship.

The same story plays out for many other anomalies: short‑term reversal, momentum, and various accounting‑based predictors often appear much stronger among micro‑caps than among large, liquid stocks. In some cases, the anomaly exists only in the smallest deciles. When a researcher reports a strategy that earns huge returns, a first question should be: “Does this survive price and size screens?” If not, the strategy may not be implementable after trading costs, or it may simply be a quirk of bad data.

This doesn’t mean the anomalies are “fake.” It means they are concentrated in a part of the market where trading frictions, limits to arbitrage, and data problems are most severe. For an investor managing real money, those frictions eat up the paper returns. For an academic, including those stocks can lead to overconfident conclusions about what drives returns. Using screened samples gives a more conservative and realistic picture of the cross‑section.

📝 Section Recap: Many apparent return anomalies are driven by small, illiquid stocks. Price and size screens remove those stocks, causing the anomalies to weaken or disappear. This highlights the importance of using investable samples to avoid mistaking data noise for genuine economic effects.

The Hidden Benefits: Reducing Measurement Error and Microstructure Noise#

Beyond removing extremes, screens also reduce subtle but pervasive measurement error. Stock prices are not perfect. They come from exchanges with bid–ask spreads, infrequent trading, and rounding.

Consider bid–ask bounce. A stock’s recorded price sometimes flips between the bid (what buyers offer) and the ask (what sellers demand). If a trade happens at the bid one minute and at the ask the next, the price appears to jump even though no new information arrived. This creates artificial volatility and negative autocorrelation in returns. Small, illiquid stocks have wider spreads, so the bounce is larger relative to their price. By screening them out, we remove the stocks where this noise is most severe.

Another issue is stale prices. Some tiny stocks don’t trade every day. Their last recorded price might be from yesterday or even last week. When the market moves, these stale prices lag behind, making the stock look less volatile than it truly is and distorting correlations with other stocks. Screens eliminate the worst offenders.

By focusing on liquid stocks, we get prices that are closer to the “true” value at each point in time. This reduces the noise‑to‑signal ratio in our data. When we run regressions or compute averages, the results reflect real economic relationships rather than trading quirks. It’s like measuring a patient’s temperature with a clean, calibrated thermometer instead of a dirty one that sticks.

📝 Section Recap: Price and size screens reduce measurement error from bid–ask bounce and stale prices by removing the stocks where these problems are most acute. Cleaner data lead to more accurate estimates of risk and return relationships.

Removing Extremes: A More Stable Sample#

Finally, it’s worth zooming out to see the screens as a form of trimming—a general statistical practice of removing extreme observations before analysis. In any dataset, a few outliers can dominate sample means, variances, and correlations. In stock returns, the outliers are not just random; they are systematically linked to size and price.

Stocks with extremely low prices often have extreme characteristics: high leverage, negative earnings, tiny market shares. Their returns can be 1,000% or –99%, which swamps the typical 1% monthly move of a normal stock. By setting a 1,000 cap trims the opposite extreme, though those are rarer.

The size screen does something similar for market capitalisation. The very smallest stocks have characteristics—like enormous volatility and high sensitivity to liquidity shocks—that are not representative of the broader market. Removing them gives a sample that is more homogeneous in terms of trading environment, making patterns among the remaining stocks easier to detect and interpret.

In summary, these screens are not arbitrary. They are based on decades of experience showing that the most reliable and replicable findings in asset pricing come from samples of liquid, reasonably sized stocks. They help us separate signal from noise.

📝 Section Recap: Price and size screens act as a systematic trim, removing stocks with extreme characteristics that would otherwise dominate statistical results. This yields a more stable and representative sample for studying the cross‑section of returns.

Summary#

We’ve seen that a few simple filters—a price screen and a size screen—can dramatically change what we learn from stock return data. By focusing on stocks that are actually tradable and whose prices reflect genuine economic news, we avoid being misled by the wild swings of penny stocks and micro‑caps. The screened sample has lower skewness and kurtosis, weaker anomalies, and less measurement error. This isn’t about hiding inconvenient facts; it’s about making sure our conclusions are built on a solid, investable foundation. When you encounter a new study claiming a huge return pattern, your first instinct should be to ask: “Did they screen out the tiny, illiquid names?” The answer often explains the whole story.