Why screening many topics needs a correction AI

Interpreting adjusted p-values when related topic branches and multiple temporal patterns are examined together.

Inspecting many histories gives random fluctuations many opportunities to look interesting. A small p-value for a single topic is less persuasive when that topic was selected after searching hundreds of other branches and possible patterns.

Define the comparison family

The temporal screen tests every automatic topic branch containing at least 100 papers. For each branch it evaluates a directional year score and an interior-bump search. In this sample, 1,140 branches produce 2,280 hypotheses in one adjustment family.

The bump calculation first corrects for the windows searched within that topic. Those corrected values and the trend p-values are then adjusted together. Changing the visible depth, minimum-count filter or list length does not recalculate this family or make the remaining results appear more convincing.

Related branches share evidence

A parent topic contains its descendants’ papers, so their counts and tests are dependent. The screen uses Benjamini–Yekutieli adjustment, which allows arbitrary dependence between valid input p-values. The SciPy documentation describes this procedure and its more conservative behavior relative to Benjamini–Hochberg.

False discovery rate refers to the expected fraction of false rejections among the rejected null hypotheses under the model’s conditions. An adjusted p-value of 0.04 does not mean there is a 4% probability that a particular topic’s change is false.

The report additionally requires effect-size and shape criteria. Those make the displayed screen selective, but a formal false-discovery guarantee should not be assumed for every further subset chosen after adjustment.

The statistical model has a narrower scope than the research question

These calculations treat records as exchangeable under a constant topic share, conditional on the frozen memberships and annual sample sizes. Correlated papers, uncertain cluster membership and changing collection coverage are not modeled. The hierarchy was also learned from this same collection.

Passing the screen therefore identifies an exploratory pattern under those assumptions. It does not independently validate a topic, establish growth across the full literature, or identify a cause. The sample timeline note explains the coverage and denominator limits that remain even for a very small adjusted p-value.