Inspect the clusters
Search topic names and keywords, then navigate from broad fields to specialist branches. The 3D overview keeps the larger context in view.
Explore topics →Research demo · Scientific abstracts · 2010–2020
Exploring topics
in scientific abstracts.
A bounded prototype using a sample of a larger abstract collection to try out topic clustering, hierarchical navigation and views of change over time. The aim is a small web demo of the analysis.
Research areas · 3D
Topic detail · 2D
Topics over time
Explore temporal patterns
Compare rises, falls and temporary peaks in the sample. Select a topic to inspect its annual history and the size of the change.
View temporal patterns →Compare years
Compare annual paper counts or each topic’s own normalized history. Open representative papers to inspect what sits behind a pattern.
Explore topic histories →Follow the processing steps
See how the source records become a bounded sample, fitted clusters and a compact browser demo.
View the pipeline →This is a demo of the analysis and visualization approach. Labels are provisional, and the timelines describe this sample rather than the volume of research as a whole.
Research questions & methods
What can we learn
from grouping abstracts?
This prototype packages data preparation, document embeddings, hierarchical clustering and interactive plots into a small web demo. It uses 200,000 sampled papers to test an approach that could later be applied to more of the collection.
Retrospective explorationMultiple levels of detailEvidence within reach9,596,200 unique records available11 years of abstracts200,000 papers in the current atlasCurrent analysis: 200k-paper demo
01 / Purpose
Questions to explore.
The main question is whether clusters of abstracts give a useful way to browse subjects and inspect differences between years. This sample-based prototype keeps the analysis and web display manageable while testing an approach that could be used with a larger corpus.
01
How do topics vary across years?
Compare yearly counts within the sample, either directly or scaled to each topic’s own peak. Check whether a pattern comes from one subtopic or several.
02
What changes inside a field?
Open a topic to examine the specialist subjects beneath it, rather than relying on a single trend for a whole discipline.
03
What do earlier papers look like?
Inspect earlier examples within a topic, while remembering that the groups were fitted using papers from the whole period.
04
What supports the apparent pattern?
Inspect representative papers, characteristic terms, and coverage. Check whether a trend reflects coherent research or an imperfect grouping.
02 / Experience
How to use the demo.
Topical explorer
A stable overview of research areas, with progressively finer topics available as the reader zooms or opens a branch.
- Navigate through a visible hierarchy.
- Inspect papers from a selected year.
- Keep the topic hierarchy and map layout stable.
- Open a paper panel for supporting evidence.
Timeline
A view of annual activity for a selected topic and its children, preserving the reader’s topic selection when switching views.
- Switch between paper counts and each topic’s own normalized history.
- Compare a parent topic with its subtopics.
- Select a period to inspect its papers.
- Keep the limits of sample-based trends visible.
Zoom changes the level of explanation. A broad topic opens into more specific groups. The Ward hierarchy provides the navigation, but does not imply that every scientific subject fits neatly into one branch.
The evidence panel shows representative paper titles, publication years, journals and DOI links where available. Abstract bodies are not included in the public export. A paper can span several subjects even though this tree places it in one branch.
The Explorer link keeps the report navigation and opens the plots at full width. The analysis and labels can be inspected, but should not be treated as authoritative topic definitions.
03 / Research approach
What was done.
A sample keeps the experiment manageable. The current fit uses 200,000 papers, with approximately equal numbers from each year. Some small or unusual subjects may still be poorly represented.
- Select a discovery sample.
The current discovery sample contains 200,000 papers, allocated approximately equally across 2010–2020 and sampled uniformly within each year after duplicate checks and strict English-abstract eligibility. Earlier diversity and public-anchor experiments are retained as research history; they are not the current selection method.
- Cluster all years together.
A subject that becomes prominent late in the period can help define a group whose earlier papers are then recovered. This creates a retrospective view of the collection.
- Try assignment on additional papers.
A pilot assigned 2,200 additional papers using nearest-neighbor voting. Full-corpus assignment is deferred. A later run could improve timeline coverage while leaving the existing topic hierarchy and map fixed; weak and ambiguous matches should remain explicit.
- Keep the results inspectable.
Keep paper identities, saved vectors, cluster memberships and build settings so the plots can be reproduced and questionable groups revisited.
Technical outline: embeddings, PCA, Ward, and UMAP
SPECTER2 represents each title and abstract as a normalized 768-dimensional vector. Long abstracts are processed in chunks. Unwhitened PCA reduces these vectors to 128 dimensions; scores are not normalized again. Ward clustering builds the hierarchy in this space. Adaptive navigation uses branch size and dendrogram structure to choose useful topic expansions.
UMAP supplies display coordinates separately from clustering. The 3D surfaces and shaded 2D contours summarize sampled density, rather than exact topic boundaries. Topic spacing changes display positions only. Papers are fetched from static JSON as a reader opens smaller groups, and search covers topic labels and keywords.
Topic names at the first three levels were generated from keywords and representative papers; deeper groups retain keyword labels. All names remain provisional. Corpus-wide assignment would be a separate, approximate classification operation, without refitting this hierarchy.
Could the sample miss small or emerging topics?
Yes. A large sample does not guarantee that every specialty appears, and a small subject may be absorbed into a broader group. The demo does not yet include a systematic missing-topic evaluation. Inspecting examples and coverage is useful, but does not establish completeness.
What runs when browsing the report?
The embeddings and clustering were computed locally beforehand. Browsing loads saved topic summaries and, for smaller branches, paper metadata from static JSON files. It does not run a model or recluster papers.
Processing & algorithms
From local exports to a small web demo.
The larger collection is the source material. The expensive embedding, clustering and layout steps below were fitted on a 200,000-paper sample. The browser uses saved results; it does not load or analyze the full corpus.
1. Prepare a bounded sample
2. Fit the hierarchy and display coordinates separately
What this prototype encapsulates: data preparation, a bounded model fit, and a compact browser export. A later full-corpus assignment run could reuse the fitted embedding/PCA setup and topic definitions to improve timeline coverage. That larger run is deferred; it would not automatically improve or refit the current hierarchy.
04 / Interpretation
What a change can—and cannot—tell us.
- Counts
- How many included papers are assigned to a topic in a year. Counts can grow because the collection itself grows.
- Corpus share
- The fraction of included papers assigned to a topic, using an explicit denominator. Parent-relative shares answer a different question and should be labeled separately.
- Retrospective categories
- The atlas uses information from the full period to define topics. It does not show what a model could have detected using only evidence available at an earlier date.
- Model-defined topics
- Useful groupings of similar papers, supported by examples and reviewed labels. They are not definitive boundaries between scientific fields.
An early assigned paper is not a date of birth. Establishing when a research topic first emerged requires historical evidence beyond a cluster assignment. The demo shows dates within the available sample; it does not establish when an idea first appeared.
The current collection covers 2010–2020. Its yearly counts reflect this export’s coverage and should not be presented as the total output of science. The balanced pilot is useful for model development; its equal yearly sample sizes cannot be used directly to estimate publication growth.
The Temporal patterns view screens rises and falls separately from temporary peaks. Both use adjustments for multiple comparisons to account for the number of patterns examined. These screens identify candidates for closer reading within the sample.
Map position supports navigation. Apparent distance, boundaries, or visual islands need to be checked against the underlying papers. Changes to a topic’s terms and representative examples can help reveal when its meaning shifts over time.
05 / Current evidence
Current results and limitations.
CompletedCorpus preparation
Streaming import, duplicate checks, source-year and journal coverage audits, and strict English preprocessing for the current sample.
Completed200,000-paper atlas
SPECTER2 embeddings, PCA-128 Ward hierarchy, adaptive topic navigation, linked maps, representative papers and a timeline.
DeferredCorpus-wide timelines
A 2,200-paper assignment pilot is complete. The full run is estimated at roughly 35 hours, with 35–70 hours reserved for planning; it has not been started.
The assignment pilot reproduced depth-three Ward labels on about 82% of test papers, rising to about 90% among accepted assignments. These figures measure agreement with the fitted atlas, not scientific correctness. On the additional pilot papers, about 27% remained unresolved.
The saved runtime estimate is a planning extrapolation for approximately 9.3 million remaining eligible papers. Collection coverage, fine-topic assignment errors and sparse early-year evidence still limit temporal interpretation. The current timeline continues to describe the balanced discovery sample.
Review the processing pipeline →
External documentation
Research & design v0.3 · Updated 23 October 2026. Current demo fitted; corpus-wide assignment deferred. Topic labels and scientific interpretation remain provisional.