Academic Analysis
← Home

Methodology

This system integrates independent public records into one comparable picture of academic scholarship and its compensation. It is descriptive. It is nota performance evaluation, a ranking of anyone’s worth, or a causal analysis. This page states plainly what it measures, how, and — just as importantly — what it does not.

How records are linked

People are matched across independent sources (state salary records, institutional course schedules, federal grant databases, OpenAlex) by name, because there is no shared identifier. Matching is deliberately tuned for precision over recall: a missing link is preferred to a wrong one. High-confidence matches are accepted, uncertain ones are withheld, and specific guards are applied (exact email where available, restricting candidates to faculty, requiring a paid salary record, crediting only an award’s lead principal investigator, and keeping only grants an institution actually leads).

This means some real people and real work go unlinked, and — rarely — a name can still be matched wrongly. If you find an error, please report a correction.

How accurate is the name matching? Where an independent identifier exists we can check it. On the ~960 Georgia Tech grant awards that also carry a PI email (an independent ground truth the name matcher never sees), the name matcher agreed with the email 97.5% of the time and made a call on 97% of them — i.e. roughly 1 in 40 of its confident calls disagreed, and it abstained on the rest. That is a sample estimate from the one place we have ground truth, not a guarantee for every source; treat individual links as strong but fallible.

What we measure — and its limits

  • Salary is funds paid in a fiscal year from the Open Georgia export — not a contract rate. It mixes 9- and 12-month appointments, clinical and administrative pay, summer salary, and overload, which limits direct comparison across very different appointment types.
  • Federal grants are public NIH + NSF awards credited to the single lead PI (co-PIs uncredited; the amount is a multi-year obligation booked once), so totals are a defensible lower bound. Grant scale differs by field convention, so cross-field dollar comparisons are not comparisons of merit.
  • Research output (works, citations, h-index from OpenAlex) is indicative. Citation rates vary enormously between fields, h-index is all-time and favors large, fast-citing fields, and books, patents, datasets, software, and creative or clinical work are under-counted. Author disambiguation can merge common names.
  • Teaching is measured by enrollment and contact hours — how many students and scheduled hours, not teaching quality, preparation, advising, or mentoring. It structurally understates seminars, studios, labs, and dissertation direction.
  • Institutional R&D (NSF HERD) is self-reported expenditure by agency — authoritative at the campus level, but a different measure from per-person awards.

What we do not measure

Much of what makes a scholar valuable is not in any public dataset. Its absence here is a limit of the data, not a judgment that it lacks worth:

  • Institutional and professional service (committees, review, editing, governance)
  • Mentorship and advising, and dissertation / thesis direction
  • Teaching quality and curriculum development
  • Books and monographs, especially in the humanities and social sciences
  • Artistic and creative work; performance and exhibition
  • Clinical care and professional practice
  • Public and community scholarship and outreach

Coverage & selection bias

Because matching favors precision, unmatched people are omitted rather than guessed at — so any “no matched output” group contains people we could not link, not only people who produced nothing. Coverage varies by source (last-name-only historical schedules link far worse than full-name feeds). Read every result with this in mind; the data-quality dashboard shows coverage and the normalization log per source, and each institution’s “About this data” page carries its linkage rate.

Uncertainty & sample size

Figures are estimates from partial data, not exact counts. Medians are shown with their sample size, and cells built on very few people are suppressed rather than shown with false precision. A single number should be read as approximate, and small differences between groups should not be over-interpreted.

Correlation, not causation

Where the site shows one thing rising with another (for example, pay with citations or grants), that is a description of the reward system— what it happens to reward. It does not establish that one causes the other, that any individual’s pay is deserved or undeserved, or that a higher or lower number reflects a person’s value. These are structural patterns, not verdicts on people.

Public records & corrections

Every figure derives from public records and is shown as published — names included, consistent with the openness of the underlying sources. The full dataset is deliberately not offered for bulk download or via an API, so the site remains a place to read the analysis rather than a channel for wholesale redistribution or mass profiling.

If a figure is inaccurate, please report a correction. Because the records are public, individual entries are not removed on request — the data reflects the public record, and requests cannot be identity-verified.

The engineering decisions and data corrections behind these points are recorded in the project’s architectural decision records and adjustments log.