Job Posting Observatory
Which postings are copies, how often roles are relisted, who discloses pay, and how gendered the wording is.
1. Duplicates across companies
Word 5-gram shingles, MinHash (64 permutations) with LSH banding, candidates verified by exact Jaccard ≥ 0.6. Shingles that appear in more than 5% of postings are dropped first: without that, a shared equal-opportunity paragraph makes unrelated postings look like copies.
2. Repost behaviour
What this does and doesn't show. It measures how often a role (company + title + location) is listed again, including relists under a new job ID that the platform does not link. It does not show that a job is fake: there is no ground truth for ghost jobs, and companies relist for many legitimate reasons (continuous hiring, high turnover, an expired slot).
3. Pay disclosure
"Pay-range law" states are CA, NY, CO and WA (an illustrative set; check current statutes and thresholds before drawing compliance conclusions). Postings without a range there are candidates for review, not proven violations: the dataset does not show employer size, remote scope or posting dates relative to each law.