Creating Custom Event Data: A Bag-of-Tricks

Abstract

Political scientists who employ event data—structured records of “who did what to whom”—would ideally use custom datasets tailored to their questions, but the cost of producing custom data pushes them toward existing general event datasets. We introduce a modular framework for producing custom event data that decomposes the process into five independently customizable steps: event detection, event “attribute” extraction (actors, recipients, dates, and locations), entity linking, entity categorization, and date/location resolution. Researchers can use models we provide and invest effort only in the steps their project requires. To address the two most challenging steps, we develop new models: a small language model for extracting event attributes, trained on synthetic and real data, that generalizes to new event types; and a Wikipedia-based entity linking system that disambiguates and categorizes political entities. We validate the pipeline against held-out expert and human-adjudicated annotations and by evaluating it, without retraining, against the hand-coded Electoral Contention and Violence dataset. Applied to ECAV, the pipeline matches ECAV’s actor category for 62% of events and 45% for recipients. Given ECAV’s own spans, categorization reaches 89%, suggesting that customization e!ort should focus on extraction. The pipeline is open source, runs on consumer hardware, and requires no proprietary APIs.