Data and sources - HackJersey/politics GitHub Wiki
A sample of what can be done with this data at FollowTheMoney:
#The winners We looked at a broad swath of possible data sources for this application, noted below, and ultimately decided for many reasons to focus on these few.
- Sunlight Foundation's Influence Explorer API -- Try it here: http://tryit.sunlightfoundation.com/influenceexplorer. Also check out the Python library for interacting with the AP, and the documentation.
- Sunlight's Open States API. The OpenStates API has its own Python library as well.
On a side note, we loved that Sunlight has a fast and easy way to explore their APIs through their website. Try it yourself.
###Terms of Service for the APIs
- We reserve the right to terminate or throttle your access to our APIs if we decide we have to. This happens very, very rarely. The best way to avoid it is not to hammer our servers -- capping your queries at 2 per second is a good rule of thumb. If that's too little, get in touch with us so we can figure something out...
- Our Influence Explorer API exposes data from upstream partners that license their offerings underCC-BY-NC-SA terms. If you use the Influence Explorer API's campaign finance data you must credit the Center for Responsive Politics and/or the National Institute on Money in State Politics (for federal- and state-level data, respectively). If you want to use their data for commercial purposes you'll need to talk to them about a separate licensing agreement (we'd be happy to make an introduction). Earmark data must be attributed to Taxpayers for Common Sense. Also, keep in mind that there are federal and state restrictions on how you use campaign finance data. For instance, you can't use FEC data to solicit political donations. You're responsible for understanding and obeying these laws.
#A close second We'll also consider bringing in data from these secondary sources, if the need arises.
- Sunlight's "Political Party Time" API
- The U.S. Census offers demographic snapshots of legislative districts, if we need it.
- The state Election Law Enforcement Commission (ELEC) has annual lobbying reports in PDF format.
#The also-rans We examined a number of other data sources that we decided not to use for this project for a number of reasons. Often these sources were not available in good machine-readable formats, or the nature of the data itself required more processing and additional reporting than we'd be able to do programmatically in this app.
We thought about:
- Campaign data directly from ELEC, but the format of the data and its relatively dirty nature pushed us toward the Influence Explorer API. It also has searchable database, but that would require too much labor to grab and process what we need on an ongoing basis.
- TV ad buys from reports hosted in PDFs from the FCC, which the Sunlight Foundation and Free Press have used on a joint project looking at political advertising spending.
- The ActBlue API, which is a website that facillitates fundraising for Democratic candidates and organizations.
- New Jersey election results in PDF format.
- The New Jersey Office of Legislative Services
- State property tax data
- The U.S. Bureau of Labor Statistics' new API.
- Payrolls with the salaries of elected officials.
- A plethora of other public data sources.