Feasability of distributing legislator social media page data


#1

Has OpenStates considered including social media links for legislator accounts (Twitter,Facebook,LinkedIn) in their legislator data?

I noticed it was included in a proposed schema for people entities in this post. I understand the sites that are scraped may not include social links, but the data is out there and perhaps a method for obtaining and including such data could be defined.

Is data which is not scrape-able from official state websites (or official state APIs) considered out of scope?


#2

Hi. This might help you. Results vary depending on accuracy of google civic api.


#3

Very cool–and impressive, thank you for sharing! I had considered doing something similar but the civic APIs location-required querying did not look enjoyable to work with to query all of a states legislators.


#4

This is sort of a weird two part answer due to how we’re set up –

RE:Openstates specifically – if the state provides an official source for them, then we scrape them, otherwise they’re out of scope for the scrapers themselves. That said, this year we’ve moved from “just serve the output of the scrapers” to “use the output of the scrapers as a base for openstates-people, and then import that, including any user additions”. We’re hoping interested parties will start adding useful data like social media to openstates-people.

If someone wanted to take something like @dsherman’s code and apply it to the openstates-people data as PR’s, we’d gladly take it, then serve that in the API. Ideally we’d have one more script that checks that the id’s are still valid after being generated (i’ve found google’s api sometimes serves deleted profiles).

We’re at a bit of a transition period – we considered dropping member data completely because it’s so hard to keep accurate with our resources, but the demand is obviously really high. We’re hoping openstates-people takes off as a community project (it’s off to a great start!), especially once we have time to make the tooling around it better, once legislative sessions cool down.


#5

@tims . - Just checked out openstates legislator data for the first time in a while. It looks like a “channels” component was added that matches the channels provided by google’s civic api. Is this correct?

If so nice job!


#6

that’s all @james and the other awesome folks who’ve contributed to https://github.com/openstates/people ; I can’t take any credit. :sunglasses: