Future of openstates?


#1

There’s been a drastic decline in activity and support at OpenStates over the last few months. Few questions are asked; github issues and discourse queries usually go unresolved and unanswered.

Is there some other open source for this data that’s become available?

Are users walking away because of the new API?

Are the project members pretty much giving up, or overwhelmed with other commitments?

I hope this doesn’t seem rude.


#2

Hi Ed,

Thanks for the note and for everything else you do for the community. Important and not at all rude!!!

No, not giving up. A bit frustrated, maybe… We continue using OS and would like to continue.

I can’t speak for others, but for what its worth, I/we felts that:

  1. The attempt to generalize the APIs made them too obscure, and
  2. The lack of real life examples with working code samples, made using the new APIs a frustrating exercise in wasteful trial-and-error.
  3. Therefore, I won’t be surprised if many users are waiting for the documentation and samples to catch up.

Thanks for everything,

Jake


#3

Thanks Jake.

Given the low level of commits, people waiting for change aren’t going to see it unless they push for it.

When I was getting up to speed on V2, I filtered the issues I reported to just real show-stoppers, to both prioritize them and, frankly, to avoid the appearance of being a whiner. In hindsight, that was wrong; it would be great to have a more comprehensive backlog of doc issues, API tweak suggestions, etc. (There are a dozen unresolved API issues.) My memory is such that I can’t recall my other concerns without revisiting them. I should probably read through my blog posts for implicit issues that aren’t already written up.

If you recall specific pain points that need doc, design, or code change, write them up!

The attempt to generalize the APIs made them too obscure

I suspect I agree. Can you be more specific? Are there specific changes you’d recommend?


#4

I’m hesitant to respond here since I’m new to the community and haven’t contributed yet, so please don’t take anything I’m about to say as criticism. The mere fact this exists in any form is an incredible accomplishment!

I think the project could attract more users (and thus maintainers) by making it possible to download the underlying data directly. An API is great for application developers, but anyone looking to do research will probably end up running the scrapers locally. Based on other threads I’ve read, this additional load creates problems with state websites and is generally discouraged.

Downloading directly seems like a much better solution. If supporting regular bulk downloads is too costly, an easy option would be to set up a Requester Pays bucket on S3.


#5

Hey all.

Thanks for the comments, I appreciate people’s feedback & keeping it constructive. I want to address a few points:

  • As for the core team, our involvement is somewhat “seasonal” as job/life demands exist. I took several weeks of time to sprint on launching the new site/etc. in December & January, and have been on a bit of a hiatus since then. Stay tuned for an upcoming announcement on some paid work, that’ll see another sprint, and maybe the chance to address some of these things.

  • I know it is tautological, but issues are open because no volunteers have stepped up to take them. I got an email from someone asking why so many issues were open, and the answer is that we have to find ways to onboard more volunteers. The core team addresses things that are critical as often as we can, but some things (e.g. API tweaks) are probably a good place for a volunteer to dive in. We can do better about onboarding those people, and I’d be glad to help anyone that is interested get engaged.

  • If people need API examples, I’d encourage them to contribute there! That seems like a great place to get involved. API usage is 230% what it was in January though (measured by daily active user) so from my point of view things were going well for people, if there are pain points please submit patches or suggestions!

  • Bulk data downloads are back for people data, and bill data can be prioritized. Researchers are a smaller fraction of our user base and have historically gotten in touch directly which is why we hadn’t prioritized regular updates, but I know we got behind on requests.