For many states, all legislators and/or all bills for the current session are marked with a new “updatedAt” time every time they are scraped. Has anyone researched why this happens? I’m about to, and don’t want to reinvent too many wheels. I can see that there’s some code in pupa that tries to avoid it.
I suspect much of this is billy-specific (v1 API only) bills. Using the v2 API, I wrote something which flags legislators and/or bills in each state and flags when:
- some have been created in 2018, and,
- more than 90% of those created this year have been updated in the last 5 days
- No legislator groups were flagged.
- The bills in these 6 states were flagged: IA, MT, NV, OK, TN, TX.
Because the list is so small for the v2 API, I’m not going any farther with this.
can confirm v2 infrastructure is much more conservative about setting the updated flag