Running States Separately -- Pupa error


#1

Hey all,

My aim is to create separate domains for each state – (i.e. MnActivist.org)

(I am taking the scrapers for each state out of the OS platform, and trying to use them as the top-level for sites that provide similar information for counties, cites, etc in their respected states.)

I am getting few errors: for Colorado, I got this error:

http://dpaste.com/1HFEPNF

And for Georgia, and a few others, I get this type of error:

http://dpaste.com/3BS1TX5

Any insight on how to go about resolve this would be greatly appreciated –

Thank you,

Canin


#2

Please stick to using the API and contributing fixes there if at all possible. We can’t support people running the scrapers for custom sites as we aim to reduce the load on the scraped sites.


#3

We’ll be good - I can pull from the API and the rest is cake – thank you!!!

  • I’m trying to build on-top of this work to make more information accessible to more constituents – and expand the scrapers to include cites, counties and other governmental boards that are not getting scraped as is — just placing the states as the top-level – not even trying to run them often — just insert the structure as a top level container for the cities and counties in the state …

Any other suggestions as to how I can do that?

If not, we will continue building a work-around – I appreciate all this work, and will continue to contribute –

thank you,


#4

Gotcha. Are you looking to use the existing tooling (pupa/etc.) for the foundation for the cities/gov boards? It isn’t really something many groups do, but it was intended to work for jurisdictions at any level: http://docs.opencivicdata.org/en/latest/ . has some docs that might be somewhat useful.

You’d wind up with a single scraper/etc. per jurisdiction if you went that route, I’m not sure if it’d be good for your needs or if you’ll be collecting different data at different levels (if so, I don’t know if this is helpful)

I hope that helps, glad to answer questions if you do go the OpenCivicData/pupa route.