Scaping into a postgres database

Hey All,

I am new to openstates and new to programming altogether so excuse me if this should be an obvious answer. I personally work for a State Senator in Pennsylvania and I am looking for an easy way to create a database of all the bills and votes in PA. I was following the steps for scraping and I was able to get all the data into json files. However, I was wondering how I get it into a postgres database. I read the contributing to scrapers section where it states that “fast mode” will skip this step. What is the “mode” I am looking for to not skip creating a postgres database.

Hi Callen,

Have you considered using the API? If you don’t scrape in fast mode, things will import to a database as described in the docs, but we strongly encourage people not to scrape the state site. (your situation is a bit unique, but it is still advised to use the API over running your own scrapers unless there’s a strong reason not to)

James

1 Like