Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Alabama, Hawaii, Louisiana, New Jersey Scrapers #132

Open
wants to merge 17 commits into
base: master
Choose a base branch
from

Conversation

lrestrepo0001
Copy link
Collaborator

@lrestrepo0001 lrestrepo0001 commented Oct 17, 2020

This closes #94, closes #100, and closes #106, and closes #113

@Acesonnall Acesonnall self-requested a review October 17, 2020 04:30
@Acesonnall Acesonnall changed the title Leo dev Alabama, Hawaii, Louisiana Scrapers Oct 17, 2020
Copy link
Owner

@Acesonnall Acesonnall left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great job with this!

One thing I identified is that the "locationName" is missing from the physicalAddress or mailingAddress of AL, HI, and LA (only has to be in one or the other, but preferred both when possible).

Also, and perhaps this is nothing, but in the LA scraper, I see that "county_name {county_name} is the culprit gets printed out if there isn't a city in the parsed dict. Is that just debugging?

From Paulo about LA: "Make sure they’re referred to as parishes. They’re distinctively NOT counties"

…a location name and paste the parish to the phrase " Parish Registrar of Voters"

Let me know if I need to go further1
Copy link
Owner

@Acesonnall Acesonnall left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One other thing is that the website field is missing from the schema in all the scraper files. I think if those get corrected along with the upper case physicalAddress field values we'll be looking kosher

lib/scrapers/louisiana/louisiana_scraper.py Show resolved Hide resolved
@Acesonnall
Copy link
Owner

There's a merge conflict for this one and it looks like you actually were trying to add the website field, but something got messed up while using git.

pnasciment0
pnasciment0 previously approved these changes Oct 22, 2020
Copy link
Collaborator

@pnasciment0 pnasciment0 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lets go to mars

@Acesonnall Acesonnall self-requested a review October 22, 2020 23:24
@Acesonnall
Copy link
Owner

Acesonnall commented Oct 22, 2020

New jersey has some outliers and AL has one that aren't uploading (all outliers missing street addresses):
image
image
image

Everything else runs clean.

@Acesonnall Acesonnall changed the title Alabama, Hawaii, Louisiana Scrapers Alabama, Hawaii, Louisiana, New Jersey Scrapers Oct 22, 2020
@Acesonnall
Copy link
Owner

One other thing. It looks like the name of "City & County of Honolulu" came up as something unexpected:
image

… and yeeted PO Box from New Jersey physical address(Pretty sure they typed it by hand), fixed up Honolulu in HI
… learning opportunity; once this is all over, I'd want to redo this one using the South Dakota Courthouse website
@lrestrepo0001
Copy link
Collaborator Author

Hey- do I need to do anything else to squash and merge with the main branch?

@Acesonnall
Copy link
Owner

Nah, you're good for now. I haven't gotten around to testing it yet, but I should have that done today

…ized that some don't have periods in between number sequences, and others had hyphens
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants