Comment on: Why you should never, ever, ever use MongoDB
1 20 Jul 2015 23:10 u/lenwood in v/programmingComment on: Open Source Web Crawlers, any advice?
Python's Requests library is the easiest thing I've ever used for scraping. Usually 2-3 lines of code for Requests and an XPath statement gets me exactly what I need. I've used these for half a dozen or so projects, very easy to work with. They're also both mature (stable) projects.
But its web scale...
I used it on one small project & didn't have any issues. Selfishly I'd rather not use any database that doesn't have native support within SQLAlchemy. My past few projects have used Postgres & SQLite.