Django-Scrapy Integration
This project integrates the Django and Scrapy frameworks. For example, a scrapping of blog posts http://blog.wattpanel.com.br/ is done, creating in django a Blog model with title, posted_at date and short_text.
- Install requirements
$ pip install -r requirements.txt
- Migrate DB
$ make django-migrate
- Copy staticfiles
$ make django-static
- Scrapping blog data
$ make scrapy-wattpanel
- Run project
You access the project on browser by localhost url.
$ make django-run
- Tests
- Improve front-end
- Improve spider to scrapping more data