Python Forum
Scrapy-cut: Advanced Cookiecutter Scrapy Templating
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Scrapy-cut: Advanced Cookiecutter Scrapy Templating
#1
This post is to inquire any references or resources is... any pointers  to the following project scope whilst using cookie cutter framework.

can check it out at https://github.com/3rawkz/scrapy-cut


Please review the git repository. You might get a sense of what I'm trying to accomplish, I literally just threw up that Reba the past half-hour and no I don't feel it's up to a draft completion my question is the following.

 Using Jinja,is it possible have multiple templates but in some sort of dictionary be able to call just one? For example I know the end official documents for cookiecutter I first learned that while creating the cookiecutter-pypackage, the license content is located in one text file, of which using else if statement creates a dictionary which is defined  in the json file... Is similar process for python template? Or is it creating a text file with the python code and then figuring out how to do code to create the spider?

Secondly, if I refer you to the "demo" spider code snippet...

yield {

           'name': "{{cookiecutter.item_name_selector}}",

           'event': "{{cookiecutter.item_event_selector}}",

           'vidurl': "{{cookiecutter.item_vidurl_selector}}",

           'tags': "{{cookiecutter.item_tags_selector}}",


           }
 

As you can Imagine... (or if you are slightly familiar with cookiecutter would have check out the json file), the variables are defined and in this scrapy project not using items.py for the sake of starting easy... In the given code snip while using cookicutter... Is there some way I can loop to input x desired amount of items and the "Item selector" to add as many as I need?
Reply


Messages In This Thread
Scrapy-cut: Advanced Cookiecutter Scrapy Templating - by scriptso - Feb-02-2017, 07:36 PM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Scrapy does not show all data in iteration loop georgekasa 0 2,033 Jul-31-2021, 09:10 AM
Last Post: georgekasa
Lightbulb Hypertag. New language for HTML templating w/ Django support mwojnars 0 1,806 Apr-06-2021, 12:53 PM
Last Post: mwojnars
  pip install scrapy fail Barak 10 5,693 Aug-31-2020, 07:45 AM
Last Post: Barak
  Scrapy-Splash installation on windows 10 dsuk 0 2,275 Aug-27-2020, 05:45 AM
Last Post: dsuk
  Python Scrapy Date Extraction Issue tr8585 1 3,420 Aug-05-2020, 04:32 AM
Last Post: tr8585
  Python Scrapy tr8585 2 2,425 Aug-04-2020, 04:11 AM
Last Post: tr8585
  Scraping using Scrapy userskipper 0 2,087 Jul-23-2020, 02:20 PM
Last Post: userskipper
  A problem with installing scrapy Truman 17 9,784 May-24-2020, 07:55 PM
Last Post: Truman
  Merging selenium and scrapy Veakyp 0 1,578 Apr-21-2020, 09:58 AM
Last Post: Veakyp
  How to crawl schema markup data type using scrapy? Nuwan16 1 3,134 Mar-31-2020, 03:42 PM
Last Post: stranac

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020