Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Learning WebScraping
#1
I am very new to web scraping. I am learning web Scraping online.
website i am trying to scrape is http://econpy.pythonanywhere.com/ex/001.html
i have written a code that will scrape

from bs4 import BeautifulSoup
from urllib.request import urlopen

url = urlopen("http://econpy.pythonanywhere.com/ex/001.html")

def getTitle():
    global url
    bs0bj = BeautifulSoup(url, "html.parser")
    for i in bs0bj.find_all(title="buyer-name"):
        print(i.get_text())
getTitle()


#def getTitle():
#    global url
#    bs0bj = BeautifulSoup(url, "html.parser")
#    for i in bs0bj.find_all(title="buyer-info"):
#        print(i.get_text())
#getTitle()

def getPrice():
    global url
    bs0bj = BeautifulSoup(url, "html.parser")
    for i in bs0bj.find_all("span", {"class":"item-price"}):
        print(i.get_text())
getPrice()
now there are few questions:(please uncomment the codes)
Q1: when i run this code using buyer info it prints price,how to get the data of next pages also?
Q2: why it doesn't print price , when run individually(just buyername)?
Q3: how to write these data into CSV?
Reply


Messages In This Thread
Learning WebScraping - by Prince_Bhatia - Aug-25-2017, 11:57 AM
RE: Leaning WebScraping - by Larz60+ - Aug-25-2017, 12:00 PM
RE: Leaning WebScraping - by snippsat - Aug-25-2017, 02:30 PM
RE: Leaning WebScraping - by Prince_Bhatia - Aug-28-2017, 06:47 AM
RE: Leaning WebScraping - by metulburr - Aug-28-2017, 11:58 AM
RE: Learning WebScraping - by Prince_Bhatia - Aug-28-2017, 12:22 PM
RE: Learning WebScraping - by metulburr - Aug-28-2017, 12:33 PM
RE: Learning WebScraping - by snippsat - Aug-28-2017, 02:23 PM
RE: Learning WebScraping - by Prince_Bhatia - Aug-29-2017, 07:32 AM
RE: Learning WebScraping - by Prince_Bhatia - Aug-29-2017, 09:04 AM
RE: Learning WebScraping - by snippsat - Aug-29-2017, 09:31 AM
RE: Learning WebScraping - by Prince_Bhatia - Aug-29-2017, 09:48 AM
RE: Learning WebScraping - by snippsat - Aug-29-2017, 10:39 AM
RE: Learning WebScraping - by Prince_Bhatia - Aug-29-2017, 11:03 AM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Webscraping news articles by using selenium cate16 7 3,460 Aug-28-2023, 09:58 AM
Last Post: snippsat
  Webscraping with beautifulsoup cormanstan 3 2,244 Aug-24-2023, 11:57 AM
Last Post: snippsat
  Webscraping returning empty table Buuuwq 0 1,485 Dec-09-2022, 10:41 AM
Last Post: Buuuwq
  WebScraping using Selenium library Korgik 0 1,109 Dec-09-2022, 09:51 AM
Last Post: Korgik
  How to get rid of numerical tokens in output (webscraping issue)? jps2020 0 2,007 Oct-26-2020, 05:37 PM
Last Post: jps2020
  Python Webscraping with a Login Website warriordazza 0 2,693 Jun-07-2020, 07:04 AM
Last Post: warriordazza
  Help with basic webscraping Captain_Snuggle 2 4,053 Nov-07-2019, 08:07 PM
Last Post: kozaizsvemira
  Can't Resolve Webscraping AttributeError Hass 1 2,377 Jan-15-2019, 09:36 PM
Last Post: nilamo
  How to exclude certain links while webscraping basis on keywords Prince_Bhatia 0 3,304 Oct-31-2018, 07:00 AM
Last Post: Prince_Bhatia
  Webscraping homework Ghigo1995 1 2,723 Sep-23-2018, 07:36 PM
Last Post: nilamo

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020