+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: Web Scraping & Web Development (https://python-forum.io/forum-13.html)
+--- Thread: Follow Up: Web Calendar based Extraction (/thread-24617.html)
Follow Up: Web Calendar based Extraction - AgileAVS - Feb-23-2020
Hey, I am not a programmer but thanks to this forum, I have been learning a lot. Continuing on with my last time question to this link:
https://python-forum.io/Thread-Extracting-Data-from-Calendar
I did learn to use selenium, however I do not understand on now how to extract data from each pages separately. The code that I have written (not by me but mostly by the help of @Snippsat) doesn't seem to work.
Any help on how to extract data from specific web pages after getting it done through a calendar.
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from bs4 import BeautifulSoup
import time
import requests
browser = webdriver.Chrome()
def stock_data(date):
url = 'https://www.sharesansar.com/today-share-price'
browser.get(url)
inputElement = browser.find_element_by_id('fromdate')
inputElement.clear()
inputElement.send_keys(date)
inputElement.send_keys(Keys.RETURN)
#time.sleep(5) #seconds
inputElement=browser.find_element_by_id('btn_todayshareprice_submit')
inputElement.click()
response = requests.get('https://www.sharesansar.com/ajaxtodayshareprice')
return response
def parse_data(response, date):
soup = BeautifulSoup(response.content, 'lxml')
print(f'-----| {date} |-----')
companyname=soup.select('td:nth-child(7)')
for i in companyname:
try:
CP=i.text
print('Closing Price:',CP)
except:
AttributeError
if __name__ == '__main__':
for day in range(17,22):
date = f'2020-02-{day}'
response = stock_data(date)
parse_data(response, date)Output:DevTools listening on ws://127.0.0.1:50193/devtools/browser/1696afe4-6409-4739-a193-e19e18808298
-----| 2020-02-17 |-----
-----| 2020-02-18 |-----
-----| 2020-02-19 |-----
-----| 2020-02-20 |-----
-----| 2020-02-21 |-----