Selenium - "Element is not currently visible and may not be manipulated"

[tkj80]

Hi,

I have been working on the following code to scrape data from the website for my research. However, I got the following error message. "errorMessage":"Element is not currently visible and may not be manipulated"

When I try out each line of the code on the Python interpreter, they would work fine. However, when I save the code as scrapeEcoData_weekly.py and run the file, that's when I get the error message.

from selenium import webdriver
from bs4 import BeautifulSoup
from selenium.webdriver.common.by import By
import time

driver = webdriver.PhantomJS(executable_path='C:/Users/AppData/Local/Programs/Python/Python35-32/myScript/Misc/PhantomJS')
driver.get("https://www.dailyfx.com/calendar")

weeklyView = driver.find_element_by_xpath('//*[@class="btn btn-default webinar-hover dfx-has-spinner hidden-xs"]')
weeklyView.click()
time.sleep(10)
pageSource = driver.page_source
bsObj = BeautifulSoup(pageSource, "html.parser")
tblTag = bsObj.findAll("table", {"class":"table dfx-calendar-table tab-pane fade in active "})[0]
tRows = tblTag.findAll("tr")
for index, item in enumerate(tRows):
print("index: ", index, " ", "item: ", item.get_text())

I tried to locate the "Weekly View" button element using the code instead. The codes would run fine. I took a screenshot of the website after weeklyView.click(), the weekly page didn't show up. It was still showing the daily page as if the button was never clicked on.

weeklyView = driver.find_element_by_xpath('//*[contains(., "Weekly View")]

However, if someone could help me with this.
Thank you!