Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Python Scraping Error
#1
I wanted to get .text of <a>, but it simply doesn't work.

Here is my code:
from bs4 import BeautifulSoup
import requests

generatedLink = "MyLink"

page = requests.get(generatedLink)
contents = page.text
soup = BeautifulSoup(contents, "html.parser")

name = soup.find('a',class_=["yt-simple-endpoint", "style-scope", "ytd-video-renderer"])

print(name)
And it returns "None"
Output:
<a id="video-title" class="yt-simple-endpoint style-scope ytd-video-renderer" aria-label="TURNIR 1 VS 1 U LOLU FINALEE!! od korisnika KaLuu Vrijeme streaminga: prije 3 dana 3 sata i 49 minuta 644 pregleda" href="/watch?v=5N4X4hjkzOw" title="TURNIR 1 VS 1 U LOLU FINALEE!!"> TURNIR 1 VS 1 U LOLU FINALEE!! </a>
Ik that something is rly wrong here, but if you can, help me please. Thank you!
Reply
#2
Look at what soup source return.
If you see <a> tag in browser dos not always mean that you get that in source Requests get.
This is because of JavaScript,what you see in browser inspect is what has been rendered in DOM.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  I am scraping a web page but got an Error Sarmad54 3 1,540 Mar-02-2023, 08:20 PM
Last Post: Sarmad54
  error in code web scraping alexisbrunaux 5 3,922 Aug-19-2020, 02:31 AM
Last Post: alexisbrunaux
  error zomato scraping data syxzetenz 3 3,450 Jun-23-2020, 08:53 PM
Last Post: Gribouillis
  Web scraping error jithin123 0 2,499 Mar-22-2020, 08:13 PM
Last Post: jithin123
  Web Scraping Error : Not getting expected result adminravi 4 2,492 Oct-08-2019, 09:53 AM
Last Post: snippsat
  Scraping data saving to DB error with Cursor cubangt 3 2,880 May-20-2019, 08:30 PM
Last Post: Yoriz
  Error while scraping item price from online store mgtheboss 2 5,307 Jan-12-2018, 06:42 PM
Last Post: snippsat
  Error while scraping links with beautiful soup mgtheboss 4 8,551 Dec-22-2017, 12:41 PM
Last Post: mgtheboss

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020