Python Forum
Web Crawler: How to find all divs that starts with... - Printable Version

+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: Web Scraping & Web Development (https://python-forum.io/forum-13.html)
+--- Thread: Web Crawler: How to find all divs that starts with... (/thread-217.html)

Web Crawler: How to find all divs that starts with... - amandacstr - Sep-30-2016

Hello!

I'm using BeautifulSoup to make a web crawler and i would like to know how can i get the list of all divs that starts with a certain name.

Here what i tried:
divs = soup.findAll('div', {'class':'postContainer*'})


RE: Web Crawler: How to find all divs that starts with... - snippsat - Oct-01-2016

(Sep-30-2016, 11:25 PM)amandacstr Wrote: Here what i tried:
divs = soup.findAll('div', {'class':'postContainer*'})
No you can not do it like this.

You can use CSS selector.
soup.select('div[class^="foo"]')
So this will match all div and css class name that start with foo.
<div class="foo_something">
<div class="foo123">
I have a tutorial on this site,
where i give a demo of using CSS selector "Web-Scraping part-1".

RE: Web Crawler: How to find all divs that starts with... - metulburr - Oct-01-2016

you can always use your lambda approach  :P 

divs = soup.findAll('div', {'class':lambda x: x and x.startswith('postContainer')})
or re

divs = soup.findAll('div', {'class':re.compile('postContainer.*')})


RE: Web Crawler: How to find all divs that starts with... - amandacstr - Oct-01-2016

Thanks for helping me guys! Problem fixed.