+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: Web Scraping & Web Development (https://python-forum.io/forum-13.html)
+--- Thread: Multiple File Downloader (/thread-5063.html)
Multiple File Downloader - Josh_Python890 - Sep-16-2017
I'm trying to create a program which downloads multiple files from the url below. To download each file you have to click on the link of the game, then click on the 'download' link near the bottom of the game. I'd then have to go back onto the main page and repeat. The main problem I have is that I have no idea how to save the files to a specific direcotry. Below is the small amount I've pieced together from other forums.
from urllib.request import urlopen, urlretrieve, quote
from bs4 import BeautifulSoup
url = 'http://www.chessgames.com/perl/chesscollection?cid=1014492'
u = urlopen(url)
try:
html = u.read().decode('utf-8')
finally:
u.close()
soup = BeautifulSoup(html)
for link in soup.select('br a'):
href = link.get('href')
file_name = link.get_text()
urlretrieve(href, file_name)
RE: Multiple File Downloader - Larz60+ - Sep-16-2017
see https://python-forum.io/Thread-Web-Scraping-part-1
and https://python-forum.io/Thread-Web-scraping-part-2