Python provides a variety of ways to download files from the Internet, which can be downloaded via HTTP using the urllib package or the requests library. This tutorial will explain how to use these libraries to download files from URLs from Python.

requests Library

requests is one of the most popular libraries in Python. It allows sending HTTP/1.1 requests without manually adding query strings to URLs or form encoding of POST data.

requests The library can perform many functions, including:

• Add form data
• Add multi-part file
• Access Python response data

Send a request

First of all, you need to install the library, the method is very simple:

pip install requests

To test whether the installation is successful, you can simply enter the following command in the Python interpreter:

import requests

If the installation is successful, no error will occur.

HTTP requests include:

• GET
• POST
• PUT
• DELETE
• OPTIONS
• HEAD

Send a GET request

Sending a request is very simple, as shown below:

import requests
req = requests.get("https://www.google.com")

The above command will get the Google page and store the information in status_code.

import requests
req = requests.get("https://www.google.com")
req.status_code
200  # 200 表示請(qǐng)求成功

What if you want to know the encoding type of Google web page?

req.encoding
'ISO-8859-1'

Maybe I would like to know what the response is:

req.text

This is just a truncated part of the response content.

<code>'<meta content="Search the world\'s information, including webpages, imag...'</code>

Send POST request

Simply put, POST requests are used to create or update data, especially for form submissions.

Suppose there is a registration form and you need to enter your email address and password. When you click the Register Submit button, the POST request looks like this:

data = {"email": "info@tutsplus.com",
        "password": "12345"}
req = requests.post("http://www.google.com", params=data)

Send PUT request

PUT request is similar to a POST request and is used to update data. For example, the following API shows how to issue a PUT request:

data = {"name": "tutsplus",
        "telephone": "12345"}
r.put("http://www.contact.com", params=data)

Send DELETE request

As the name implies, a DELETE request is used to delete data. Here is an example of a DELETE request:

data = {'name': 'Tutsplus'}
url = "https://www.contact.com/api/"
response = requests.delete(url, params=data)

urllib Package

urllib package collects multiple modules for processing URLs:

• urllib.error Contains the exception raised by urllib.parse to parse URL
• robots.txt File

urllib.request As shown below:

import urllib.request
with urllib.request.urlopen('http://python.org/') as response:
    html = response.read()

If you want to retrieve and store Internet resources, you can do it through the urlretrieve() function:

import urllib.request
filename, headers = urllib.request.urlretrieve('http://python.org/')
html = open(filename)

Use Python to download images

In this example, the requests library and urllib module are used to download this example image.

url = 'https://www.python.org/static/opengraph-icon-200x200.png'
# 使用 urllib 下載
# 導(dǎo)入 urllib 庫(kù)
import urllib.request
# 將網(wǎng)絡(luò)對(duì)象復(fù)制到本地文件
urllib.request.urlretrieve(url, "python.png")
# 使用 requests 下載
# 導(dǎo)入 requests 庫(kù)
import requests
# 以二進(jìn)制格式下載 url 內(nèi)容
r = requests.get(url)
# open 方法打開系統(tǒng)上的文件并寫入內(nèi)容
with open("python1.png", "wb") as code:
    code.write(r.content)

Use Python to download PDF files

In this example, a PDF file about Google Trends is downloaded.

url = 'https://static.googleusercontent.com/media/www.google.com/en//googleblogs/pdfs/google_predicting_the_present.pdf'
# 使用 urllib 下載
# 導(dǎo)入 urllib 包
import urllib.request
# 將網(wǎng)絡(luò)對(duì)象復(fù)制到本地文件
urllib.request.urlretrieve(url, "tutorial.pdf")
# 使用 requests 下載
# 導(dǎo)入 requests 庫(kù)
import requests
# 以二進(jìn)制格式下載文件內(nèi)容
r = requests.get(url)
# open 方法打開系統(tǒng)上的文件并寫入內(nèi)容
with open("tutorial1.pdf", "wb") as code:
    code.write(r.content)

Use Python to download Zip files

In this example, the contents of the GitHub repository are downloaded and the files are stored locally.

pip install requests

Use Python to download videos

In this example, a video lecture will be downloaded.

import requests

Use Python to download CSV files

You can also use the requests and urllib libraries to download CSV files and use the csv module to process the response. Let's use some example CSV address data.

import requests
req = requests.get("https://www.google.com")

Conclusion

This tutorial introduces the most commonly used file download methods and the most common file formats. Although less code is written when using the urllib module, the .netrc module is more recommended due to its simplicity, popularity, and many additional features including: keep-alive and connection pooling, sessions with cookie persistence, browser-style SSL verification, automatic content decoding, authentication, automatic decompression, Unicode response body, HTTP(S) proxy support, multipart file upload, streaming download, connection timeout, chunked request, requests support).

The above is the detailed content of How to Download Files in Python. For more information, please follow other related articles on the PHP Chinese website!