想请教一下python爬虫中文乱码问题

小白最近在学爬虫,请问各位大哥们我想爬取的这句话“第一回·宴桃园豪杰三结义 斩黄巾英雄首立功”为啥是乱码呢
我用利用正则解析下面这个
第一回·宴桃园豪杰三结义 斩黄巾英雄首立功
代码如下,问题可能有些弱智,但恳请各位不吝赐教

import requests
import re

url = 'https://www.shicimingju.com/book/sanguoyanyi.html'
headers = {"user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.114 Safari/537.36 Edg/91.0.864.59"}
page_text = requests.get(url=url, headers=headers).text

ex = '<a href="/book/sanguoyanyi/1.html">(.*?)</a>'
text_data = re.findall(ex, page_text, re.S)
print(text_data)

这个是乱码情况:['第ä¸\x80å\x9b\x9e·宴æ¡\x83å\x9b\xad豪æ\x9d°ä¸\x89ç»\x93ä¹\x89 æ\x96©é»\x84å·¾è\x8b±é\x9b\x84é¦\x96ç«\x8bå\x8a\x9f']

教你一招,请求数据之前先右键查看网页源代码,看看最上面的charset是啥,一般都是utf-8或者gbk的,要按它给的写程序,你这个网页是utf8编码,所以可以这样写,参考:

import requests
import re
url = 'https://www.shicimingju.com/book/sanguoyanyi.html'
headers = {"user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.114 Safari/537.36 Edg/91.0.864.59"}
res = requests.get(url=url, headers=headers)
res.encoding='utf-8'
page_text = res.text
ex = '<a href="/book/sanguoyanyi/1.html">(.*?)</a>'
text_data = re.findall(ex, page_text, re.S)
print(text_data)

获取
requests.get(url=url, headers=headers)

之后要设置编码
res.encoding = "utf-8"

再取.text

代码如下:(如有帮助,望采纳!谢谢! 点击我这个回答右上方的【采纳】按钮)


import requests
import re
url = 'https://www.shicimingju.com/book/sanguoyanyi.html'
headers = {"user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.114 Safari/537.36 Edg/91.0.864.59"}
res = requests.get(url=url, headers=headers)
res.encoding = "utf-8"
page_text = res.text
ex = '<a href="/book/sanguoyanyi/1.html">(.*?)</a>'
text_data = re.findall(ex, page_text, re.S)
print(text_data)

img

可以用chardet来自动识别编码
chardet.detect(content)

再指定:
res.encoding = chardet.detect(content)