python-创建mysql线程池进程变量报错 TypeError: cannot pickle '_io.BufferedReader' object

 ###### 问题遇到的现象和发生背景
我想将mysql线程池,放入到多进程中使用

 ###### 问题相关代码,请勿粘贴截图

    # 多进程对象共享
    manager = BaseManager()
    # 一定要在start前注册,不然就注册无效
    manager.register('MySQLhelper', MySQLhelper)
    manager.start()
    sq_lhelper = manager.MySQLhelper("192.168.42.128", int(3306), "root", "root", "ip_pool")  # 错误
    # sq_lhelper= MySQLhelper("192.168.42.128", int(3306), "root", "root", "ip_pool")  # 正确
    combinationIp = combination(sq_lhelper)
    combinationIp.ip_pool_new_turn_ip_pool_rubbish(60)

 ###### 运行结果及报错内容

C:\Users\huanmin\Desktop\python学习\代码\venv\Scripts\python.exe C:/Users/huanmin/Desktop/python学习/代码/爬虫/代理ip验证.py
Traceback (most recent call last):
File "C:\Users\huanmin\Desktop\python学习\代码\爬虫\爬虫数据库操作\combination.py", line 51, in ip_pool_new_turn_ip_pool_rubbish
conn, cursor = self.sq_lhelper.create_conn_cursor()
File "", line 2, in create_conn_cursor
File "C:\Users\huanmin\AppData\Local\Programs\Python\Python310\lib\multiprocessing\managers.py", line 833, in _callmethod
raise convert_to_error(kind, result)
multiprocessing.managers.RemoteError:


Unserializable message: Traceback (most recent call last):
File "C:\Users\huanmin\AppData\Local\Programs\Python\Python310\lib\multiprocessing\managers.py", line 308, in serve_client
send(msg)
File "C:\Users\huanmin\AppData\Local\Programs\Python\Python310\lib\multiprocessing\connection.py", line 211, in send
self._send_bytes(_ForkingPickler.dumps(obj))
File "C:\Users\huanmin\AppData\Local\Programs\Python\Python310\lib\multiprocessing\reduction.py", line 51, in dumps
cls(buf, protocol).dump(obj)
TypeError: cannot pickle '_io.BufferedReader' object


During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Users\huanmin\Desktop\python学习\代码\爬虫\代理ip验证.py", line 104, in
combinationIp.ip_pool_new_turn_ip_pool_rubbish(60)
File "C:\Users\huanmin\Desktop\python学习\代码\爬虫\爬虫数据库操作\combination.py", line 76, in ip_pool_new_turn_ip_pool_rubbish
conn.rollback()
UnboundLocalError: local variable 'conn' referenced before assignment

Process finished with exit code 1

 ###### 我的解答思路和尝试过的方法
不使用多进程变量就是好的 ,使用多进程变量就不行

 ###### 我想要达到的结果
mysql线程池在多进程共享

如果真没啥,大神能解决这个问题的话,我只能使用多线程了o(╥﹏╥)o,........ 多线程太慢了

原文件76行,也就是ip_pool_new_turn_ip_pool_rubbish函数内部有问题

我排查发现是MySQLhelper这个类无法被多进程序列化,问题代码如下:

img


这个是官方库里的代码

img

img

但是上代码在单线程环境下是没问题的

读取文件错误,单次读取有没有问题呢?

尝试了,不好解决,用多线程吧。