[0/21][564520/564581] Loss: 54.766731 time elapsed 0
[0/21][564530/564581] Loss: 64.998718 time elapsed 0
[0/21][564540/564581] Loss: 50.452267 time elapsed 0
[0/21][564550/564581] Loss: 66.096046 time elapsed 0
[0/21][564560/564581] Loss: 41.591263 time elapsed 0
[0/21][564570/564581] Loss: 60.138813 time elapsed 0
[0/21][564580/564581] Loss: 47.567059 time elapsed 0
Start val
SystemError: deallocated bytearray object has exported buffers
Traceback (most recent call last):
File "/home/fuyining/Ocr/train.py", line 371, in <module>
main()
File "/home/fuyining/Ocr/train.py", line 367, in main
train(opt, train_loader, test_dataset, image, converter, encoder, encoder_optimizer, decoder, decoder_optimizer, criterion, loss_avg)
File "/home/fuyining/Ocr/train.py", line 173, in train
val(opt, image, converter, encoder, decoder, criterion, 1, dataset=test_dataset, teach_forcing=False) # batchsize:1
File "/home/fuyining/Ocr/train.py", line 53, in val
data = val_iter.next()
File "/home/fuyining/miniconda3/envs/torch/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 521, in __next__
data = self._next_data()
File "/home/fuyining/miniconda3/envs/torch/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1203, in _next_data
return self._process_data(data)
File "/home/fuyining/miniconda3/envs/torch/lib/python3.8/site-packages/torch/utils/data/dataloader.py", line 1229, in _process_data
data.reraise()
File "/home/fuyining/miniconda3/envs/torch/lib/python3.8/site-packages/torch/_utils.py", line 425, in reraise
raise self.exc_type(msg)
MemoryError: Caught MemoryError in DataLoader worker process 1.
Original Traceback (most recent call last):
File "/home/fuyining/miniconda3/envs/torch/lib/python3.8/site-packages/torch/utils/data/_utils/worker.py", line 287, in _worker_loop
data = fetcher.fetch(index)
File "/home/fuyining/miniconda3/envs/torch/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py", line 28, in fetch
data.append(next(self.dataset_iter))
File "/home/fuyining/miniconda3/envs/torch/lib/python3.8/site-packages/tfrecord/reader.py", line 219, in example_loader
for record in record_iterator:
File "/home/fuyining/miniconda3/envs/torch/lib/python3.8/site-packages/tfrecord/reader.py", line 94, in tfrecord_iterator
yield from read_records(start_byte, end_byte)
File "/home/fuyining/miniconda3/envs/torch/lib/python3.8/site-packages/tfrecord/reader.py", line 71, in read_records
datum_bytes = datum_bytes.zfill(int(length * 1.5))
MemoryError
Process finished with exit code 1
其中报错的部位代码是使用pytroch的TFrecord包：
length_bytes = bytearray(8)
crc_bytes = bytearray(4)
datum_bytes = bytearray(1024 * 1024)
length, = struct.unpack("<Q", length_bytes)
if length > len(datum_bytes):
datum_bytes = datum_bytes.zfill(int(length * 1.5))
这里内存溢出？请问应该如何处理？
你好，我是有问必答小助手，非常抱歉，本次您提出的有问必答问题，技术专家团超时未为您做出解答
本次提问扣除的有问必答次数，将会以问答VIP体验卡（1次有问必答机会、商城购买实体图书享受95折优惠）的形式为您补发到账户。
因为有问必答VIP体验卡有效期仅有1天，您在需要使用的时候【私信】联系我，我会为您补发。