TensorFlow加载模型就报错:ValueError: Received incompatible tensor with shape,怎么回事啊?

用TensorFlow的Object Detection API做目标识别,一直能正常训练、导出模型、加载训练好的模型、在图片里有效搜索识别。
以前训练好的模型现在还都正常(再次导出、加载、识别均正常)。

现在新训练的模型和以前一样的环境:python 3.9,tensorflow==2.8.0。
新训练的也能导出模型,加载训练好的模型就出错,就错在这一句:
detect_fn = tf.saved_model.load(PATH_TO_SAVED_MODEL)
改成detect_fn = tf.saved_model.load(PATH_TO_SAVED_MODEL, tags=None, options=None)还是出错。

报错好像是:恢复检查点有问题。
我故意把checkpoint文件第一行里model_checkpoint_path: "ckpt-11"改成10,导出模型后还是加载不了模型。

怎么会非得要shape (3, 3, 256, 12)和shape (3, 3, 256, 24)兼容呢?
老板找事,实在是没办法,束手无策。


完整错误信息如下:

Traceback (most recent call last):
  File "C:\TensorFlowObjectDetectionAPITest\Bill\inference_main_v2.py", line 73, in <module>
    detect_fn = tf.saved_model.load(PATH_TO_SAVED_MODEL)
  File "C:\ProgramData\Anaconda3\envs\tensorflow2\lib\site-packages\tensorflow\python\saved_model\load.py", line 782, in load
    result = load_partial(export_dir, None, tags, options)["root"]
  File "C:\ProgramData\Anaconda3\envs\tensorflow2\lib\site-packages\tensorflow\python\saved_model\load.py", line 912, in load_partial
    loader = Loader(object_graph_proto, saved_model_proto, export_dir,
  File "C:\ProgramData\Anaconda3\envs\tensorflow2\lib\site-packages\tensorflow\python\saved_model\load.py", line 189, in __init__
    self._restore_checkpoint()
  File "C:\ProgramData\Anaconda3\envs\tensorflow2\lib\site-packages\tensorflow\python\saved_model\load.py", line 507, in _restore_checkpoint
    load_status = saver.restore(variables_path, self._checkpoint_options)
  File "C:\ProgramData\Anaconda3\envs\tensorflow2\lib\site-packages\tensorflow\python\training\tracking\util.py", line 1468, in restore
    base.CheckpointPosition(
  File "C:\ProgramData\Anaconda3\envs\tensorflow2\lib\site-packages\tensorflow\python\training\tracking\base.py", line 295, in restore
    restore_ops = trackable._restore_from_checkpoint_position(self)  # pylint: disable=protected-access
  File "C:\ProgramData\Anaconda3\envs\tensorflow2\lib\site-packages\tensorflow\python\training\tracking\base.py", line 1060, in _restore_from_checkpoint_position
    current_position.checkpoint.restore_saveables(tensor_saveables,
  File "C:\ProgramData\Anaconda3\envs\tensorflow2\lib\site-packages\tensorflow\python\training\tracking\util.py", line 349, in restore_saveables
    new_restore_ops = functional_saver.MultiDeviceSaver(
  File "C:\ProgramData\Anaconda3\envs\tensorflow2\lib\site-packages\tensorflow\python\training\saving\functional_saver.py", line 415, in restore
    restore_ops = restore_fn()
  File "C:\ProgramData\Anaconda3\envs\tensorflow2\lib\site-packages\tensorflow\python\training\saving\functional_saver.py", line 398, in restore_fn
    restore_ops.update(saver.restore(file_prefix, options))
  File "C:\ProgramData\Anaconda3\envs\tensorflow2\lib\site-packages\tensorflow\python\training\saving\functional_saver.py", line 112, in restore
    restore_ops[saveable.name] = saveable.restore(
  File "C:\ProgramData\Anaconda3\envs\tensorflow2\lib\site-packages\tensorflow\python\training\saving\saveable_object_util.py", line 133, in restore
    raise ValueError(
ValueError: Received incompatible tensor with shape (3, 3, 256, 12) when attempting to restore variable with shape (3, 3, 256, 24) and name _model/_box_predictor/variables/2/.ATTRIBUTES/VARIABLE_VALUE.

 

PATH_TO_SAVED_MODEL,变量没定义?

如果不是上一位学者说的情况,就可能是保存模型是出错,导致参数量不一致,你可以再检查下,具体可以参考https://blog.csdn.net/weixin_42416791/article/details/120085728