from tensorflow.keras.callbacks import ReduceLROnPlateau, ModelCheckpoint
from tensorflow.keras.preprocessing.image import ImageDataGenerator
from sklearn.metrics import confusion_matrix
import matplotlib.pyplot as plt
import tensorflow as tf
from PIL import Image
import numpy as np
import itertools
import os
im_height = 512
im_width = 512
batch_size = 32
epochs = 10
image_path = "./input/chest-xray-pneumonia/chest_xray/"
train_dir = image_path + "train"
validation_dir = image_path + "test"
test_dir = image_path + "valid"
train_image_generator = ImageDataGenerator(
rescale=1. / 255,
shear_range=0.2,
zoom_range=0.2,
horizontal_flip=True)
validation_image_generator = ImageDataGenerator(rescale=1. / 255)
test_image_generator = ImageDataGenerator(rescale=1. / 255)
train_data_gen = train_image_generator.flow_from_directory(directory=train_dir,
batch_size=batch_size,
shuffle=True,
target_size=(im_height, im_width),
class_mode='categorical')
total_train = train_data_gen.n
val_data_gen = validation_image_generator.flow_from_directory(directory=validation_dir,
batch_size=batch_size,
shuffle=False,
target_size=(im_height, im_width),
class_mode='categorical')
total_val = val_data_gen.n
test_data_gen = test_image_generator.flow_from_directory(directory=test_dir,
batch_size=batch_size,
shuffle=False,
target_size=(im_height, im_width),
class_mode='categorical')
total_test = test_data_gen.n
covn_base = tf.keras.applications.DenseNet201(weights='imagenet', include_top = False,input_shape=(im_height,im_width,3))
covn_base.trainable = False
model = tf.keras.Sequential()
model.add(covn_base)
model.add(tf.keras.layers.Flatten())
model.add(tf.keras.layers.Dropout(rate=0.2))
model.add(tf.keras.layers.Dense(2, activation='softmax'))
model.summary()
model.compile(
optimizer=tf.keras.optimizers.Adam(learning_rate=0.0001),
loss = 'categorical_crossentropy',
metrics=['accuracy']
)
reduce_lr = ReduceLROnPlateau(
monitor='val_loss',
factor=0.1,
patience=2,
mode='auto',
verbose=1
)
checkpoint = ModelCheckpoint(
filepath='DenseNet201.ckpt',
monitor='val_accuracy',
save_weights_only=False,
save_best_only=True,
mode='auto',
period=1
)
history = model.fit(x=train_data_gen,
steps_per_epoch=total_train // batch_size,
epochs=epochs,
validation_data=val_data_gen,
validation_steps=total_val // batch_size,
callbacks=[checkpoint, reduce_lr])
train_acc和val_acc在这3个epoch都是在增长的,从网上直接下载的代码精度一个是0.9+,一个0.6+也算是正常的。接下来就需要题主针对自己的数据集找特点进行网络调参,或者数据预处理的优化了。比如数据的预处理优化,或者学习率,优化器的调整。
另外,不知道你的测试集和验证集的比例,一般7:3,以及它们是否随机分配产生(即训练集和数据集是否符合同一分布特点)。这个也会是影响训练集和验证集精度相差较多的原因。
可能出现过拟合了,检查一下数据是不是不具有代表性。
以下5种情况可供参考:
train loss 不断下降,test loss不断下降,说明网络仍在学习;(最好的)
train loss 不断下降,test loss趋于不变,说明网络过拟合;(max pool或者正则化)
train loss 趋于不变,test loss不断下降,说明数据集100%有问题;(检查dataset)
train loss 趋于不变,test loss趋于不变,说明学习遇到瓶颈,需要减小学习率或批量数目;(减少学习率)
train loss 不断上升,test loss不断上升,说明网络结构设计不当,训练超参数设置不当,数据集经过清洗等问题。(最不好的情况)
Tensorflow2.0 医学图像分类(X光胸片肺炎图像诊断) 这是csdn原作者的文章 我看他似乎没出现这个问题 请问如何修改
您好,我是有问必答小助手,您的问题已经有小伙伴解答了,您看下是否解决,可以追评进行沟通哦~
如果有您比较满意的答案 / 帮您提供解决思路的答案,可以点击【采纳】按钮,给回答的小伙伴一些鼓励哦~~
ps:问答VIP仅需29元,即可享受5次/月 有问必答服务,了解详情>>>https://vip.csdn.net/askvip?utm_source=1146287632