生成对抗网络损失变化问题

在训练GAN网络的时候,判别器损失一直减小,生成器损失一直增大是什么原因?

img

import tensorflow as tf
from tensorflow.keras.layers import Input,Conv2D,LeakyReLU,Dense,Reshape,Conv2DTranspose,Flatten,Dropout
import numpy as np
from keras.layers import BatchNormalization, ReLU
from tensorflow.python.keras.layers import Activation
from tensorflow.python.keras.models import Sequential

latent_dim = 100
height = 64
width = 64
channels = 3

generator1 = Sequential()
generator1.add(Dense(512*4*4,input_dim=100))
generator1.add(Reshape((4, 4, 512)))

generator1.add(Conv2DTranspose(256,4,strides=2,padding='same'))
generator1.add(BatchNormalization(momentum=0.9))
generator1.add(Activation('ReLU'))

generator1.add(Conv2DTranspose(128,4,strides=2,padding='same'))
generator1.add(BatchNormalization(momentum=0.9))
generator1.add(Activation('ReLU'))

generator1.add(Conv2DTranspose(64,4,strides=2,padding='same'))
generator1.add(BatchNormalization(momentum=0.9))
generator1.add(Activation('ReLU'))

generator1.add(Conv2DTranspose(3,4,strides=2,padding='same'))

generator1.add(Activation('tanh'))



generator1.summary()
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras.layers import Input,Conv2D,LeakyReLU,Dense,Reshape,Conv2DTranspose,Flatten,Dropout
import numpy as np
from keras.layers import BatchNormalization, ReLU
from tensorflow.python.keras.layers import Activation
from tensorflow.python.keras.models import Sequential

discriminator1 = Sequential()

input_shape = (64,64,3)

discriminator1.add(Conv2D(64,4,strides=2,input_shape=input_shape,padding='same'))
discriminator1.add(BatchNormalization(momentum=0.9))
discriminator1.add(LeakyReLU(alpha=0.2))


discriminator1.add(Conv2D(128,4,strides=2,padding='same'))
discriminator1.add(BatchNormalization(momentum=0.9))
discriminator1.add(LeakyReLU(alpha=0.2))


discriminator1.add(Conv2D(256,4,strides=2,padding='same'))
discriminator1.add(BatchNormalization(momentum=0.9))
discriminator1.add(LeakyReLU(alpha=0.2))


discriminator1.add(Conv2D(512,4,strides=2,padding='same'))


discriminator1.add(Flatten())

discriminator1.add(Dropout(0.4))
discriminator1.add(Dense(1))
discriminator1.add(Activation('sigmoid'))
discriminator1.summary()

discriminator_optimizer = tf.keras.optimizers.RMSprop(learning_rate=0.0004,
                                   clipvalue=1.0,
                                   decay=1e-8)

discriminator1.compile(optimizer=discriminator_optimizer,
                  loss='binary_crossentropy')
from tensorflow import keras

from generator import latent_dim,generator1
from discriminator import discriminator1
import tensorflow as tf
from tensorflow.python.keras.models import Sequential

discriminator1.trainable = False#冻结鉴别器网络

gan1 = Sequential()
gan1.add(generator1)
gan1.add(discriminator1)
gan_optimizer = tf.keras.optimizers.RMSprop(learning_rate=0.0001,
                                   clipvalue=1.0,
                                   decay=1e-8)

gan1.compile(optimizer=gan_optimizer, loss='binary_crossentropy')

import keras
import tensorflow as tf
from tensorflow.keras.layers import Input,Conv2D,LeakyReLU,Dense,Reshape,Conv2DTranspose,Flatten,Dropout
import numpy as np
import os
from keras.preprocessing import image
from generator import height,width,channels
from generator import latent_dim,generator1
from discriminator import discriminator1
from gan import gan1
from data import x_train



x_train = x_train.reshape((x_train.shape[0],)+(height,width,channels)).astype("float32")/255#数据标准化。.astype()转换numpy数组的数据类型


iterations = 10000#迭代10000次
batch_size = 120
save_dir1 = '/root/autodl-tmp/gen_img'#图片保存地址
#save_dir2 = '/root/autodl-tmp/img1/real_img'#图片保存地址

start = 0
i = 0
for step in range(iterations):#开始循环10000次

    #
    random_latent_vectors = np.random.normal(size=(batch_size,latent_dim))#在潜在空间中采样随机点
    generated_images = generator1.predict(random_latent_vectors)#将这些点解码为虚假图像

    #将这些虚假图像与真实图像合在一起
    stop = start + batch_size#20个一轮
    real_images = x_train[start:stop]#提取相对应的真实图像
    combined_images = np.concatenate([generated_images,real_images])#虚假图像与真实图像相融合

    #合并标签,区分真实和虚假的图像
    labels = np.concatenate([np.ones((batch_size,1)),np.zeros((batch_size,1))])
    labels += 0.05 * np.random.random(labels.shape)#向标签中添加随机噪声,这是一个很重要的技巧

    #训练判别器
    d_loss = discriminator1.train_on_batch(combined_images,labels)#discriminator1模型编译的时候只有loss,所以.train_on_batch输出的是discriminator1的损失



    #在潜在空间中采样随机点
    random_latent_vectors = np.random.normal(size=(batch_size,latent_dim))

    #合并标签,全都是”真实图像“(这是在撒谎)
    misleading_targets = np.zeros((batch_size,1))

    #通过gan模型训练生成器,此时判别器是冻结的
    a_loss = gan1.train_on_batch(random_latent_vectors, misleading_targets)


    start += batch_size
    if start>len(x_train)-batch_size:
        start = 0



    #每100步保存绘图
    if step % 100 == 0:
        gan1.save_weights('gan.h5')#保存模型权重
        i += 1
        print('第', i, '00次训练')
        print("discriminator loss", d_loss)
        print("adversarial loss", a_loss)

        img = image.array_to_img(generated_images[0]*255.,scale=False)
        img.save(os.path.join(save_dir1,"generated"+str(step)+'.png'))

【GAN网络在训练的是判别器的loss在不断的减小,生成器的loss在不断增加】

这是因为判别器太强了,无法给G有效的指导,所以需要适当降低D的网络能力:

1 减小D的层数,但是减的太过了也会出现最后无法指导G的情况

2 减小D的学习率
参考链接【关于GAN网络调参的经验】,期望对你有所帮助:https://blog.csdn.net/qq_45368632/article/details/124489978

神经网络对抗可能陷入局部最优

朋友,训练初始,生成器和判别器的损失函数快速波动,但是大致都分别朝着增大或减小的方向。
趋于稳定后,生成器和判别器的损失函数在小的范围内做上下波动,此时模型趋于稳定。
望采纳谢谢啦

朋友,训练初始,生成器和判别器的损失函数快速波动,但是大致都分别朝着增大或减小的方向。
趋于稳定后,生成器和判别器的损失函数在小的范围内做上下波动,此时模型趋于稳定。
望采纳谢谢啦

【GAN网络在训练的是判别器的loss在不断的减小,生成器的loss在不断增加】

这是因为判别器太强了,无法给G有效的指导,所以需要适当降低D的网络能力:

1 减小D的层数,但是减的太过了也会出现最后无法指导G的情况

2 减小D的学习率
参考链接【关于GAN网络调参的经验】,期望对你有所帮助:https://blog.csdn.net/qq_45368632/article/details/124489978

这是因为判别器太强了,无法给G有效的指导,所以需要适当降低D的网络能力:

1 减小D的层数,但是不可减的太过了
2 减小D的学习率

减小D的学习率