BEGAN-边界均衡生成对抗网络-代码解读

小鱼儿 2022-08-28 01:56 442阅读 0赞

# 当前论文代码 #

首先注意：

> 不同点： 该论文的`输入是噪音`，鉴别器和生成器都是`哑铃型结构`，  
> 相同点： 输出是一张图片，D都是用真实图像去比对.

#### 已知信息 ####

![bd112a92dd2f479ba2ea9e38a26ea45e.png][]![f2691d159f5146f786c4d9f8acfcb444.png][]可见,是从`main.py`开始训练的.测试的时候,只需要将`is_train`设置为False,就可以指示进行训练。

## 代码解读(关键点:\[G/D/总\]`Loss`,\[G,D\]网络`API`) ##

#### from config import `get_config` ####

超参数设置:在实际中,它应该是一个列表。  
[argarse.ArgumentParser.parse\_known\_args()解析][argarse.ArgumentParser.parse_known_args]

arg_lists:{
        
    Network:{
        input_scale_size:64 ,conv_hidden_num:128, z_num:64,},
    
    	Data:{
        dataset:'CelebA',split:'train',batch_size:16,grayscale:False,num_worker:4},
    
    Training:{
        is_train:True,optimizer:'adam',max_step:500000,lr_update_step:100000,d_lr:0.00008,g_lr:0.00008,
    	lr_lower_boundary:0.00002,beta1:0.5,beta2:0.999,gamma:0.5,lambda_k:0.001,use_gpu:True}
    
    Misc:{
        load_path:'',log_step:50,save_step:5000,num_log_samples:3,log_level:'INFO',log_dir:'logs',data_dir:'data',
    	test_data_path:None，sample_per_image:64，random_seed:123}
    }

setattr(config, 'data_format', data_format)   # config.data_format='NCHW' 即使用GPU

#### from trainer import `Trainer` ####

#### from data\_loader import `get_loader` ####

#### from utils import `prepare_dirs_and_logger, save_config` ####

[1.Python 中logging使用指南][1.Python _logging][2.logger.removeHandler(\*)不写会有什么影响][2.logger.removeHandler][3.python中logging使用方法][3.python_logging]

#### `main.py`中 ####

[numpy.random.RandomState函数用法][numpy.random.RandomState]  
[Tensorflow中关于随机数生成种子tf.set\_random\_seed()][Tensorflow_tf.set_random_seed]

rng = np.random.RandomState(config.random_seed)  # 设置伪随机数

## 动态解读 ##

#### 环境设置(基于tf\_1环境): ####

在 `trainer.py`中

> conda install tqdm  
> 192行改为`format(self.config.optimizer)`

在`download.py`中

> conda install requests

添加训练图片:  
![9848a46233bb4324bc182bbe8bdf0b1e.png][]

添加源图像到data/img文件夹下,然后批量改名(新加的一个函数,先执行)

import os
    for pic in os.listdir('img/'):
        os.rename('img/'+pic,'img/'+os.path.splitext(pic)[0]+'.jpg')

修改一行:

data_arg.add_argument('--dataset', type=str, default='img')

# 运行 #

### 一次失误,搞成了test ###

原因: 设置成了`config.load_path='img'`  
data\_path = ‘data\\CelebA’

##### `prepare_dirs_and_logger`中 #####

添加:

config.model_name = 'CelebA_img' 
    config.model_dir = 'logs\\CelebA_img'
    config.data_path = 'data\\CelebA'

创建了`logs`,`data`,`logs\\CelebA_img`3个目录

##### `get_loader`中 #####

root = 'data\\CelebA\\splits\\train'

## 正确的一次训练 ##

修改一行:

data_arg.add_argument('--dataset', type=str, default='img')

### `main`调用准备文件夹函数 ###

`prepare_dirs_and_logger(config)`

添加:

config.model_name = 'img_1008_172021'
    config.model_dir = 'logs\\img_1008_172021'
    config.data_path = 'data\\img'

创建了`logs`,`'logs\\img_1008_172021'`2个目录

### 回到`main`中 ###

data_path = 'data\\img'

### `main`调用加载数据函数`data_loader` ###

data_loader = get_loader(
                data_path, config.batch_size, config.input_scale_size,
                config.data_format, config.split)

调用了函数:

`get_loader(root, batch_size, scale_size, data_format, split=None, is_grayscale=False, seed=None)`中  
取出了`data/img/*.jpg`,但其实也可以是`png`格式,只是,data/img中的图片只能是其中的一种格式。

dataset_name='img'

[1. python glob.glob() 函数][1. python glob.glob_][2. glob.glob() 函数][2. glob.glob_]  
使用`PIL.Image`打开文件: [图片读取：Image.open(ImgPath)][Image.open_ImgPath]

paths = glob("{}/*.{}".format(root, ext))
    with Image.open(paths[0]) as img:
        w, h = img.size
        shape = [h, w, 3]

[详解TensorFlow数据读取机制（tf.train.string\_input\_producer）][TensorFlow_tf.train.string_input_producer]

paths = glob("{}/*.{}".format(root, "jpg"))	# 获取data/img下的全部jpg文件
    filename_queue = tf.train.string_input_producer(list(paths), shuffle=False, seed=seed)	# 产生一个文件名队列
    
    reader = tf.WholeFileReader()		# reader从文件名队列中读数据。对应的方法是reader.read
    filename, data = reader.read(filename_queue)
    
    # 使用start_queue_runners之后，才会开始填充队列(后面的)
    threads = tf.train.start_queue_runners(sess=sess)

原图: `268px`X`360px`,设置成shape=`[268,360,3]`  
[1. tf.read\_file和tf.image.decode\_jpeg处理图片][1. tf.read_file_tf.image.decode_jpeg][2. 编码解码处理（ tf.image.decode\_jpeg）][2. _ tf.image.decode_jpeg]

tf_decode = tf.image.decode_jpeg
    
    image = tf_decode(data, channels=3)		# 将图像解码为3通道数字矩阵
    image.set_shape(shape)

下面这段代码,写的是从\[image\]利用4个线程读取16个数据作为一个batch,且队列长度`capacity`为5048,出队后队列里至少还有`min_after_dequeue=5000`个数据。  
queue: `(16, 268, 360, 3)` , resize后为: `shape=(16, 64, 64, 3)`,转置后的: `shape=(16, 3, 64, 64))`  
[1. tf.train.batch和tf.train.shuffle\_batch的理解][1. tf.train.batch_tf.train.shuffle_batch][ 2.tf.train.shuffle\_batch函数解析][2.tf.train.shuffle_batch]。[参数解读：TensorFlow学习－－tf.train.batch与tf.train.shuffle\_batch][TensorFlow_tf.train.batch_tf.train.shuffle_batch]  
[ 3. TensorFlow函数:tf.image.resize\_nearest\_neighbor函数][3. TensorFlow_tf.image.resize_nearest_neighbor]  
[ 4. tensorflow之tf.to\_float][4. tensorflow_tf.to_float]

queue = tf.train.shuffle_batch([image], batch_size=batch_size,num_threads=4, 
    	capacity=capacity,min_after_dequeue=min_after_dequeue, name='synthetic_inputs')	 # 按batch读取shuffle的数据
    	
    queue = tf.image.resize_nearest_neighbor(queue, [scale_size, scale_size])	#  使用最近邻插值调整images为64*64.
    queue = tf.transpose(queue, [0, 3, 1, 2])		# 使用GPU,这里转置了一下
    return tf.to_float(queue)		# 将张量转换为float32类型

### `main`创建`Trainer`类对象`trainer` ###

trainer = Trainer(config, data_loader)

[tf.assign()函数简单解释][tf.assign]

self.g_lr = tf.Variable(config.g_lr, name='g_lr')	#  config.g_lr
    self.d_lr = tf.Variable(config.d_lr, name='d_lr')
    
    // 赋值 并 定义新的节点
    self.g_lr_update = tf.assign(self.g_lr, tf.maximum(self.g_lr * 0.5, config.lr_lower_boundary), name='g_lr_update')	
    self.d_lr_update = tf.assign(self.d_lr, tf.maximum(self.d_lr * 0.5, config.lr_lower_boundary), name='d_lr_update')	
    
    _, height, width, self.channel = get_conv_shape(self.data_loader, self.data_format)	 # height: 64, width:64, channel: 3
    self.repeat_num = int(np.log2(height)) - 2		 #  4
    self.start_step = 0
    
    self.build_model()		# 开始调用对象方法

### `trainer`对象调用 `build_model` ###

先进行图片数值规范化,然后`self.z`

self.x = self.data_loader	#  Tensor("ToFloat:0", shape=(16, 3, 64, 64), dtype=float32)
    x = norm_img(self.x)	# shape=(16, 3, 64, 64),这个x,代表源图像输入
    self.z = tf.random_uniform((tf.shape(x)[0], self.z_num), minval=-1.0, maxval=1.0)	# shape=(16, 64)
    self.k_t = tf.Variable(0., trainable=False, name='k_t')
    
    # 调用生成器网络,z应该是噪音输入
    G, self.G_var = GeneratorCNN(self.z, self.conv_hidden_num, self.channel,self.repeat_num, self.data_format, reuse=False)
    # G ： (16, 3, 64, 64)   self.G_var：1个全连接+9个卷积的weight,bias

调用了`norm_img(self.x)`函数

def norm_img(image, data_format=None):
        image = image/127.5 - 1.	#  调整到-1~1范围内
        if data_format:
            image = to_nhwc(image, data_format)
        return image

#### 生成器G ####

1.  `GeneratorCNN(z, hidden_num, output_num, repeat_num, data_format, reuse)`中

> `z`:shape=(16, 64) `hidden_num`:128 `repeat_num`=4 `data_format`=‘NCHW’

[ 1. np.prod()][1. np.prod]  
全连接层 [ 2.tf.contrib.slim简介][2.tf.contrib.slim]  
[tensorflow 恢复部分参数、加载指定参数][tensorflow]  
在变量域名`G`下:

with tf.variable_scope("G", reuse=reuse) as vs:
    	slim = tf.contrib.slim
    	num_output = int(np.prod([8, 8, hidden_num]))	 # 8192
    	
    	# 先进行一次全连接
    	x = slim.fully_connected(z, num_output, activation_fn=None)			#全连接通道变化64-8192, 生成了16X8192
    	x = reshape(x, 8, 8, hidden_num, data_format)	# (16, 128, 8, 8)
    	
    	# 下面： 前三层，每层2个卷积，一个上采样  后面一层，2个卷积
    	for idx in range(repeat_num):
    	    x = slim.conv2d(x, hidden_num, 3, 1, activation_fn=tf.nn.elu, data_format=data_format)	
    	    #卷积:  			输入,通道数,	kernel_size,strides,	
    	    x = slim.conv2d(x, hidden_num, 3, 1, activation_fn=tf.nn.elu, data_format=data_format)
    	    if idx < repeat_num - 1:
    	        x = upscale(x, 2, data_format)		# 调用下面的函数，将图像填充为2倍尺寸16-32-64，最后(16, 128, 64, 64)
    	
    	# 再进行一次卷积，输出为3通道，卷积核为3X3，步长为1	
    	out = slim.conv2d(x, 3, 3, 1, activation_fn=None, data_format=data_format)	# (16, 3, 64, 64)
    variables = tf.contrib.framework.get_variables(vs)	# 一个全连接,9个卷积的weight,bias全部拿到
    return out, variables

调用reshape函数: `x`:(16,8192),`h,w,c`=8,8,128

def reshape(x, h, w, c, data_format):
        x = tf.reshape(x, [-1, c, h, w])	# Tensor("G/Reshape:0", shape=(16, 128, 8, 8), dtype=float32) 
        return x

调用upscale函数: `x`:shape=(16, 128, 8, 8),`scale`=2

def upscale(x, scale, data_format):
    	shape = x.get_shape().as_list()		#  [16, 128, 8, 8]
    	if data_format == 'NCHW':
            shape =[shape[0], shape[2], shape[3], shape[1]]	# [16, 8, 8, 128]
        _, h, w, _ = get_conv_shape(x, data_format)		# h:8  w: 8
    	
    	x = tf.transpose(x, [0, 2, 3, 1])	# (16, 8, 8, 128)
        x = tf.image.resize_nearest_neighbor(x, (h*scale, w*scale))	# (16, 16, 16, 128)
        x = tf.transpose(x, [0, 3, 1, 2])	# (16, 128, 16, 16)
    
        return X

#### 判别器D ####

1.  在`build_model(self)`中继续调用判别器: 得到了解码器结果d\_out:(32,3,64,64) 中间编码器结果self.D\_z:(32, 64) 以及D的所有变量self.D\_var

d_out, self.D_z, self.D_var = DiscriminatorCNN(tf.concat([G, x], 0), self.channel, self.z_num, self.repeat_num,self.conv_hidden_num, self.data_format)	# x和G形状一样,都是(16, 3, 64, 64),这里是源图像和生成图像拼接

调用了`DiscriminatorCNN(x, input_channel, z_num, repeat_num, hidden_num, data_format)函数`

其中：`x`:shape=(32, 3, 64, 64) `input_channel`:3 `z_num`:64 `repeat_num`:4 `hidden_num`:128

with tf.variable_scope("D") as vs:
            # Encoder---------------将图像编码
            x = slim.conv2d(x, hidden_num, 3, 1, activation_fn=tf.nn.elu, data_format=data_format)	# (32, 128, 64, 64)
    		
            # for循环中共4层，每层2次卷积，通道128-256-384-512   图像尺寸 64-32-16-8 输出形状：(32, 512, 8, 8)
            for idx in range(repeat_num):
                channel_num = hidden_num * (idx + 1)	# 128,下面是通道为128的两层卷积
                x = slim.conv2d(x, channel_num, 3, 1, activation_fn=tf.nn.elu, data_format=data_format)
                x = slim.conv2d(x, channel_num, 3, 1, activation_fn=tf.nn.elu, data_format=data_format)
                if idx < repeat_num - 1:
                    x = slim.conv2d(x, channel_num, 3, 2, activation_fn=tf.nn.elu, data_format=data_format) # (32, 128, 32, 32)
                    #x = tf.contrib.layers.max_pool2d(x, [2, 2], [2, 2], padding='VALID')
    
            x = tf.reshape(x, [-1, np.prod([8, 8, channel_num])])  # 重新折回2层 (32, 32768)
            z = x = slim.fully_connected(x, z_num, activation_fn=None)	# 重新折回初始尺寸 (32, 64)
    
            # Decoder---------------
            num_output = int(np.prod([8, 8, hidden_num]))	# 8192
            x = slim.fully_connected(x, num_output, activation_fn=None)	# (32, 8192)
            x = reshape(x, 8, 8, hidden_num, data_format)	# (32, 128, 8, 8) 调用了reshape函数
            
            for idx in range(repeat_num):
                x = slim.conv2d(x, hidden_num, 3, 1, activation_fn=tf.nn.elu, data_format=data_format)
                x = slim.conv2d(x, hidden_num, 3, 1, activation_fn=tf.nn.elu, data_format=data_format)
                if idx < repeat_num - 1:
                    x = upscale(x, 2, data_format)	# 调用上面的函数，将图像填充为2倍尺寸8-16-32-64，最后(32, 128, 64, 64)
    		# 卷积,将channel: 128->32 输出尺寸: (32, 3, 64, 64)
            out = slim.conv2d(x, input_channel, 3, 1, activation_fn=None, data_format=data_format)
    
        variables = tf.contrib.framework.get_variables(vs)	# 获取Encode中12个卷积,1个全连接,以及Decode中1个全连接,9个卷积的参数
        return out, z, variables	# 输出 解码器结果out:(32,3,64,64) 中间编码器结果z:(32, 64) 以及D的所有变量

上面调用了`reshape(x, 8, 8, hidden_num, data_format)`,原型:

def reshape(x, h, w, c, data_format):
        if data_format == 'NCHW':
            x = tf.reshape(x, [-1, c, h, w])	# 这里直接自动填充batch_size	
        return x

#### 回到build\_model里 ####

[tf.split()函数的用法][tf.split]

AE_G, AE_x = tf.split(d_out, 2)		# 将D的生成结果平分,即将生成图像G和原始图像x的判别结果分开
    
    self.G = denorm_img(G, self.data_format)  # （函数在下面）调整成正常图片形状 (16, 64, 64, 3) 和大小 0~255，这是G
    self.AE_G, self.AE_x = denorm_img(AE_G, self.data_format), denorm_img(AE_x, self.data_format)  # 这是经过D网络的 G 和 X
    
    optimizer = tf.train.AdamOptimizer
    g_optimizer, d_optimizer = optimizer(self.g_lr), optimizer(self.d_lr)

调用了以下函数

[tf.clip\_by\_value()是啥？][tf.clip_by_value]

def nchw_to_nhwc(x):	# x: (16, 3, 64, 64)
        return tf.transpose(x, [0, 2, 3, 1])	# x: (16, 64, 64, 3)
    def to_nhwc(image, data_format):
        if data_format == 'NCHW':
            new_image = nchw_to_nhwc(image)    
        return new_image
    def denorm_img(norm, data_format):
        return tf.clip_by_value(to_nhwc((norm + 1)*127.5, data_format), 0, 255) # 交叉熵方式将像素值转到0~255

###### 损失函数: ######

\{ L D = L ( x ) − k t ⋅ L ( G ( z D ) ) for θ D , 其 中 : L ( x ) = ∣ x − D ( x ) ∣ η , L ( G ( z ) ) = ∣ G ( z ) − D ( G ( z ) ) ∣ η , z D , z G 采 样 自 z , k 0 = 0 , λ k = 0.001 L G = L ( G ( z G ) ) for θ G , 其 中 : L ( G ( z ) ) = ∣ G ( z ) − D ( G ( z ) ) ∣ η , η ∈ \{ 1 , 2 \} is the target norm. k t + 1 = k t + λ k ( γ L ( x ) − L ( G ( z G ) ) ) for each training step t \\begin\{cases\}\\mathcal\{L\}\_\{D\}=\\mathcal\{L\}(x)-k\_\{t\} \\cdot \\mathcal\{L\}\\left(G\\left(z\_\{D\}\\right)\\right) & \\text \{ for \} \\theta\_\{D\} ,其中:L(x)=|x-D(x)|^\{\\eta\}, L(G(z))=|G(z)-D(G(z))|^\{\\eta\},z\_\{D\} ,z\_\{G\}采样自z,k\_0=0,\\lambda\_k = 0.001 \\\\ \\mathcal\{L\}\_\{G\}=\\mathcal\{L\}\\left(G\\left(z\_\{G\}\\right)\\right) & \\text \{ for \} \\theta\_\{G\},其中: L(G(z))=|G(z)-D(G(z))|^\{\\eta\},\\eta \\in\\\{1,2\\\} \\text \{ is the target norm. \} \\\\ k\_\{t+1\}=k\_\{t\}+\\lambda\_\{k\}\\left(\\gamma \\mathcal\{L\}(x)-\\mathcal\{L\}\\left(G\\left(z\_\{G\}\\right)\\right)\\right) & \\text \{ for each training step \} t\\end\{cases\} ⎩⎪⎨⎪⎧LD=L(x)−kt⋅L(G(zD))LG=L(G(zG))kt\+1=kt\+λk(γL(x)−L(G(zG))) for θD,其中:L(x)=∣x−D(x)∣η,L(G(z))=∣G(z)−D(G(z))∣η,zD,zG采样自z,k0=0,λk=0.001 for θG,其中:L(G(z))=∣G(z)−D(G(z))∣η,η∈\{ 1,2\} is the target norm.  for each training step t

self.d_loss_real = tf.reduce_mean(tf.abs(AE_x - x))		# L(x)  作差-取绝对-求平均值
    self.d_loss_fake = tf.reduce_mean(tf.abs(AE_G - G))		# L(G(z))
    
    self.d_loss = self.d_loss_real - self.k_t * self.d_loss_fake	# Loss_D
    self.g_loss = tf.reduce_mean(tf.abs(AE_G - G))	# Loss_G
    
    # 优化器目标
    d_optim = d_optimizer.minimize(self.d_loss, var_list=self.D_var)
    g_optim = g_optimizer.minimize(self.g_loss, global_step=self.step, var_list=self.G_var)

避 免 目 标 : E \[ L ( x ) \] = E \[ L ( G ( z ) ) \] , 设 置 超 参 数 : γ = E \[ L ( G ( z ) ) \] E \[ L ( x ) \] 37 , 总 的 进 度 查 看 : M g l o b a l = L ( x ) + ∣ γ L ( x ) − L ( G ( z G ) ) ∣ , 避免目标: \\mathbb\{E\}\[\\mathcal\{L\}(x)\]=\\mathbb\{E\}\[\\mathcal\{L\}(G(z))\], 设置超参数: \\gamma=\\frac\{\\mathbb\{E\}\[\\mathcal\{L\}(G(z))\]\}\{\\mathbb\{E\}\[\\mathcal\{L\}(x)\]\_\{37\}\}, 总的进度查看: \\mathcal\{M\}\_\{g l o b a l\}=\\mathcal\{L\}(x)+\\left|\\gamma \\mathcal\{L\}(x)-\\mathcal\{L\}\\left(G\\left(z\_\{G\}\\right)\\right)\\right|, 避免目标:E\[L(x)\]=E\[L(G(z))\],设置超参数:γ=E\[L(x)\]37E\[L(G(z))\],总的进度查看:Mglobal=L(x)\+∣γL(x)−L(G(zG))∣,

[tf.control\_dependencies()函数用法][tf.control_dependencies]

# 总体进度的一个表示
    self.balance = self.gamma * self.d_loss_real - self.g_loss	
    self.measure = self.d_loss_real + tf.abs(self.balance)
    
    # 使得self.k_update的计算在d_optim,g_optim之后进行,注意这里有个clip_by_value
    with tf.control_dependencies([d_optim, g_optim]):
        self.k_update = tf.assign(
            self.k_t, tf.clip_by_value(self.k_t + self.lambda_k * self.balance, 0, 1))

#### 回到`Trainer`里 ####

[tensorflow教程6：Supervisor长期训练帮手][tensorflow_6_Supervisor],[轻松配置分布式TensorFlow][TensorFlow]

sv = tf.train.Supervisor(logdir=self.model_dir,		# 每300s用saver在logdir保存一个“model.ckpt-”+global_step的模型
                                    is_chief=True,
                                    saver=self.saver,
                                    summary_op=None,
                                    summary_writer=self.summary_writer,
                                    save_model_secs=300,
                                    global_step=self.step,
                                    ready_for_local_init_op=None)

[Tensorflow中tf.ConfigProto()详解][Tensorflow_tf.ConfigProto],\[TensorFlow 分布式 managed\_session 和 prepare\_or\_wait\_for\_session\](TensorFlow 分布式 managed\_session 和 prepare\_or\_wait\_for\_session)

gpu_options = tf.GPUOptions(allow_growth=True)	# 当使用GPU时候，Tensorflow运行自动慢慢达到最大GPU的内存
    sess_config = tf.ConfigProto(allow_soft_placement=True,gpu_options=gpu_options) # 当运行设备不满足要求时，会自动分配GPU或者CPU。
    
    self.sess = sv.prepare_or_wait_for_session(config=sess_config) # 同步模式: 参数初始化完成，且主节点也准备好了，其他节点才开始干活

### 回到`main`里 ###

save_config(config)		# 调用了下方的函数
    trainer.train()

调用了函数

def save_config(config):	# 传入的config的值在下面
        param_path = os.path.join(config.model_dir, "params.json")	# 'logs\\img_1011_090228\\params.json'
    
        print("[*] MODEL dir: %s" % config.model_dir)	# [*] MODEL dir: logs\img_1011_090228
        print("[*] PARAM path: %s" % param_path)		# [*] PARAM path: logs\img_1011_090228\params.json
    
        with open(param_path, 'w') as fp:
            json.dump(config.__dict__, fp, indent=4, sort_keys=True)

#### 这时候,`config`的值如下: ####

Namespace(batch_size=16, beta1=0.5, beta2=0.999, conv_hidden_num=128, d_lr=8e-05, data_dir='data', data_format='NCHW', data_path='data\\img', dataset='img', g_lr=8e-05, gamma=0.5, grayscale=False, input_scale_size=64, is_train=True, lambda_k=0.001, load_path='', log_dir='logs', log_level='INFO', log_step=50, lr_lower_boundary=2e-05, lr_update_step=100000, max_step=500000, model_dir='logs\\img_1011_090228', model_name='img_1011_090228', num_log_samples=3, num_worker=4, optimizer='adam', random_seed=123, sample_per_image=64, save_step=5000, split='train', test_data_path=None, use_gpu=True, z_num=64)

self的属性如下：

AE_G:Tensor("clip_by_value_1:0", shape=(16, 64, 64, 3), dtype=float32),
    AE_x:Tensor("clip_by_value_2:0", shape=(16, 64, 64, 3), dtype=float32),
    ...太多了抄的手酸,索性在这儿设个断点,每次慢慢看得了.

### `main`对象调用方法`trainer.train`:运行出错 ###

z_fixed = np.random.uniform(-1, 1, size=(self.batch_size, self.z_num))	# (16, 64)
    
    x_fixed = self.get_image_from_loader()		# 这是第353行
    save_image(x_fixed, '{}/x_fixed.png'.format(self.model_dir))

调用了以下方法:

def get_image_from_loader(self):
        x = self.data_loader.eval(session=self.sess)
        if self.data_format == 'NCHW':
            x = x.transpose([0, 2, 3, 1])
        return x

[TensorFlow（九）eval函数][TensorFlow_eval]

超出范围：RandomShuffleQueue `_1_synthetic_inputs/random_shuffle_queue`已关闭且元素不足（请求 16，当前大小为 0）

> tensorflow.python.framework.errors\_impl.OutOfRangeError: 2 root error(s) found.  
> (0) Out of range: RandomShuffleQueue ‘\_1\_synthetic\_inputs/random\_shuffle\_queue’ is closed and has insufficient elements (requested 16, current size 0)  
> \[\[node synthetic\_inputs (defined at D:\\ProgramData\\Anaconda3\\envs\\tf\_1\\lib\\site-packages\\tensorflow\_core\\python\\framework\\ops.py:1748) \]\]  
> \[\[ResizeNearestNeighbor/\_841\]\]  
> (1) Out of range: RandomShuffleQueue ‘\_1\_synthetic\_inputs/random\_shuffle\_queue’ is closed and has insufficient elements (requested 16, current size 0)  
> \[\[node synthetic\_inputs (defined at D:\\ProgramData\\Anaconda3\\envs\\tf\_1\\lib\\site-packages\\tensorflow\_core\\python\\framework\\ops.py:1748) \]\]  
> 0 successful operations.  
> 0 derived errors ignored.

根据王老师的确认,是因为数据没装进去。初步想法是，将`threads = tf.train.start_queue_runners(sess=self.sess)`提前一下。

## 整合:记录 ##

#### 保存模型 ####

self.saver = tf.train.Saver()	#  创建一个tf.train.Saver对象，用来checkpointing
    
    sv = tf.train.Supervisor(logdir=self.model_dir,		# 每300s用saver在logdir保存一个“model.ckpt-”+global_step的模型
                             is_chief=True,
                             saver=self.saver,
                             save_model_secs=300,
                             global_step=self.step)

#### 保存变量 ####

self.summary_op = tf.summary.merge([
                tf.summary.image("G", self.G),
                tf.summary.image("AE_G", self.AE_G),
                tf.summary.image("AE_x", self.AE_x),
    
                tf.summary.scalar("loss/d_loss", self.d_loss),
                tf.summary.scalar("loss/d_loss_real", self.d_loss_real),
                tf.summary.scalar("loss/d_loss_fake", self.d_loss_fake),
                tf.summary.scalar("loss/g_loss", self.g_loss),
                tf.summary.scalar("misc/measure", self.measure),
                tf.summary.scalar("misc/k_t", self.k_t),
                tf.summary.scalar("misc/d_lr", self.d_lr),
                tf.summary.scalar("misc/g_lr", self.g_lr),
                tf.summary.scalar("misc/balance", self.balance),
            ])
    self.summary_writer = tf.summary.FileWriter(self.model_dir)
    
    sv = tf.train.Supervisor(logdir=self.model_dir,
                                    is_chief=True,
                                    saver=self.saver,
                                    summary_op=None,	# 这个会使存储占用翻倍,故不使用
                                    summary_writer=self.summary_writer,	# 这个未查到,应该是用于记录的
                                    save_model_secs=300,
                                    global_step=self.step,
                                    ready_for_local_init_op=None)	# 这个未查到
    # 在训练时执行
    fetch_dict = {
        
                    "k_update": self.k_update,
                    "measure": self.measure,
                }
    if step % self.log_step == 0:
        fetch_dict.update({
        
            "summary": self.summary_op,
            "g_loss": self.g_loss,
            "d_loss": self.d_loss,
            "k_t": self.k_t,
        })
    result = self.sess.run(fetch_dict)

## 运行出错后的代码解读 ##

下面是`trainer.train`的调用引到的:

[1. python之collection-deque][1. python_collection-deque],[2. python tqdm模块的简单使用][2. python tqdm],[3.tensorflow – 何时在Tensorboard中使用writer.flush()][3.tensorflow _ _Tensorboard_writer.flush],

def train(self):
        z_fixed = np.random.uniform(-1, 1, size=(self.batch_size, self.z_num))	
    	x_fixed = self.get_image_from_loader()		# 应该是要得到图片的，没有得到！
        save_image(x_fixed, '{}/x_fixed.png'.format(self.model_dir))	# 得到一张大图,保存路径: logs\\img_1015_161351/x_fixed.png
    
        measure_history = deque([0]*self.lr_update_step, self.lr_update_step)	# 参数([0]*100000,100000),最大长度为100000的双头队列
    
        for step in trange(self.start_step, self.max_step):	# 计算并打印进度条 0~500000
            fetch_dict = {
        					# 定义fetch_dict
                "k_update": self.k_update,
                "measure": self.measure,
            }
            if step % self.log_step == 0:	# self.log_step: 50
                fetch_dict.update({
        			# 每50步保存一下这些
                    "summary": self.summary_op,
                    "g_loss": self.g_loss,
                    "d_loss": self.d_loss,
                    "k_t": self.k_t,
                })
            result = self.sess.run(fetch_dict)	# 运行一次,得到要保存的值
    
            measure = result['measure']		
            measure_history.append(measure)		# 得到总测量值measure,添加到历史记录器-双头队列中
    
            if step % self.log_step == 0:
                self.summary_writer.add_summary(result['summary'], step) # 调用add_summary方法将训练过程以及训练步数保存 
                self.summary_writer.flush()		# 填充缓冲区
    
                g_loss = result['g_loss']
                d_loss = result['d_loss']
                k_t = result['k_t']
    
                print("[{}/{}] Loss_D: {:.6f} Loss_G: {:.6f} measure: {:.4f}, k_t: {:.4f}". \
                      format(step, self.max_step, d_loss, g_loss, measure, k_t))
    
                if step % (self.log_step * 10) == 0:	# 每500次执行一次
                    x_fake = self.generate(z_fixed, self.model_dir, idx=step)	# 调用了下面的函数generate(self)
                    self.autoencode(x_fixed, self.model_dir, idx=step, x_fake=x_fake)	# 调用了下面的函数autoencode(self)
    
                if step % self.lr_update_step == self.lr_update_step - 1:  # 每100000次更新一次d_lr,g_lr
                    self.sess.run([self.g_lr_update, self.d_lr_update])

### save\_image:好像是bz的图拼成一张大图 ###

[1. Image.fromarray的用法（实现array到image的转换）][1. Image.fromarray_array_image],[ 2.Python图像处理PIL各模块详细介绍][2.Python_PIL]

函数调用:`save_image(x_fixed, '{}/x_fixed.png'.format(self.model_dir))`

def save_image(tensor, filename, nrow=8, padding=2,normalize=False, scale_each=False):
        # filename: 'logs\\img_1015_161351/x_fixed.png'
        ndarr = make_grid(tensor, nrow=nrow, padding=padding,
                                normalize=normalize, scale_each=scale_each)
        im = Image.fromarray(ndarr)		# array转换成image
        im.save(filename)				# 保存图像,名字就是filename

继续调用:

### make\_grid:批大小16还要分成网格化的2\*8 ###

def make_grid(tensor, nrow=8, padding=2, normalize=False, scale_each=False):
        """Code based on https://github.com/pytorch/vision/blob/master/torchvision/utils.py
        tensor: 形状为(B,C,H,W)的4维小批次,或者list(所有相同大小的图像列表)
        nrow: 网格中每行显示的图像数,最终网格尺寸: (B / nrow, nrow),默认nrow = 8
        padding: 填充量,默认为2
       	normalize=False,不将图像移至范围 (0, 1)
       	scale_each=False, 如果为“True”，则分别缩放一批图像中的每个图像，而不是在所有图像上缩放 (min, max)。
       	返回值: 网格（Tensor类型）,包含图像网格的张量。
        """
        nmaps = tensor.shape[0]
        xmaps = min(nrow, nmaps)
        ymaps = int(math.ceil(float(nmaps) / xmaps))
        height, width = int(tensor.shape[1] + padding), int(tensor.shape[2] + padding)
        grid = np.zeros([height * ymaps + 1 + padding // 2, width * xmaps + 1 + padding // 2, 3], dtype=np.uint8)
        k = 0
        for y in range(ymaps):
            for x in range(xmaps):
                if k >= nmaps:
                    break
                h, h_width = y * height + 1 + padding // 2, height - padding
                w, w_width = x * width + 1 + padding // 2, width - padding
    
                grid[h:h+h_width, w:w+w_width] = tensor[k]
                k = k + 1
        return grid

### generate(self):检验当前模型的生成结果,输入噪音,输出生成图 ###

`x_fake = self.generate(z_fixed, self.model_dir, idx=step)`,其中:

z_fixed: np.random.uniform(-1, 1, size=(self.batch_size, self.z_num)), 它的shape: (16, 64)
    self.model_dir: 'logs\\img_1015_161351'
    idx:每500次进入

def generate(self, inputs, root_path=None, path=None, idx=None, save=True):
        # root_path: 'logs\\img_1015_161351', idx:每500次进入
        x = self.sess.run(self.G, {
        self.z: inputs})	
        # self.G: Tensor("clip_by_value:0", shape=(16, 64, 64, 3), dtype=float32)
        # self.z: Tensor("random_uniform:0", shape=(16, 64), dtype=float32)
        if path is None and save:
            path = os.path.join(root_path, '{}_G.png'.format(idx))	# logs\\img_1015_161351\\500_G.png
            save_image(x, path)			# 得到一张大图,路径在上面
            print("[*] Samples saved: {}".format(path))
            return x

### autoencode(self): 对`real&fake`图像的D结果保存 ###

`self.autoencode(x_fixed, self.model_dir, idx=step, x_fake=x_fake)`,其中:

x_fixed: 崩了的那句,是一张大图.例如,batchsize=16时,是一张2行8列的大图
    self.model_dir:'logs\\img_1015_161351'
    x_fake: 一张当前模型(训练到step步了),它的生成效果图.shape=(16, 64, 64, 3)

def autoencode(self, inputs, path, idx=None, x_fake=None):
        items = {
        
            'real': inputs,
            'fake': x_fake,
        }
        for key, img in items.items():
            if img is None:
                continue
            if img.shape[3] in [1, 3]:	# 如果通道数(1或3),是在最后一位,那就转置
                img = img.transpose([0, 3, 1, 2])	# 此时,img.shape=(16,3, 64, 64)
    
                x_path = os.path.join(path, '{}_D_{}.png'.format(idx, key))	# 'logs\\img_1015_161351/500_D_fake.png'
                x = self.sess.run(self.AE_x, {
        self.x: img})	# 生成图像经D得到的图像矩阵值
                save_image(x, x_path)		# 保存生成图像经D后得到的一张大图,这个是D对真实&虚假图像的Encode-Decode结果
                print("[*] Samples saved: {}".format(x_path))

[bd112a92dd2f479ba2ea9e38a26ea45e.png]: https://img-blog.csdnimg.cn/bd112a92dd2f479ba2ea9e38a26ea45e.png
[f2691d159f5146f786c4d9f8acfcb444.png]: /images/20220828/8f93f1133b3b41eda5f4acff64a20323.png
[argarse.ArgumentParser.parse_known_args]: https://blog.csdn.net/m0_37041325/article/details/77934623
[1.Python _logging]: https://blog.csdn.net/alxe_made/article/details/80509428
[2.logger.removeHandler]: https://www.zhihu.com/question/270973580/answer/357833099
[3.python_logging]: https://www.cnblogs.com/lixiaoting/p/11131289.html
[numpy.random.RandomState]: https://blog.csdn.net/w372845589/article/details/80347619
[Tensorflow_tf.set_random_seed]: https://blog.csdn.net/qq_31878983/article/details/79495810
[9848a46233bb4324bc182bbe8bdf0b1e.png]: /images/20220828/7395d1e237714572805e6921039711da.png
[1. python glob.glob_]: https://www.cnblogs.com/yibeimingyue/p/14679614.html
[2. glob.glob_]: https://blog.csdn.net/GeorgeAI/article/details/81035422
[Image.open_ImgPath]: https://blog.csdn.net/weixin_43723625/article/details/108158375
[TensorFlow_tf.train.string_input_producer]: https://blog.csdn.net/lty_sky/article/details/104925112
[1. tf.read_file_tf.image.decode_jpeg]: https://blog.csdn.net/aaon22357/article/details/82953796
[2. _ tf.image.decode_jpeg]: https://blog.csdn.net/qq_43225437/article/details/88018408
[1. tf.train.batch_tf.train.shuffle_batch]: https://blog.csdn.net/ying86615791/article/details/73864381
[2.tf.train.shuffle_batch]: https://blog.csdn.net/u013555719/article/details/77679964
[TensorFlow_tf.train.batch_tf.train.shuffle_batch]: https://blog.csdn.net/akadiao/article/details/79645221
[3. TensorFlow_tf.image.resize_nearest_neighbor]: https://blog.csdn.net/Crystal_LYP/article/details/103923101
[4. tensorflow_tf.to_float]: https://www.cnblogs.com/elitphil/p/11823987.html
[tf.assign]: https://blog.csdn.net/a19990412/article/details/82917734
[1. np.prod]: https://blog.csdn.net/tian_jiangnan/article/details/106658416
[2.tf.contrib.slim]: https://blog.csdn.net/MOU_IT/article/details/82717745
[tensorflow]: https://blog.csdn.net/b876144622/article/details/79962727
[tf.split]: https://blog.csdn.net/mls0311/article/details/82052472
[tf.clip_by_value]: https://blog.csdn.net/qq_31900497/article/details/83245817
[tf.control_dependencies]: https://blog.csdn.net/dongdong9223/article/details/83574333
[tensorflow_6_Supervisor]: https://www.jianshu.com/p/7490ebfa3de8
[TensorFlow]: https://zhuanlan.zhihu.com/p/37168868
[Tensorflow_tf.ConfigProto]: https://blog.csdn.net/qq_31261509/article/details/79746114
[TensorFlow_eval]: https://blog.csdn.net/jiaoyangwm/article/details/79248535
[1. python_collection-deque]: https://zhuanlan.zhihu.com/p/32201189
[2. python tqdm]: https://blog.csdn.net/ruguowoshiyu/article/details/79902444
[3.tensorflow _ _Tensorboard_writer.flush]: http://www.voidcn.com/article/p-bonpdrqa-bvk.html
[1. Image.fromarray_array_image]: https://blog.csdn.net/weixin_43593330/article/details/106708364
[2.Python_PIL]: https://blog.csdn.net/zhangziju/article/details/79123275