搭建神经网络的基本步骤

末蓝、 2022-03-10 00:54 739阅读 0赞

# 1 搭建神经网络的步骤 #

搭建神经网络，可分为四步完成：准备工作、前向传播、反向传播、循环迭代

## 1-0 准备工作 ##

导入模块，生成数据集

> import
> 
> 常量定义
> 
> 生成数据集

## 1-1 前向传播 ##

定义输入、输出和参数

> x = ,y\_=
> 
> w1 = , w2 = 
> 
> a = ,y =

## 1-2 反向传播 ##

定义损失函数、反向传播的方法

> loss = 
> 
> train\_step =

## 1-3 循环迭代 ##

生成会话、训练STEPS轮

with tf.Session() as sess:
        init_op = tf.global_variable_initializer()
        sess.run(init_op)
        for i in range(STEPS):
            start = 
            end = 
            sess.run(train_step,feed_dict:)

## 1-4 简单的例子 ##

求式子![y = (x-5)^\{2\}][y _ _x-5_2]取得最小值时，x的值。

#step0 准备工作
    import tensorflow as tf
    import os
    
    os.environ["TF_CPP_MIN_LOG_LEVEL"] = "2"  # 忽略tensorflow警告信息
    
    learning_rate = 0.2
    STEPS = 40
    
    #step1 前向传播
    w = tf.Variable(tf.random_normal([1,1],stddev=1,seed=1)) #3×1
    
    #step2 反向传播
    loss = tf.square(w-8) #直接从loss开始，这里没有数据集和标签
    train_step = tf.train.GradientDescentOptimizer(learning_rate).minimize(loss)
    
    #step3 循环迭代
    with tf.Session() as sess:
        init_op = tf.global_variables_initializer()
        sess.run(init_op)
        for i in range(STEPS):
            sess.run(train_step)
            print("step:%d , loss:%g , w:%g" % (i,sess.run(loss),sess.run(w)))
        print('final w:',sess.run(w))

运行结果：

> step:0 , loss:27.9502 , w:2.71321  
> step:1 , loss:10.0621 , w:4.82793  
> step:2 , loss:3.62234 , w:6.09676  
> step:3 , loss:1.30404 , w:6.85805  
> step:4 , loss:0.469456 , w:7.31483  
> step:5 , loss:0.169004 , w:7.5889  
> step:6 , loss:0.0608415 , w:7.75334  
> step:7 , loss:0.0219029 , w:7.852  
> step:8 , loss:0.00788509 , w:7.9112  
> step:9 , loss:0.00283864 , w:7.94672  
> step:10 , loss:0.0010219 , w:7.96803  
> step:11 , loss:0.000367884 , w:7.98082  
> step:12 , loss:0.000132434 , w:7.98849  
> step:13 , loss:4.76735e-05 , w:7.9931  
> step:14 , loss:1.71625e-05 , w:7.99586  
> step:15 , loss:6.17896e-06 , w:7.99751  
> step:16 , loss:2.22471e-06 , w:7.99851  
> step:17 , loss:8.01067e-07 , w:7.9991  
> step:18 , loss:2.88282e-07 , w:7.99946  
> step:19 , loss:1.03904e-07 , w:7.99968  
> step:20 , loss:3.74794e-08 , w:7.99981  
> step:21 , loss:1.35369e-08 , w:7.99988  
> step:22 , loss:4.8467e-09 , w:7.99993  
> step:23 , loss:1.76078e-09 , w:7.99996  
> step:24 , loss:6.38693e-10 , w:7.99997  
> step:25 , loss:2.32831e-10 , w:7.99998  
> step:26 , loss:8.20819e-11 , w:7.99999  
> step:27 , loss:2.75122e-11 , w:7.99999  
> step:28 , loss:1.11413e-11 , w:8  
> step:29 , loss:3.63798e-12 , w:8  
> step:30 , loss:9.09495e-13 , w:8  
> step:31 , loss:2.27374e-13 , w:8  
> step:32 , loss:2.27374e-13 , w:8  
> step:33 , loss:2.27374e-13 , w:8  
> step:34 , loss:2.27374e-13 , w:8  
> step:35 , loss:2.27374e-13 , w:8  
> step:36 , loss:2.27374e-13 , w:8  
> step:37 , loss:2.27374e-13 , w:8  
> step:38 , loss:2.27374e-13 , w:8  
> step:39 , loss:2.27374e-13 , w:8  
> final w: \[\[7.9999995\]\]

可见，结果和直观求解是一样的！

# 2 搭建模块化神经网络八股 #

## 2-0 步骤 ##

主要可以分为两大部分：

第一部分：前向传播

第二部分：反向传播

前向传播

#part1
    '''
    前向传播，就是搭建网络，设计网络结构（一般就是建立一个forward.py文件）
    文件中包括
    '''
    # 前向传播过程
    def forward(x,regularizer):
        w =
        b =
        y =
        return y
    
    # 获取权重参数
    def get_weight(shape,regularizer):
        w = tf.Variable() #参数赋初值
        tf.add_to_collection('losses',tf.contrib.layers.l2_reqularizer(regularizer)(w)) #将正则化损失加上，返回w
        return w
    
    # 获取偏置参数
    def get_bias(shape):
        b = tf.Variable()
        return b

反向传播

'''
    part2
    反向传播，就是训练网络，优化网络参数（建立一个backward.py文件）
    文件包括：
    '''
    # 定义常量
    STEPS =
    LEARNING_RATE =
    ......
    
    
    def backward():
        x = tf.placeholder()
        y_ = tf.placeholder()
        y = backward(x, regularizer)  # 前向传播获得输出结果y
        global_step = tf.Variable(0, trainable=False)  # 定义轮数计数器
    
        # loss可以是
        
        
        # 如果是均方误差
        loss = tf.reduce_mean(tf.square(y - y_))
        # 如果是交叉商
        ce = tf.nn.softmax_cross_entropy_with_logits(logits=y, labels=tf.argmax(y_, 1))
        loss = tf.reduce_mean(ce)
        # 如果使用正则化
        loss = loss + tf.add_n(tf.get_collection('losses'))
        
        # 如果加入指数衰减，加入
        learning_rate = tf.train.exponential_decay(
            learning_rate_base
            global_steps
            数据集中样本数 / batch_size
            learning_rate_decay
            staircase = True)
        
        # 定义训练过程
        train_step = tf.train.GradientDescentOptimizer(learning_rate).minimize(loss, global_step=global_step)
        
        # 如果有滑动平均
        ema = tf.train.ExponentialMovingAverage(moving_average_decay, global_step)
        ema_op = ema.apply(tf.trainable_variables())
        with tf.control_dependencies([train_step, ema_op]):
            train_op = tf.no_op(name='train')
        
        # 建立会话，迭代
        with tf.Session() as sess:
            init_op = tf.global_variables_initializer()
            sess.run(init_op)
        
            for i in range(stops):
                sess.run(train_step, feed_dict={x:, y:})
                if i % 轮数 == 0:
                    print(..)
        
                # 判断该文件是否是主文件，是主文件，执行backward()
                if __name__ == '__main__':
                    backward()

## 2-2 实例 ##

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3BsU29uZ19DU0RO_size_16_color_FFFFFF_t_70][]

一共用了三个文件

opt4\_8\_generateds.py：生成数据集

opt4\_8\_forward.py：前向传播

opt4\_8\_backward.py：反向传播

[y _ _x-5_2]: https://private.codecogs.com/gif.latex?y%20%3D%20%28x-5%29%5E%7B2%7D
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3BsU29uZ19DU0RO_size_16_color_FFFFFF_t_70]: /images/20220310/7fdb8b36bd7f49b0a9f4375c0d03a6d9.png