tensorflow学习（2）

朴灿烈づ我的快乐病毒、 2022-02-22 14:42 230阅读 0赞

莫凡2017tensorflow（使用更简便版）[https://github.com/MorvanZhou/Tensorflow-Tutorial][https_github.com_MorvanZhou_Tensorflow-Tutorial]

10.激励函数activation function：即让某一部分的神经元先激活起来，把激活效应的信息传递到后面一层的神经系统里面。（激励方程、存放位置）

11.添加层：如何定义并添加一个神经层

[https://morvanzhou.github.io/tutorials/machine-learning/tensorflow/3-1-add-layer/][https_morvanzhou.github.io_tutorials_machine-learning_tensorflow_3-1-add-layer]

import tensorflow as tf
    
    def add_layer(inputs, in_size, out_size, activation_function=None):
        Weights = tf.Variable(tf.random_normal([in_size, out_size]))
        biases = tf.Variable(tf.zeros([1, out_size]) + 0.1)
        Wx_plus_b = tf.matmul(inputs, Weights) + biases
        if activation_function is None:
            outputs = Wx_plus_b
        else:
            outputs = activation_function(Wx_plus_b)
        return outputs

其中，定义添加神经层的函数`def add_layer()`,它有四个参数：输入值、输入的大小、输出的大小和激励函数，我们设定默认的激励函数是`None`。在生成初始参数时，随机变量(normal distribution)会比全部为0要好很多，所以我们这里的`weights`为一个`in_size`行, `out_size`列的随机变量矩阵。在机器学习中，`biases`的推荐值不为0，所以我们这里是在0向量的基础上又加了`0.1`。

12.建造神经网络：在11的基础上，继续讲解如何构造神经层

xs = tf.placeholder(tf.float32, [None, 1])

`tf.placeholder()`就是代表占位符，这里的`None`代表无论输入有多少都可以，因为输入只有一个特征，所以这里是`1`。

通常神经层都包括输入层、隐藏层和输出层。这里的输入层只有一个属性， 所以我们就只有一个输入；隐藏层我们可以自己假设，这里我们假设隐藏层有10个神经元； 输出层和输入层的结构是一样的，所以我们的输出层也是只有一层。 所以，我们构建的是——输入层1个、隐藏层10个、输出层1个的神经网络。

import tensorflow as tf
    import numpy as np
    
    def add_layer(inputs, in_size, out_size, activation_function=None):
        # add one more layer and return the output of this layer
        Weights = tf.Variable(tf.random_normal([in_size, out_size]))
        biases = tf.Variable(tf.zeros([1, out_size]) + 0.1)
        Wx_plus_b = tf.matmul(inputs, Weights) + biases
        if activation_function is None:
            outputs = Wx_plus_b
        else:
            outputs = activation_function(Wx_plus_b)
        return outputs
    
    # Make up some real data
    x_data = np.linspace(-1,1,300)[:, np.newaxis]  
    noise = np.random.normal(0, 0.05, x_data.shape)  #噪声生成
    y_data = np.square(x_data) - 0.5 + noise
    
    # define placeholder for inputs to network
    xs = tf.placeholder(tf.float32, [None, 1])
    ys = tf.placeholder(tf.float32, [None, 1])
    # add hidden layer
    l1 = add_layer(xs, 1, 10, activation_function=tf.nn.relu)
    # add output layer
    prediction = add_layer(l1, 10, 1, activation_function=None)
    
    # the error between prediciton and real data
    loss = tf.reduce_mean(tf.reduce_sum(tf.square(ys - prediction),
                         reduction_indices=[1]))  #对二者差的平方求和再取平均
    train_step = tf.train.GradientDescentOptimizer(0.1).minimize(loss)
    
    # important step
    init = tf.initialize_all_variables()
    sess = tf.Session()
    sess.run(init)
    
    for i in range(1000):
        # training
        sess.run(train_step, feed_dict={xs: x_data, ys: y_data})
        if i % 50 == 0:
            # to see the step improvement
            print(sess.run(loss, feed_dict={xs: x_data, ys: y_data}))
    '''
    0.066639826
    0.009021594
    0.006373957
    0.005147705
    0.004446703
    0.004053277
    0.0037726625
    0.003591398
    0.0034521723
    0.0033391833
    0.003239072
    0.0031602075
    0.0031008963
    0.003050547
    0.0030073707
    0.0029714515
    0.002940909
    0.0029167237
    0.0028977306
    0.0028768817
    '''

13.可视化

1.   注意：`plt.ion()`用于连续显示。（一般情况下，plt.show()之后的程序不会执行；plt.ion()这个函数，使matplotlib的显示模式转换为交互（interactive）模式）[https://blog.csdn.net/zbrwhut/article/details/80625702][https_blog.csdn.net_zbrwhut_article_details_80625702]
2.  try的用法
3.  抹除上次绘制的曲线

"""
    Please note, this code is only for python 3+. If you are using python 2+, please modify the code accordingly.
    """
    import tensorflow as tf
    import numpy as np
    import matplotlib.pyplot as plt
    
    def add_layer(inputs, in_size, out_size, activation_function=None):
        # add one more layer and return the output of this layer
        Weights = tf.Variable(tf.random_normal([in_size, out_size]))
        biases = tf.Variable(tf.zeros([1, out_size]) + 0.1)
        Wx_plus_b = tf.matmul(inputs, Weights) + biases
        if activation_function is None:
            outputs = Wx_plus_b
        else:
            outputs = activation_function(Wx_plus_b)
        return outputs
    
    # Make up some real data
    x_data = np.linspace(-1,1,300)[:, np.newaxis]
    noise = np.random.normal(0, 0.05, x_data.shape)
    y_data = np.square(x_data) - 0.5 + noise
    
    # define placeholder for inputs to network
    xs = tf.placeholder(tf.float32, [None, 1])
    ys = tf.placeholder(tf.float32, [None, 1])
    # add hidden layer
    l1 = add_layer(xs, 1, 10, activation_function=tf.nn.relu)
    # add output layer
    prediction = add_layer(l1, 10, 1, activation_function=None)
    
    # the error between prediciton and real data
    loss = tf.reduce_mean(tf.reduce_sum(tf.square(ys - prediction),
                         reduction_indices=[1]))
    train_step = tf.train.GradientDescentOptimizer(0.1).minimize(loss)
    
    # important step
    init = tf.initialize_all_variables()
    sess = tf.Session()
    sess.run(init)
    
    # plot the real data
    fig = plt.figure()
    ax = fig.add_subplot(1,1,1)
    ax.scatter(x_data, y_data)
    plt.ion()
    plt.show()
    
    for i in range(1000):
        # training
        sess.run(train_step, feed_dict={xs: x_data, ys: y_data})
        if i % 50 == 0:
            # to visualize the result and improvement
            try:
                ax.lines.remove(lines[0])
            except Exception:
                pass
            prediction_value = sess.run(prediction, feed_dict={xs: x_data})
            # plot the prediction
            lines = ax.plot(x_data, prediction_value, 'r-', lw=5)
            plt.pause(0.1)

其中一个运行结果截图如下：

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2p1bGlhcmp1bGlhcg_size_16_color_FFFFFF_t_70][]

pycharm运行时，开始会有不出现红色这条动态线的情况，参考[https://morvanzhou.github.io/tutorials/machine-learning/tensorflow/3-3-visualize-result/][https_morvanzhou.github.io_tutorials_machine-learning_tensorflow_3-3-visualize-result]，将Setting->Tools->Python Scientific->Show Toolwindow ,然后取消勾选就可以里，sciview就可以弹出来，然后就好用了。

14.优化器optimizer

tensorflow的7种优化器

![3_4_1.png][]

莫凡的更多关于optimizer的解释，参考[https://morvanzhou.github.io/tutorials/machine-learning/ML-intro/3-06-speed-up-learning/][https_morvanzhou.github.io_tutorials_machine-learning_ML-intro_3-06-speed-up-learning]

15.可视化好助手tensorboard

[https://morvanzhou.github.io/tutorials/machine-learning/tensorflow/4-1-tensorboard1/][https_morvanzhou.github.io_tutorials_machine-learning_tensorflow_4-1-tensorboard1]

学会用 Tensorflow 自带的 tensorboard 去可视化我们所建造出来的神经网络是一个很好的学习理解方式. 用最直观的流程图告诉你你的神经网络是长怎样,有助于你发现编程中间的问题和疑问.与 tensorboard 兼容的浏览器是 “Google Chrome”. 使用其他的浏览器不保证所有内容都能正常显示.

"""
    Please note, this code is only for python 3+. If you are using python 2+, please modify the code accordingly.
    """
    import tensorflow as tf
    import os
    
    os.environ['TF_CPP_MIN_LOG_LEVEL'] = '2'
    
    def add_layer(inputs, in_size, out_size, activation_function=None):
        # add one more layer and return the output of this layer
        with tf.name_scope('layer'):
            with tf.name_scope('weights'):
                Weights = tf.Variable(tf.random_normal([in_size, out_size]), name='W')
            with tf.name_scope('biases'):
                biases = tf.Variable(tf.zeros([1, out_size]) + 0.1, name='b')
            with tf.name_scope('Wx_plus_b'):
                Wx_plus_b = tf.add(tf.matmul(inputs, Weights), biases)
            if activation_function is None:
                outputs = Wx_plus_b
            else:
                outputs = activation_function(Wx_plus_b, )
            return outputs
    
    
    # define placeholder for inputs to network
    with tf.name_scope('inputs'):
        xs = tf.placeholder(tf.float32, [None, 1], name='x_input')
        ys = tf.placeholder(tf.float32, [None, 1], name='y_input')
    
    # add hidden layer
    l1 = add_layer(xs, 1, 10, activation_function=tf.nn.relu)
    # add output layer
    prediction = add_layer(l1, 10, 1, activation_function=None)
    
    # the error between prediciton and real data
    with tf.name_scope('loss'):
        loss = tf.reduce_mean(tf.reduce_sum(tf.square(ys - prediction),
                                            reduction_indices=[1]))
    
    with tf.name_scope('train'):
        train_step = tf.train.GradientDescentOptimizer(0.1).minimize(loss)
    
    sess = tf.Session()
    #writer = tf.train.SummaryWriter("logs/", sess.graph)
    writer = tf.summary.FileWriter("logs/", sess.graph)  #https://blog.csdn.net/zeuseign/article/details/72771598
    # important step
    sess.run(tf.initialize_all_variables())

其中

os.environ[“TF_CPP_MIN_LOG_LEVEL”]=‘1’ # 这是默认的显示等级，显示所有信息
    os.environ[“TF_CPP_MIN_LOG_LEVEL”]=‘2’ # 只显示 warning 和 Error
    os.environ[“TF_CPP_MIN_LOG_LEVEL”]=‘3’ # 只显示 Error

我们一般用第二个就可以了。

使用`with tf.name_scope('inputs')`可以将`xs`和`ys`包含进来，形成一个大的图层，图层的名字就是`with tf.name_scope()`方法里的参数。

莫凡编译文件目录如下，

![20190410102314843.png][]

想直接运行，

1.打开终端

2.在终端中打开至logs上层目录

3.输入 tensorboard --logdir logs

出现以下问题：（http://PC201710130840，在谷歌浏览器中打不开）

![20190410102623259.png][]

且会报错如下：

![20190410103530270.png][]

参考[https://morvanzhou.github.io/tutorials/machine-learning/tensorflow/4-1-tensorboard1/][https_morvanzhou.github.io_tutorials_machine-learning_tensorflow_4-1-tensorboard1]下评论的windows上的解决方法，[https://blog.csdn.net/u011228094/article/details/85634261][https_blog.csdn.net_u011228094_article_details_85634261]，直接在相应的F下执行

tensorboard --logdir=F:\\BaiduNetdiskDownload\\04-深度学习课程\\TensorFlow教程\\Tensorflow视频教程\\tensorflowTUT源码\\tf14\_tensorboard\\logs，出现了http://127.0.0.1:6006，但是在谷歌浏览器中仍然打不开[https://blog.csdn.net/Sunny\_Guang/article/details/79914068][https_blog.csdn.net_Sunny_Guang_article_details_79914068]

![20190410103407837.png][]

考虑主要是因为Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 提示，原因是下载的tensorflow版本不支持cpu的AVX2编译。参考[https://blog.csdn.net/yimixgg/article/details/80676619][https_blog.csdn.net_yimixgg_article_details_80676619]未找到解决办法。可能需要重新安装有AVX2编译的tensorflow版本[https://github.com/fo40225/tensorflow-windows-wheel][https_github.com_fo40225_tensorflow-windows-wheel]和[https://github.com/lakshayg/tensorflow-build][https_github.com_lakshayg_tensorflow-build]。

16.可视化好助手2

上一篇讲到了 如何可视化TesorBorad整个神经网络结构的过程。 其实tensorboard还可以可视化训练过程( biase变化过程) , 这节重点讲一下可视化训练过程的图标是如何做的 。

"""
    Please note, this code is only for python 3+. If you are using python 2+, please modify the code accordingly.
    """
    import tensorflow as tf
    import numpy as np
    import os
    
    os.environ['TF_CPP_MIN_LOG_LEVEL'] = '2'
    
    def add_layer(inputs, in_size, out_size, n_layer, activation_function=None):
        # add one more layer and return the output of this layer
        layer_name = 'layer%s' % n_layer
        with tf.name_scope(layer_name):
            with tf.name_scope('weights'):
                Weights = tf.Variable(tf.random_normal([in_size, out_size]), name='W')
                #tf.histogram_summary(layer_name + '/weights', Weights)
                tf.summary.histogram(layer_name + '/weights', Weights)  # tensorflow >= 0.12
            with tf.name_scope('biases'):
                biases = tf.Variable(tf.zeros([1, out_size]) + 0.1, name='b')
                #tf.histogram_summary(layer_name + '/biases', biases)
                tf.summary.histogram(layer_name + '/biases', biases)  # Tensorflow >= 0.12
            with tf.name_scope('Wx_plus_b'):
                Wx_plus_b = tf.add(tf.matmul(inputs, Weights), biases)
            if activation_function is None:
                outputs = Wx_plus_b
            else:
                outputs = activation_function(Wx_plus_b, )
            #tf.histogram_summary(layer_name + '/outputs', outputs)
            tf.summary.histogram(layer_name + '/outputs', outputs)  # Tensorflow >= 0.12
            return outputs
    
    
    # Make up some real data
    x_data = np.linspace(-1, 1, 300)[:, np.newaxis]
    noise = np.random.normal(0, 0.05, x_data.shape)
    y_data = np.square(x_data) - 0.5 + noise
    
    # define placeholder for inputs to network
    with tf.name_scope('inputs'):
        xs = tf.placeholder(tf.float32, [None, 1], name='x_input')
        ys = tf.placeholder(tf.float32, [None, 1], name='y_input')
    
    # add hidden layer
    l1 = add_layer(xs, 1, 10, n_layer=1, activation_function=tf.nn.relu)
    # add output layer
    prediction = add_layer(l1, 10, 1, n_layer=2, activation_function=None)
    
    # the error between prediciton and real data
    with tf.name_scope('loss'):
        loss = tf.reduce_mean(tf.reduce_sum(tf.square(ys - prediction),
                                            reduction_indices=[1]))
        #tf.scalar_summary('loss', loss)
        tf.summary.scalar('loss', loss)  # tensorflow >= 0.12
    
    with tf.name_scope('train'):
        train_step = tf.train.GradientDescentOptimizer(0.1).minimize(loss)
    
    sess = tf.Session()
    #merged = tf.merge_all_summaries()  # tensorflow < 0.12
    merged = tf.summary.merge_all() # tensorflow >= 0.12
    #writer = tf.train.SummaryWriter("logs/", sess.graph)
    writer = tf.summary.FileWriter("logs/", sess.graph) # tensorflow >=0.12
    # important step
    #sess.run(tf.initialize_all_variables()) # tf.initialize_all_variables() # tf 马上就要废弃这种写法
    sess.run(tf.global_variables_initializer())  # 替换成这样就好
    for i in range(1000):
        sess.run(train_step, feed_dict={xs: x_data, ys: y_data})
        if i % 50 == 0:
            result = sess.run(merged,
                              feed_dict={xs: x_data, ys: y_data})
            writer.add_summary(result, i)

在.py文件平行文件中成功生成logs及其下级文件。

[https_github.com_MorvanZhou_Tensorflow-Tutorial]: https://github.com/MorvanZhou/Tensorflow-Tutorial
[https_morvanzhou.github.io_tutorials_machine-learning_tensorflow_3-1-add-layer]: https://morvanzhou.github.io/tutorials/machine-learning/tensorflow/3-1-add-layer/
[https_blog.csdn.net_zbrwhut_article_details_80625702]: https://blog.csdn.net/zbrwhut/article/details/80625702
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2p1bGlhcmp1bGlhcg_size_16_color_FFFFFF_t_70]: /images/20220222/dee5f28228ee43b6ab4bcae0250faa82.png
[https_morvanzhou.github.io_tutorials_machine-learning_tensorflow_3-3-visualize-result]: https://morvanzhou.github.io/tutorials/machine-learning/tensorflow/3-3-visualize-result/
[3_4_1.png]: /images/20220222/0ee2c58fe9e04f6eaa70fb76106019ba.png
[https_morvanzhou.github.io_tutorials_machine-learning_ML-intro_3-06-speed-up-learning]: https://morvanzhou.github.io/tutorials/machine-learning/ML-intro/3-06-speed-up-learning/
[https_morvanzhou.github.io_tutorials_machine-learning_tensorflow_4-1-tensorboard1]: https://morvanzhou.github.io/tutorials/machine-learning/tensorflow/4-1-tensorboard1/
[20190410102314843.png]: /images/20220222/5d1c1d9e7e33427eb6a7a246fd5c1ccb.png
[20190410102623259.png]: /images/20220222/2052a631e512403bae05edd6d9475ce2.png
[20190410103530270.png]: /images/20220222/957b86b67ce444dab47f7ae71ca68adb.png
[https_blog.csdn.net_u011228094_article_details_85634261]: https://blog.csdn.net/u011228094/article/details/85634261
[https_blog.csdn.net_Sunny_Guang_article_details_79914068]: https://blog.csdn.net/Sunny_Guang/article/details/79914068
[20190410103407837.png]: /images/20220222/1036f3cc22154f5da805895310cac726.png
[https_blog.csdn.net_yimixgg_article_details_80676619]: https://blog.csdn.net/yimixgg/article/details/80676619
[https_github.com_fo40225_tensorflow-windows-wheel]: https://github.com/fo40225/tensorflow-windows-wheel
[https_github.com_lakshayg_tensorflow-build]: https://github.com/lakshayg/tensorflow-build