TensorFlow入门：MNIST预测[restore问题]

╰半夏微凉° 2021-10-25 12:28 374阅读 0赞

变量的恢复可按照两种方式导入：

saver=tf.train.Saver()
    
    saver.restore(sess,'model.ckpt')

或者：

saver=tf.train.import_meta_graph(r'D:\tmp\tensorflow\mnist\model.ckpt.meta')
    
    saver.restore(sess,'model.ckpt')

两种方法的效果应该一致，但是实际结果不一样：

使用前者时预测结果是一致的；使用后者时，每次运行结果都不一致。无论是否重启spyde，现象都一样。

在使用前者时，必须在运行前重启spyde，否则会报错，为什么？Out\_1等参数会随运行次数增加

INFO:tensorflow:Restoring parameters from D:/tmp/tensorflow/mnist/model.ckpt
    Traceback (most recent call last):
    
      File "<ipython-input-2-61410824b24c>", line 1, in <module>
        runfile('D:/wangjc/pythonTest/TensorFlow/TestMNIST_Predict.py', wdir='D:/wangjc/pythonTest/TensorFlow')
    
     ......
    
      File "D:\software\anaconda\envs\tensorflow\lib\site-packages\tensorflow\python\client\session.py", line 1052, in _do_call
        raise type(e)(node_def, op, message)
    
    NotFoundError: Key out_1/bias/bias not found in checkpoint
         [[Node: save_1/RestoreV2_14 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_recv_save_1/Const_0, save_1/RestoreV2_14/tensor_names, save_1/RestoreV2_14/shape_and_slices)]]
    
    Caused by op 'save_1/RestoreV2_14', defined at:
      File "D:\software\anaconda\envs\tensorflow\lib\site-packages\spyder\utils\ipython\start_kernel.py", line 241, in <module>
        main()
      ......File "D:\software\anaconda\envs\tensorflow\lib\site-packages\tensorflow\python\framework\ops.py", line 1228, in __init__
        self._traceback = _extract_stack()
    
    NotFoundError (see above for traceback): Key out_1/bias/bias not found in checkpoint
         [[Node: save_1/RestoreV2_14 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_recv_save_1/Const_0, save_1/RestoreV2_14/tensor_names, save_1/RestoreV2_14/shape_and_slices)]]

NotFoundError: Key out_2/weight/weight not found in checkpoint
         [[Node: save_2/RestoreV2_23 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/cpu:0"](_recv_save_2/Const_0, save_2/RestoreV2_23/tensor_names, save_2/RestoreV2_23/shape_and_slices)]]
    
    Caused by op 'save_2/RestoreV2_23', defined at:

以上需要重启spyder的原因为saver恢复一次之后不能再次恢复，否则报错。

导致*saver=tf.train.Saver()*与*saver=tf.train.import\_meta\_graph(r'D:\\tmp\\tensorflow\\mnist\\model.ckpt.meta')*结果不同的原因是，后者在使用中可直接加载模型的参数，操作数等。

tf.get\_default\_graph()获取图

.get\_tensor\_by\_name()获取张量

.get\_operation\_by\_name()获取操作

注意对各部分命名。

[参考1][1]，[参考2][2]

使用下面方法的效果与直接读ckpt文件一致

saver = tf.train.Saver()
    ckpt=tf.train.get_checkpoint_state(r'D:\tmp\tensorflow\mnist')
    
    saver.restore(sess, ckpt.model_checkpoint_path)

可使用*tf.get\_collection('name')*来读取恢复的变量

注意定义变量时最好标注标量名称，否则可能出现预测时加载参数不正确，定义方法为：

def weight_variable(shape):
        #use normal distribution numbers with stddev 0.1 to initial the weight
        initial=tf.truncated_normal(shape, stddev=0.1)
        return tf.Variable(initial,name='weight')

\-------------------------------------------------------------------------------------------------------------------------------

训练并保存模型 代码

# -*- coding: utf-8 -*-
    """
    Created on Mon Sep 11 10:16:34 2017
    
    multy layers softmax regression
    
    @author: Wangjc
    """
    
    import tensorflow as tf
    import os
    import tensorflow.examples.tutorials.mnist.input_data as input_data
    #need to show the full address, or error occus.
    mnist = input_data.read_data_sets('MNIST_data', one_hot=True)
    #use read_data_sets to download and load the mnist data set. if has the data, then load.
    #need a long time about 5 minutes
    
    sess = tf.InteractiveSession()
    #link the back-end of C++ to compute.
    #in norm cases, we should create the map and then run in the sussion.
    #now, use a more convenient class named InteractiveSession which could insert compute map when running map.
    
    x=tf.placeholder("float",shape=[None,784])
    y_=tf.placeholder("float",shape=[None,10])
    
    
    def weight_variable(shape):
        #use normal distribution numbers with stddev 0.1 to initial the weight
        initial=tf.truncated_normal(shape, stddev=0.1)
        return tf.Variable(initial,name='weight')
        
    def bias_variable(shape):
        #use constant value of 0.1 to initial the bias
        initial=tf.constant(0.1, shape=shape)
        return tf.Variable(initial,name='bias')
    
    def conv2d(x,W):
        #convolution by filter of W,with step size of 1, 0 padding size
        #x should have the dimension of [batch,height,width,channels]
        #other dimension of strides or ksize is the same with x
        return tf.nn.conv2d(x,W,strides=[1,1,1,1],padding='SAME')
    
    def max_pool_2x2(x):
        #pool by windows of ksize,with step size of 2, 0 padding size
        return tf.nn.max_pool(x,ksize=[1,2,2,1],
                              strides=[1,2,2,1],padding='SAME')
    
    
    #------------------------------------------------
    x_image = tf.reshape(x, [-1,28,28,1])
    #to use conv1, need to convert x to 4D, in form of [batch,height,width,channels]
    # -1 means default
        
    with tf.name_scope('conv1'):
        #use 'with' and name_scope to define a name space which will show in tensorboard as a ragion
        with tf.name_scope('weight'):
            W_conv1=weight_variable([5,5,1,32])
            tf.summary.histogram('conv1'+'/weight',W_conv1)
            #summary the variation ('name', value) 
        with tf.name_scope('bias'):
            b_conv1=bias_variable([32])
            tf.summary.histogram('conv1'+'/bias',b_conv1)
    #build the first conv layer:
    #get 32 features from every 5*5 patch, so the shape is [5,5,1(channel),32]
    
        h_conv1 = tf.nn.relu(conv2d(x_image, W_conv1) + b_conv1)
    
    with tf.name_scope('pool1'):    
        h_pool1 = max_pool_2x2(h_conv1)
    
    #--------------------------------------------
    with tf.name_scope('conv2'):
        with tf.name_scope('weight'):    
            W_conv2=weight_variable([5,5,32,64])
            tf.summary.histogram('weight',W_conv2)
        with tf.name_scope('bias'):  
            b_conv2=bias_variable([64])
            tf.summary.histogram('bias',b_conv2)
    #build the 2nd conv layer:
    #get 64 features from every 5*5 patch, so the shape is [5,5,32(channel),64]
    
        h_conv2 = tf.nn.relu(conv2d(h_pool1, W_conv2) + b_conv2)
    with tf.name_scope('pool2'):    
        h_pool2 = max_pool_2x2(h_conv2)
    
    #----------------------------------------
    #image size reduce to 7*7 by pooling
    #we add a full connect layer contains 1027 nuere
    #need to flat pool tensor for caculate
    with tf.name_scope('fc1'):
        with tf.name_scope('weight'):    
            W_fc1 = weight_variable([7*7*64, 1024])
            tf.summary.histogram('weight',W_fc1)
        with tf.name_scope('bias'):
            b_fc1 = bias_variable([1024])
            tf.summary.histogram('bias',b_fc1)
    
        h_pool2_flat = tf.reshape(h_pool2,[-1, 7*7*64])
        
        h_fc1 = tf.nn.relu(tf.matmul(h_pool2_flat,W_fc1) + b_fc1)
    
    #------------------------------------
    #output layer
    with tf.name_scope('out'):
        keep_prob = tf.placeholder("float")
        h_fc1_drop = tf.nn.dropout(h_fc1,keep_prob)
    #to decrease overfit, we add dropout before output layer.
    #use placeholder to represent the porbability of a neure's output value unchange
    
        with tf.name_scope('weight'):
            W_fc2 = weight_variable([1024, 10])
            tf.summary.histogram('weight',W_fc2)
        with tf.name_scope('bias'):
            b_fc2 = bias_variable([10])
            tf.summary.histogram('bias',b_fc2)
        y_conv = tf.nn.softmax(tf.matmul(h_fc1_drop, W_fc2) + b_fc2)
    
    #---------------------------------
    #train and evaluate the module
    #use a ADAM
    
    cross_entropy=-tf.reduce_sum(y_*tf.log(y_conv))
    tf.summary.scalar('cross_entropy',cross_entropy)
    ##summary the constant ('name', value) 
    train_step=tf.train.AdamOptimizer(1e-4).minimize(cross_entropy)
    correct_prediction = tf.equal(tf.argmax(y_conv,1), tf.argmax(y_,1))
    accuracy = tf.reduce_mean(tf.cast(correct_prediction, "float"))
    
    #sess = tf.Session()
    
    merged=tf.summary.merge_all()
    #merge all the summary nodes
    writer=tf.summary.FileWriter('D:/tmp/tensorflow/mnist/',sess.graph)
    # assign the event file write directory 
    
    saver=tf.train.Saver()
    #saver for variation.Dafault to save all.
    checkpoint_file = os.path.join('D:/tmp/tensorflow/mnist/', 'model.ckpt')
    #save directroy for variation
    
    sess.run(tf.global_variables_initializer())
    for i in range(100):
        batch = mnist.train.next_batch(50)
        if i%100 == 0:
            train_accuracy = accuracy.eval(feed_dict={x:batch[0], y_:batch[1],keep_prob:1.0})
    #        saver.save(sess,checkpoint_file)
    #        saver.save(sess,checkpoint_file,global_step=i)
            #save variation
            print("step %d, training accuracy %g"%(i, train_accuracy))
            result=sess.run(merged,feed_dict={x: batch[0], y_: batch[1], keep_prob: 0.5})
            #the merged summary need to be run
            writer.add_summary(result,i)
            #add the result to summary
        train_step.run(feed_dict={x: batch[0], y_: batch[1], keep_prob: 0.5})
        
    print("test accuracy %g"%accuracy.eval(feed_dict={
            x: mnist.test.images, y_: mnist.test.labels, keep_prob: 1.0}))
    
    saver.save(sess,checkpoint_file)

读取图片，恢复参数并预测 代码

# -*- coding: utf-8 -*-
    """
    Created on Mon Sep 11 10:16:34 2017
    
    multy layers softmax regression
    
    @author: Wangjc
    """
    
    import tensorflow as tf
    import os
    import cv2
    import matplotlib.pyplot as plt
    #need to show the full address, or error occus.
    
    imgs0=cv2.imread(r'D:\tmp\tensorflow\imgs\_1.png',0)
    plt.imshow(imgs0)
    plt.show()
    imgs=imgs0/255
    #imgs=(255-imgs0)/255
    imgs.shape=(1,784)
    
    
    
    sess = tf.InteractiveSession()
    #link the back-end of C++ to compute.
    #in norm cases, we should create the map and then run in the sussion.
    #now, use a more convenient class named InteractiveSession which could insert compute map when running map.
    
    x=tf.placeholder("float",shape=[None,784])
    
    
    
    def weight_variable(shape):
        #use normal distribution numbers with stddev 0.1 to initial the weight
        initial=tf.truncated_normal(shape, stddev=0.1)
        return tf.Variable(initial,name='weight')
        
    def bias_variable(shape):
        #use constant value of 0.1 to initial the bias
        initial=tf.constant(0.1, shape=shape)
        return tf.Variable(initial,name='bias')
    
    def conv2d(x,W):
        #convolution by filter of W,with step size of 1, 0 padding size
        #x should have the dimension of [batch,height,width,channels]
        #other dimension of strides or ksize is the same with x
        return tf.nn.conv2d(x,W,strides=[1,1,1,1],padding='SAME')
    
    def max_pool_2x2(x):
        #pool by windows of ksize,with step size of 2, 0 padding size
        return tf.nn.max_pool(x,ksize=[1,2,2,1],
                              strides=[1,2,2,1],padding='SAME')
    
    
    #------------------------------------------------
    x_image = tf.reshape(x, [-1,28,28,1])
    #to use conv1, need to convert x to 4D, in form of [batch,height,width,channels]
    # -1 means default
        
    with tf.name_scope('conv1'):
        #use 'with' and name_scope to define a name space which will show in tensorboard as a ragion
        with tf.name_scope('weight'):
            W_conv1=weight_variable([5,5,1,32])
    #        tf.summary.histogram('conv1'+'/weight',W_conv1)
            #summary the variation ('name', value) 
        with tf.name_scope('bias'):
            b_conv1=bias_variable([32])
    #        tf.summary.histogram('conv1'+'/bias',b_conv1)
    #build the first conv layer:
    #get 32 features from every 5*5 patch, so the shape is [5,5,1(channel),32]
    
        h_conv1 = tf.nn.relu(conv2d(x_image, W_conv1) + b_conv1)
    
    with tf.name_scope('pool1'):    
        h_pool1 = max_pool_2x2(h_conv1)
    
    #--------------------------------------------
    with tf.name_scope('conv2'):
        with tf.name_scope('weight'):    
            W_conv2=weight_variable([5,5,32,64])
    #        tf.summary.histogram('weight',W_conv2)
        with tf.name_scope('bias'):  
            b_conv2=bias_variable([64])
    #        tf.summary.histogram('bias',b_conv2)
    #build the 2nd conv layer:
    #get 64 features from every 5*5 patch, so the shape is [5,5,32(channel),64]
    
        h_conv2 = tf.nn.relu(conv2d(h_pool1, W_conv2) + b_conv2)
    with tf.name_scope('pool2'):    
        h_pool2 = max_pool_2x2(h_conv2)
    
    #----------------------------------------
    #image size reduce to 7*7 by pooling
    #we add a full connect layer contains 1027 nuere
    #need to flat pool tensor for caculate
    with tf.name_scope('fc1'):
        with tf.name_scope('weight'):    
            W_fc1 = weight_variable([7*7*64, 1024])
    #        tf.summary.histogram('weight',W_fc1)
        with tf.name_scope('bias'):
            b_fc1 = bias_variable([1024])
    #        tf.summary.histogram('bias',b_fc1)
    
        h_pool2_flat = tf.reshape(h_pool2,[-1, 7*7*64])
        
        h_fc1 = tf.nn.relu(tf.matmul(h_pool2_flat,W_fc1) + b_fc1)
    
    #------------------------------------
    #output layer
    with tf.name_scope('out'):
        keep_prob = tf.placeholder("float")
    #    h_fc1_drop = tf.nn.dropout(h_fc1,keep_prob)
    #to decrease overfit, we add dropout before output layer.
    #use placeholder to represent the porbability of a neure's output value unchange
    
        with tf.name_scope('weight'):
            W_fc2 = weight_variable([1024, 10])
    #        tf.summary.histogram('weight',W_fc2)
        with tf.name_scope('bias'):
            b_fc2 = bias_variable([10])
    #        tf.summary.histogram('bias',b_fc2)
        y_conv = tf.nn.softmax(tf.matmul(h_fc1, W_fc2) + b_fc2)
    
    #---------------------------------
    
    
    #saver=tf.train.import_meta_graph(r'D:\tmp\tensorflow\mnist\model.ckpt.meta')
    saver=tf.train.Saver()
    #saver for variation.Dafault to save all.
    checkpoint_file = os.path.join('D:/tmp/tensorflow/mnist/', 'model.ckpt')
    #save directroy for variation
    
    sess.run(tf.global_variables_initializer())
    
    saver.restore(sess,checkpoint_file)
    #saver.recover_last_checkpoints(checkpoint_file)
    
    #prediction=tf.argmax(y_conv,1)
    #result=prediction.eval(feed_dict={x: imgs})
    
    result=sess.run(tf.argmax(y_conv,1),feed_dict={x: imgs,keep_prob: 0.5})
    #result=prediction.eval(feed_dict={x: imgs,keep_prob: 0.5})
    
    
    print('recognize result')
    print(result[0])

Stack Overflow[参考1][1 1]，[参考2][2 1]

[模型存储与恢复][Link 1]

转载于:https://www.cnblogs.com/Osler/p/7687213.html

[1]: http://www.cnblogs.com/seaspring/p/6766923.html
[2]: http://www.jianshu.com/p/8487db911d9a
[1 1]: https://stackoverflow.com/questions/38947658/tensorflow-saving-into-loading-a-graph-from-a-file
[2 1]: https://stackoverflow.com/questions/33759623/tensorflow-how-to-save-restore-a-model
[Link 1]: http://cv-tricks.com/tensorflow-tutorial/save-restore-tensorflow-models-quick-complete-tutorial/