全连接神经网络、RBF、CNN实现

柔光的暖阳◎ 2023-06-30 09:09 11阅读 0赞

### 文章目录 ###

*   *  一、实验介绍
     *   *  （1）实验目的
         *  （2）数据集简介
         *  （3）实验内容
         *  （4）评价指标
         *  二、全连接神经网络
     *  三、RBF
     *  四、CNN
     *  五、结果
     *   *  （1）acc结果对比
         *  （2）结果分析

## 一、实验介绍 ##

### （1）实验目的 ###

1.  掌握全连接神经网络分类器的训练与测试方法。
2.  掌握基于RBF分类器训练与测试方法。
3.  掌握CNN的基本原理、训练与测试方法。

### （2）数据集简介 ###

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pcmFjbGVvYQ_size_16_color_FFFFFF_t_70_pic_center]

*  上述数据集均一.mat文件存放在/datasets文件夹下。

### （3）实验内容 ###

1.  编写程序实现全连接神经网络分类器设计。
2.  编写程序实现基于RBF分类器设计。
3.  编写程序实现基于CNN分类器设计。

### （4）评价指标 ###

*  本次实验主要利用Acc指标对聚类结果进行评价，值越大表明聚类效果越好。

### 二、全连接神经网络 ###

import tensorflow as tf
    import datadvi
    import numpy as np
    
    def FCNN(filename, batch_size=50,learning_rate=0.0001,iteration_times=5000):
        dataX_train,dataX_predict,dataY_train,dataY_predict = datadvi.divdata(filename)
        X = dataX_train
        Y1 = dataY_train
        # print(Y1)
        num_label = len(np.unique(Y1))
        Y = (np.arange(num_label)+1 == Y1[:, None]).astype(np.float32)
        dataY_predict1 = (np.arange(num_label)+1 == dataY_predict[:, None]).astype(np.float32)
        x = tf.placeholder(tf.float32, shape=[None, len(X[0])])
        y = tf.placeholder(tf.float32, shape=[None, num_label])
    
        w1 = tf.Variable(tf.random_normal([len(X[0]), 256], stddev=1, seed=1))
        w2 = tf.Variable(tf.random_normal([256, 256], stddev=1, seed=1))
        w_out = tf.Variable(tf.random_normal([256, num_label], stddev=1, seed=1))
    
        b1 = tf.Variable(tf.random_normal([256]))
        b2 = tf.Variable(tf.random_normal([256]))
        b_out = tf.Variable(tf.random_normal([num_label]))
    
        def Fully_neural_network(X):
            layer_1 = tf.nn.relu(tf.add(tf.matmul(X, w1), b1))
            layer_2 = tf.nn.relu(tf.add(tf.matmul(layer_1, w2), b2))
            layer_out = tf.matmul(layer_2, w_out) + b_out
    
            return layer_out
    
        net_out = Fully_neural_network(x)
    
        pre = tf.nn.softmax(net_out)
        pre1 = tf.argmax(pre, 1)
    
        loss = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=net_out, labels=y))
        # loss = tf.reduce_mean(tf.abs(net_out-y))
    
        optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate)
        train_op = optimizer.minimize(loss)
    
        correct_pre = tf.equal(tf.argmax(pre, 1), tf.argmax(y, 1))
        accuracy = tf.reduce_mean(tf.cast(correct_pre, tf.float32))
    
        init = tf.global_variables_initializer()
    
        with tf.Session() as sess:
            sess.run(init)
    
            for i in range(1, iteration_times + 1):
                start = (i * batch_size) % len(X)
                end = min(start + batch_size, len(X))
                batch_x = X[start:end]
                batch_y = Y[start:end]
                sess.run(train_op, feed_dict={ x: batch_x, y: batch_y})
                if i == iteration_times-1:
                    l, acc = sess.run([loss, accuracy], feed_dict={ x: dataX_predict, y: dataY_predict1})
                    print("Step " + str(i) + ", Minibatch Loss= " + "{:.4f}".format(
                        l) + ", Training Accuracy= " + "{:.3f}".format(acc))
    
                    # print(pre1.eval(feed_dict={x: batch_x}))

## 三、RBF ##

import random
    
    import numpy as np
    import tensorflow as tf
    from numpy.random import RandomState
    from sklearn.cluster import KMeans
    
    import datadvi
    from Acc import ACC
    
    
    def compute_sigma(c):
        sigma = np.zeros([len(c)])
        for i in range(len(c)):
            for j in range(len(c)):
                temp_dist = np.sum(np.square(c[i] - c[j]))
                if sigma[i] < temp_dist:
                    sigma[i] = temp_dist
        # print(sigma)
        return sigma
    
    
    def init_C(dataX, num_label, label):
        c = np.empty(shape=(num_label, dataX.shape[-1]), dtype=np.float32)
        for i in range(num_label):
            c[i] = random.choice(dataX[label[:] == i + 1])
        # print(c)
        return c
    
    
    def cluster_center(dataX, num_label):
        KM = KMeans(n_clusters=num_label, random_state=0).fit(dataX)
        return KM.cluster_centers_
    
    
    def gen_data():
        rdm = RandomState(1)
        dataset_size = 1280
        X = rdm.rand(dataset_size, 2)
        Y1 = [[int(x1 + x2 < 1)] for (x1, x2) in X]
        Y = np.array(Y1).T[0]
    
        return X, X, Y, Y
    
    
    def RBFNN(filename, batch_size=50, learning_rate=0.01, iteration_times=10000):
        dataX_train, dataX_predict, dataY_train, dataY_predict = datadvi.divdata(filename)  # 划分训练集和测试集
        # dataX_train, dataX_predict, dataY_train, dataY_predict = gen_data()
    
        # 数据标准化
        # dataX_train = StandardScaler().fit_transform(dataX_train)
        # dataX_predict = StandardScaler().fit_transform(dataX_predict)
        # print(len(dataX_predict[0]))
        # dataX_train = (dataX_train-np.mean(dataX_train,axis=0))/np.std(dataX_train,axis=0)
        # dataX_predict = (dataX_predict - np.mean(dataX_predict, axis=0)) / np.std(dataX_predict, axis=0)
        # print(dataX_predict)
    
        num_label = len(np.unique(dataY_train))  # 数据种类
        num_feature = len(dataX_train[0])  # 数据维度
        # print(num_label)
    
        # 将标签转换成onehot形式
        dataY_train_onehot = (np.arange(num_label) + 1 == dataY_train[:, None]).astype(np.float32)
        dataY_predict_onehot = (np.arange(num_label) + 1 == dataY_predict[:, None]).astype(np.float32)
    
        # 使用原始标签数据
        dataY_train_origin = np.array([dataY_train]).T
        dataY_predict_origin = np.array([dataY_predict]).T
    
        # 定义占位
        X = tf.placeholder(tf.float32, shape=[None, num_feature])
        # Y = tf.placeholder(tf.float32,shape=[None,num_label])
        Y = tf.placeholder(tf.float32, shape=[None, 1])
    
        # 初始化中心点c和sigma（隐藏节点数与数据种类相同）
        # c = tf.Variable(tf.random_normal([num_label,num_feature]))
        # c = tf.Variable(c1)
        c = cluster_center(dataX_train, num_label)
        # c = tf.Variable(tf.cast(c,tf.float32))
    
        # sigma = tf.Variable(tf.ones([num_label]))
        sigma1 = compute_sigma(c)
        sigma = tf.Variable(tf.cast(sigma1, tf.float32))
        # sigma = tf.Variable(tf.cast(c[:,0],tf.float32))
        # print(sigma)
    
        # 初始化权重W（使用onehot表示则输出节点数与种类相同)
        # W = tf.Variable(tf.random_normal([num_label,num_label]))
    
        W = tf.Variable(tf.random_normal([num_label, 1]))
    
        # 计算隐藏节点输出K1
        # sigma2 = tf.square(sigma)
        K = []
        for i in range(num_label):
            K.append(tf.reduce_sum(tf.square((X - c[i])), 1) / sigma[i])
        K_tensor = tf.convert_to_tensor(K)
        K1 = tf.exp(-tf.transpose(K_tensor))
    
        # 计算输出层output
        output = tf.matmul(K1, W)
    
        # 四舍五入得到预测标签
        pred = tf.round(output)
        # pred = tf.argmax(output,1)
    
        # 定义损失函数loss（使用softmax交叉熵方法）
        # loss = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=output, labels=Y))
        loss = tf.reduce_sum(tf.abs(tf.subtract(output, Y)))
    
        # 选择优化方法
        # optimization = tf.train.AdamOptimizer(learning_rate).minimize(loss)
        optimization = tf.train.GradientDescentOptimizer(learning_rate).minimize(loss)
    
        with tf.Session() as sess:
            sess.run(tf.global_variables_initializer())
            for i in range(1, 1 + iteration_times):
                start = (i * batch_size) % dataX_train.shape[0]
                end = min(start + batch_size, dataX_train.shape[0])
                batch_x = dataX_train[start:end]
                batch_y = dataY_train_origin[start:end]
                # print(K1.eval(feed_dict={X:batch_x}))
                # print(b.eval())
                sess.run(optimization, feed_dict={ X: batch_x, Y: batch_y})
                if i == iteration_times-1:
                    print("loss of step {} is {}".format(i, loss.eval(feed_dict={ X: dataX_train, Y: dataY_train_origin})))
                    print("ACC is: ", ACC(np.array(pred.eval(feed_dict={ X: dataX_train})).T[0], dataY_train))

## 四、CNN ##

import tensorflow as tf
    import datadvi
    import numpy as np
    import math
    
    sess = tf.InteractiveSession()
    
    
    def weight_variable(shape):
        initial = tf.truncated_normal(shape, stddev=0.1)  # 标准差为0.1的正态分布
        return tf.Variable(initial)
    
    
    def bias_variable(shape):
        initial = tf.constant(0.1, shape=shape)  # 偏差初始化为0.1
        return tf.Variable(initial)
    
    
    def conv2d(x, W):
        return tf.nn.conv2d(x, W, strides=[1, 1, 1, 1], padding='SAME')
    
    
    def max_pool_2x2(x):
        return tf.nn.max_pool(x, ksize=[1, 2, 2, 1],
                              strides=[1, 2, 2, 1], padding='SAME')
    
    
    def cnn(filename, batch_size=50,learning_rate=0.0001,iteration_times=5000):
        dataX_train, dataX_predict, dataY_train, dataY_predict = datadvi.divdata(filename)
        num_label = len(np.unique(dataY_train))
    
        dataY_train_onehot = (np.arange(num_label) + 1 == dataY_train[:, None]).astype(np.float32)
        dataY_predict_onehot = (np.arange(num_label) + 1 == dataY_predict[:,None]).astype(np.float32)
    
        x = tf.placeholder(tf.float32, [None, len(dataX_train[0])])
        y_ = tf.placeholder(tf.float32, [None, num_label])
        # -1代表先不考虑输入的图片例子多少这个维度，1是channel的数量
        mat_XY = int(math.sqrt(len(dataX_train[0])))
    
        if mat_XY*mat_XY == len(dataX_train[0]):
            x_image = tf.reshape(x, [-1, mat_XY, mat_XY, 1]) #这适合数据维度能够开方得到整数的数据集, MNIST 和 COIL20等
        else:
            x_image = tf.reshape(x,[-1,50,100,1]) #这是gisette的数据参数大小设置
    
    
        keep_prob = tf.placeholder(tf.float32)
    
        # 构建卷积层1
        W_conv1 = weight_variable([5, 5, 1, 32])  # 卷积核5*5，1个channel，32个卷积核，形成32个featuremap
        b_conv1 = bias_variable([32])  # 32个featuremap的偏置
        h_conv1 = tf.nn.relu(conv2d(x_image, W_conv1) + b_conv1)  # 用relu非线性处理
        h_pool1 = max_pool_2x2(h_conv1)  # pooling池化
    
    
        # 构建卷积层2
        W_conv2 = weight_variable([5, 5, 32, 64])  # 注意这里channel值是32
        b_conv2 = bias_variable([64])
        h_conv2 = tf.nn.relu(conv2d(h_pool1, W_conv2) + b_conv2)
        h_pool2 = max_pool_2x2(h_conv2)
    
    
        # 构建全连接层1
        W_fc1 = weight_variable([ int(mat_XY/4* mat_XY/4 * 64), 1024])
        #W_fc1 = weight_variable([int(50*100/4*64),1024]) #这是gisette的数据参数大小设置
        b_fc1 = bias_variable([1024])
        h_pool3 = tf.reshape(h_pool2, [-1, int(mat_XY/4* mat_XY/4 * 64)])
        #h_pool3 = tf.reshape(h_pool2,[-1,int(50*100/4*64)]) #这是gisette的数据参数大小设置
        h_fc1 = tf.nn.relu(tf.matmul(h_pool3, W_fc1) + b_fc1)
        h_fc1_drop = tf.nn.dropout(h_fc1, keep_prob)
    
        # 构建全连接层2
        W_fc2 = weight_variable([1024, num_label])
        b_fc2 = bias_variable([num_label])
        y_conv = tf.nn.softmax(tf.matmul(h_fc1_drop, W_fc2) + b_fc2)
    
        cross_entropy = tf.reduce_mean(-tf.reduce_sum(y_ * tf.log(y_conv), reduction_indices=[1]))
        train_step = tf.train.AdamOptimizer(learning_rate).minimize(cross_entropy)
        correct_prediction = tf.equal(tf.arg_max(y_conv, 1), tf.arg_max(y_, 1))
        accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
    
        tf.global_variables_initializer().run()
    
        for i in range(iteration_times):
            start = (i * batch_size) % dataX_train.shape[0]
            end = min(start + batch_size, dataX_train.shape[0])
            batch_x = dataX_train[start:end]
            batch_y = dataY_train_onehot[start:end]
    
            # if i % 100 == 0:
            # train_accuracy = accuracy.eval(feed_dict={x: batch_x, y_: batch_y, keep_prob: 1.0})
            # print("step %d, training accuracy %g" % (i, train_accuracy))
            # print("loss is: ", cross_entropy.eval(feed_dict={x:batch_x,y_:batch_y,keep_prob:1.0}))
    
            train_step.run(feed_dict={ x: batch_x, y_: batch_y, keep_prob: 0.5})
        print("test accuracy %g" % accuracy.eval(feed_dict={ x: dataX_predict,
                                                            y_: dataY_predict_onehot, keep_prob: 1.0}))

## 五、结果 ##

### （1）acc结果对比 ###

<table> 
 <thead> 
  <tr> 
   <th>数据/方法</th> 
   <th>全连接</th> 
   <th>rbf</th> 
   <th>cnn</th> 
  </tr> 
 </thead> 
 <tbody> 
  <tr> 
   <td>minist</td> 
   <td>0.747</td> 
   <td>0.855</td> 
   <td>0.912</td> 
  </tr> 
  <tr> 
   <td>lung</td> 
   <td>0.905</td> 
   <td>0.932</td> 
   <td>0.950</td> 
  </tr> 
  <tr> 
   <td>yale</td> 
   <td>0.653</td> 
   <td>0.772</td> 
   <td>0.886</td> 
  </tr> 
 </tbody> 
</table>

### （2）结果分析 ###

*  通过实验可以发现由于lung数据集的数据量最大，标签类别只有5，因此每一类标签的训练数据集较大，因此结果较高，可以到百分之八十，yale数据量小，标签类别为15，因此每一类标签的训练数据集较小，结果也相对较差。可以通过优化模型，或者增加数据记得方式来提高精确度。
 *  这三种方法中，可以发现cnn的效果最好，rbf次之，全连接的效果最差，因此，在对图像进行分类时，推荐使用cnn神经网络模型

[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pcmFjbGVvYQ_size_16_color_FFFFFF_t_70_pic_center]: https://img-blog.csdnimg.cn/20200109125540619.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L21pcmFjbGVvYQ==,size_16,color_FFFFFF,t_70#pic_center