Keras实现 DenseNet

短命女 2023-07-17 04:58 15阅读 0赞

原文链接：https://www.jianshu.com/p/274d050d517e

> 参考自[https://github.com/titu1994/DenseNet/blob/master/densenet.py][https_github.com_titu1994_DenseNet_blob_master_densenet.py]

--------------------

先来一张图，便于理解网络结构，推荐的dense\_block一般是3。两个dense\_block之间的就是过渡层。每个dense\_block内部都使用密集连接。

![Image 1][]

--------------------

##### Conv\_block: #####

卷积操作，按照论文的说法，这里应该是一个组合函数，分别为：BatchNormalization、ReLU和3x3 Conv。

def conv_block(ip, nb_filter, bottleneck=False, dropout_rate=None, weight_decay=1e-4): ''' Apply BatchNorm, Relu, 3x3 Conv2D, optional bottleneck block and dropout Args: ip: Input keras tensor nb_filter: number of filters bottleneck: add bottleneck block dropout_rate: dropout rate weight_decay: weight decay factor Returns: keras tensor with batch_norm, relu and convolution2d added (optional bottleneck) ''' concat_axis = 1 if K.image_data_format() == 'channel_first' else -1 x = BatchNormalization(axis=concat_axis, epsilon=1.1e-5)(ip) x = Activation('relu')(x) if bottleneck: inter_channel = nb_filter * 4 x = Conv2D(inter_channel, (1, 1), kernel_initializer='he_normal', padding='same', use_bias=False, kernel_regularizer=l2(weight_decay))(x) x = BatchNormalization(axis=concat_axis, epsilon=1.1e-5)(x) x = Activation('relu')(x) x = Conv2D(nb_filter, (3, 3), kernel_initializer='he_normal', padding='same', use_bias=False)(x) if dropout_rate: x = Dropout(dropout_rate)(x) return x

其中的`concat_axis`表示特征轴，因为连接和BN都是对特征轴而言的。`bottleneck`表示是否使用瓶颈层，也就是使用1x1的卷继层将特征图的通道数进行压缩。

--------------------

##### Transition\_block: #####

过渡层，用来连接两个dense\_block。同时在最后一个dense\_block的尾部不需要使用过渡层。按照论文的说法，过渡层由四部分组成：BatchNormalization、ReLU、1x1Conv和2x2Maxpooling。

def transition_block(ip, nb_filter, compression=1.0, weight_decay=1e-4): '''Apply BatchNorm, ReLU, Conv2d, optional compressoin, dropout and Maxpooling2D Args: ip: keras tensor nb_filter: number of filters compression: caculated as 1 - reduction. Reduces the number of features maps in the transition block dropout_rate: dropout rate weight_decay: weight decay factor Returns: keras tensor, after applying batch_norm, relu-conv, dropout, maxpool ''' concat_axis = 1 if K.image_data_format() == 'channels_first' else -1 x = BatchNormalization(axis=concat_axis, epsilon=1.1e-5)(ip) x = Activation('relu')(x) x = Conv2D(int(nb_filter * compression), (1, 1), kernel_initializer='he_normal', padding='same', use_bias=False, kernel_regularizer=l2(weight_decay))(x) x = AveragePooling2D((2, 2), strides=(2, 2))(x) return x

其中的Conv2D操作实现了1x1的卷积操作，同时使用了compression\_rate，也就是论文中说的压缩率，将通道数进行调整。

--------------------

##### Dense\_block: #####

此处使用循环实现了dense\_block的密集连接。

def dense_block(x, nb_layers, nb_filter, growth_rate, bottleneck=False, dropout_rate=None, weight_decay=1e-4, grow_nb_filters=True, return_concat_list=False): '''Build a dense_block where the output of ench conv_block is fed t subsequent ones Args: x: keras tensor nb_layser: the number of layers of conv_block to append to the model nb_filter: number of filters growth_rate: growth rate bottleneck: bottleneck block dropout_rate: dropout rate weight_decay: weight decay factor grow_nb_filters: flag to decide to allow number of filters to grow return_concat_list: return the list of feature maps along with the actual output Returns: keras tensor with nb_layers of conv_block appened ''' concat_axis = 1 if K.image_data_format() == 'channels_first' else -1 x_list = [x] for i in range(nb_layers): cb = conv_block(x, growth_rate, bottleneck, dropout_rate, weight_decay) x_list.append(cb) x = concatenate([x, cb], axis=concat_axis) if grow_nb_filters: nb_filter += growth_rate if return_concat_list: return x, nb_filter, x_list else: return x, nb_filter

其中的x = concatenate(\[x, cb\], axis=concat\_axis)操作使得x在每次循环中始终维护一个全局状态，第一次循环输入为x，输出为cb1，第二的输入为cb=\[x, cb1\]，输出为cb2，第三次的输入为cb=\[x, cb1, cb2\]，输出为cb3，以此类推。增长率growth\_rate其实就是每次卷积时使用的卷积核个数，也就是最后输出的通道数。

--------------------

##### Create\_dense\_net: #####

构建网络模型：

def create_dense_net(nb_classes, img_input, include_top, depth=40, nb_dense_block=3, growth_rate=12, nb_filter=-1, nb_layers_per_block=[1], bottleneck=False, reduction=0.0, dropout_rate=None, weight_decay=1e-4, subsample_initial_block=False, activation='softmax'): ''' Build the DenseNet model Args: nb_classes: number of classes img_input: tuple of shape (channels, rows, columns) or (rows, columns, channels) include_top: flag to include the final Dense layer depth: number or layers nb_dense_block: number of dense blocks to add to end (generally = 3) growth_rate: number of filters to add per dense block nb_filter: initial number of filters. Default -1 indicates initial number of filters is 2 * growth_rate nb_layers_per_block: list, number of layers in each dense block bottleneck: add bottleneck blocks reduction: reduction factor of transition blocks. Note : reduction value is inverted to compute compression dropout_rate: dropout rate weight_decay: weight decay rate subsample_initial_block: Set to True to subsample the initial convolution and add a MaxPool2D before the dense blocks are added. subsample_initial: activation: Type of activation at the top layer. Can be one of 'softmax' or 'sigmoid'. Note that if sigmoid is used, classes must be 1. Returns: keras tensor with nb_layers of conv_block appended ''' concat_axis = 1 if K.image_data_format() == 'channel_first' else -1 if type(nb_layers_per_block) is not list: print('nb_layers_per_block should be a list!!!') return 0 final_nb_layer = nb_layers_per_block[-1] nb_layers = nb_layers_per_block[:-1] if nb_filter <= 0: nb_filter = 2 * growth_rate compression = 1.0 - reduction if subsample_initial_block: initial_kernel = (7, 7) initial_strides = (2, 2) else: initial_kernel = (3, 3) initial_strides = (1, 1) x = Conv2D(nb_filter, initial_kernel, kernel_initializer='he_normal', padding='same', strides=initial_strides, use_bias=False, kernel_regularizer=l2(weight_decay))(img_input) if subsample_initial_block: x = BatchNormalization(axis=concat_axis, epsilon=1.1e-5)(x) x = Activation('relu')(x) x = MaxPooling2D((3, 3), strides=(2, 2), padding='same')(x) for block_index in range(nb_dense_block - 1): x, nb_filter = dense_block(x, nb_layers[block_index], nb_filter, growth_rate, bottleneck=bottleneck, dropout_rate=dropout_rate, weight_decay=weight_decay) x = transition_block(x, nb_filter, compression=compression, weight_decay=weight_decay) nb_filter = int(nb_filter * compression) # 最后一个block没有transition_block x, nb_filter = dense_block(x, final_nb_layer, nb_filter, growth_rate, bottleneck=bottleneck, dropout_rate=dropout_rate, weight_decay=weight_decay) x = BatchNormalization(axis=concat_axis, epsilon=1.1e-5)(x) x = Activation('relu')(x) x = GlobalAveragePooling2D()(x) if include_top: x = Dense(nb_classes, activation=activation)(x) return x

##### 生成Model： #####

inputs = Input(tensor=img_input, shape=input_shape) x = create_dense_net(classes=1000, img_input, include_top=True, depth=169, nb_dense_block=4, growth_rate=32, nb_filter=64, nb_layers_per_block=[6, 12, 32, 32], bottleneck=True, reduction=0.5, dropout_rate=0.0, weight_decay=1e-4, subsample_initial_blockTrue, activation='softmax') model = Model(inputs, x, name='densenet169')

--------------------

[https_github.com_titu1994_DenseNet_blob_master_densenet.py]: https://link.jianshu.com?t=https%3A%2F%2Fgithub.com%2Ftitu1994%2FDenseNet%2Fblob%2Fmaster%2Fdensenet.py
[Image 1]: