一起学习用Verilog在FPGA上实现CNN----(三)激活层设计

秒速五厘米 2023-09-23 18:45 45阅读 0赞

# 1 激活层设计 #

LeNet-5网络的激活函数是双曲正切函数(TanH)，项目中tanh函数模块由完整的层UsingTheTanh构成，该层由较小的处理单元HyperBolicTangent组成

## 1.1 HyperBolicTangent ##

处理单元HyperBolicTangent，对每个输入执行Tanh操作，原理图如图所示，输入为位宽16的数，输出位宽也是16。该单元将Tanh运算分为3个乘法操作和1个加法操作：

*  首先，得到x项的增量项，即x^2
 *  然后，将当前x项与下一项相乘
 *  然后，将每个相应的最终x项与其系数相乘
 *  最后，将每个结果项与前一项相加

![在这里插入图片描述][02bb6bfbea354d63992f44d27755ea58.png_pic_center]

## 1.2 UsingTheTanh ##

UsingTheTanh是Tanh层，由HyperBolicTangent单元组成，原理图如图所示。Tanh层的输入数据位宽为75264，即为卷积层的输出数据

![在这里插入图片描述][c8c83b70b35646e986b61ccddec8ad18.png_pic_center]

# 2 代码实现 #

[一起学习用Verilog在FPGA上实现CNN----(二)卷积层设计][Verilog_FPGA_CNN----]已经完成卷积层的设计，下面我们继续激活层的代码实现

## 2.1 HyperBolicTangent16 ##

### 2.1.1 设计输入 ###

创建HyperBolicTangent16文件，操作如图：

![在这里插入图片描述][ba024527a6e44d88a996b94613802265.png_pic_center]

输入文件名：

![在这里插入图片描述][26e4af44fbf44dddbb884ef313ee8955.png_pic_center]

确认创建：

![在这里插入图片描述][f7b17f75830440eda975e1ca4c004ab8.png_pic_center]  
双击打开，输入如下代码：

module HyperBolicTangent16 (x,reset,clk,OutputFinal,Finished);
    parameter DATA_WIDTH=16;
    localparam taylor_iter=4;//I chose 4 Taylor Coefficients to undergo my tanh operation
    input signed [DATA_WIDTH-1:0] x;
    
    input clk;
    input reset;
    output reg Finished;
    output reg[DATA_WIDTH-1:0]  OutputFinal;
    reg [DATA_WIDTH*taylor_iter-1:0] Coefficients ; //-17/315 2/15 -1/3 1
    wire [DATA_WIDTH-1:0] Xsquared; //To always generate a squared version of the input to increment the power by 2 always.
    reg [DATA_WIDTH-1:0] ForXSqOrOne; //For Multiplying The power of X(1 or X^2)
    reg [DATA_WIDTH-1:0] ForMultPrevious; //output of the first multiplication which is either with 1 or x(X or Output1)
    wire [DATA_WIDTH-1:0] OutputOne; //the output of Mulitplying the X term with its corresponding power coeff.
    wire [DATA_WIDTH-1:0] OutOfCoeffMult; //the output of Mulitplying the X term with its corresponding power coeff.
    reg  [DATA_WIDTH-1:0] OutputAdditionInAlways;
    wire [DATA_WIDTH-1:0] OutputAddition; //the output of the Addition each cycle 
    
    floatMult16 MSquaring (x,x,Xsquared);//Generating x^2   得到x项的增量项，即x^2
    floatMult16 MGeneratingXterm (ForXSqOrOne,ForMultPrevious,OutputOne); //Generating the X term [x,x^3,x^5,...]  将当前x项与下一项相乘
    floatMult16 MTheCoefficientTerm (OutputOne,Coefficients[DATA_WIDTH-1:0],OutOfCoeffMult); //Multiplying the X term by its corresponding coeff. 将每个相应的最终x项与其系数相乘
    floatAdd16 FADD1 (OutOfCoeffMult,OutputAdditionInAlways,OutputAddition); //Adding the new term to the previous one     ex: x-1/3*(x^3) 将每个结果项与前一项相加
    reg [DATA_WIDTH-1:0] AbsFloat; //To generate an absolute value of the input[For Checking the convergence]
    
    always @ (posedge clk) begin
    AbsFloat=x;//Here i hold the input then i make it positive whatever its sign to be able to compare to implement the rule |x|>pi/2   which is the convergence rule
    AbsFloat[15]=0;
    if(AbsFloat>16'sb0011111001001000)begin 
      //The Finished bit is for letting the bigger module know that the tanh is finished
      if (x[15]==0)begin 
        OutputFinal= 16'b0011110000000000;Finished =1'b 1;//here i assign it an immediate value of Positive Floating one
      end 
        if (x[15]==1)begin 
        OutputFinal= 16'b1011110000000000;Finished =1'b 1;//here i assign it an immediate value of Negative Floating one
        end
    end
    //here i handle the case of it equals +- pi/2    so i got the exact value and handle it also immediately
    else if (AbsFloat==16'sb0011111001001000)
      begin 
        if (x[15]==0)begin 
      OutputFinal=16'b0011110000000000;Finished=1'b 1;
      end
      else begin 
      OutputFinal=16'b1011110000000000;Finished=1'b 1;
      end
      end
    else begin 
      //First instance of the tanh
    if(reset==1'b1)begin  
    	Coefficients=64'b1010101011101000_0011000001000100_1011010101010101_0011110000000000;//the 4 coefficients of taylor expansion
    	ForXSqOrOne=16'b0011110000000000; //initially 1
    	OutputAdditionInAlways=16'b0000000000000000; //initially 0
    	ForMultPrevious=x;
    Finished=0;
    end
    else begin
     	ForXSqOrOne=Xsquared;
    	ForMultPrevious=OutputOne; //get the output of the second multiplication to multiply with x
    	Coefficients=Coefficients>>DATA_WIDTH; //shift 32 bit to divide the out_m1 with the new number to compute the factorial
      OutputAdditionInAlways=OutputAddition;
      Finished=0;
    end
    // the end of the tanh
    if(Coefficients==64'b0000000000000000_0000000000000000_0000000000000000_0000000000000000)begin 
    	OutputFinal=OutputAddition;
    	Finished =1'b 1;
    end
    end 
    end
    endmodule

如图所示：

![在这里插入图片描述][3ee33d68e1ca4c03b79a4c88c7ebe79c.png_pic_center]

### 2.1.2 分析与综合 ###

将HyperBolicTangent16设置为顶层：

![在这里插入图片描述][eebdce110cab48918e6222e1c6fdb893.png_pic_center]

对设计进行分析，操作如图：

![在这里插入图片描述][33f6beb787c74e408ec4a16510e9fb9b.png_pic_center]

分析后的设计，Vivado自动生成原理图，如图：

![在这里插入图片描述][494aa783a14e4890b475858c54e15358.png_pic_center]

对设计进行综合，操作如图：

![在这里插入图片描述][be81d59119734b09a3c628f5c03c54b5.png_pic_center]

综合完成，关闭即可：

![在这里插入图片描述][f67ccebeb5e544978f0db4e6b7e33713.png_pic_center]

### 2.1.3 功能仿真 ###

创建TestBench，操作如图所示：

![在这里插入图片描述][2e4ffbad3c7b4e088dde09c2d35f78d9.png_pic_center]

输入激励文件名：

![在这里插入图片描述][d65c3212dee944f8996a9848d616df61.png_pic_center]

双击打开，输入激励代码：

`timescale 1ns / 1ps
    module tb_HyperBolicTangent16();
    reg clk, reset;
    reg [15:0]x;
    wire [15:0]OutputFinal;
    wire Finished;
    
    localparam PERIOD = 100;
    
    always
    	#(PERIOD/2) clk = ~clk;
    
    initial begin
    	#0 //starting the tanh 
    	clk = 1'b1;
    	reset = 1'b1;
    	// trying a random input(0.600000023842)     where tanh(0.600000023842)=0.53704958
    	x=16'b0011100011001101;
    
    	#(PERIOD/2)
    	reset = 0;	
    	
    	#400
    	// waiting for 4 clock cycles then checking the output with approx.(0.53685)
    	if(OutputFinal==32'b00111111000010010110111101111011 && Finished==1'b1)begin 
    	 $display("Result is Right [Not in convergence region]");	  
    	  end
    	  else begin 
    	 $display("Result is Wrong");	 
    	    end	 
    	    //trying another input which will be in the convergence region(3)  the output will be converged to 1    and reseting the function again
    	    x=16'b0100001000000000;
    	    
    		reset = 1'b1;
    		#(PERIOD/2)
    		reset=1'b0;
    		#200
    		// checking if the output is 1
    		if(OutputFinal==32'b00111111100000000000000000000000)begin 
    	 $display("Result is Right [ convergence region]");	  
    	  end
    	  else begin 
    	 $display("Result is Wrong ");	 
    	    end	
    	$stop;
    end
    
    HyperBolicTangent16 UUT
    (
      .x(x),
    	.clk(clk),
    	.reset(reset),
    	.OutputFinal(OutputFinal),
    	.Finished(Finished)	
    );
    endmodule

如图所示：

![在这里插入图片描述][8ce0477d20a14340bf2f8050f4a3ee1d.png_pic_center]

将tb\_HyperBolicTangent16设置为顶层：

![在这里插入图片描述][ef602b4c49524d98b77c12e08934aa1b.png_pic_center]  
开始进行仿真，操作如下:

![在这里插入图片描述][acf6a5b544fe4355a6725809d13a5e87.png_pic_center]

开始仿真，如图：

![在这里插入图片描述][b220784cde764d23a942d5d15248c007.png_pic_center]

仿真波形，如图：

![在这里插入图片描述][a3738a7658b441e2b3ae46031533bf41.png_pic_center]

关闭仿真：

![在这里插入图片描述][fbf1ad897de849c596f9bb2aaffa3e29.png_pic_center]

## 2.2 UsingTheTanh16 ##

### 2.2.1 设计输入 ###

创建UsingTheTanh16文件，如图：

![在这里插入图片描述][710b88e0eabf4df5ad12db6b7a802a57.png_pic_center]

双击打开，输入代码：

module UsingTheTanh16(x,clk,Output,resetExternal,FinishedTanh);
    parameter DATA_WIDTH=16;
    parameter nofinputs=7;// deterimining the no of inputs entering the function
    input resetExternal;// controlling this layer
    input  signed [nofinputs*DATA_WIDTH-1:0] x;
    input clk;
    output reg FinishedTanh;
    reg reset;// for the inner tanh
    output reg [nofinputs*DATA_WIDTH-1:0]Output;
    wire [DATA_WIDTH-1:0]OutputTemp;
    reg [7:0]counter=0;
    wire Finished;
    reg [7:0]i;
    // the inner tanh taking inputs in 32 bits and then increment using the i operator
    HyperBolicTangent16 TanhArray (x[DATA_WIDTH*i+:DATA_WIDTH],reset,clk,OutputTemp,Finished);
     
     
    always@(posedge clk)
    begin 
      counter=counter+1;
    // if the external reset =1 then make everything to 0
    if(resetExternal==1) begin reset=1;i=0;FinishedTanh=0; end
    //checking if the tanh is not finished so continue your operation and low down the reset to continue
      else if(FinishedTanh==0) begin 
        if(reset==1)begin reset=0; end 
        // if it is finished then store the output of the tanh and increment the input forward
          else if (Finished==1)begin Output[DATA_WIDTH*i+:DATA_WIDTH]=OutputTemp;reset=1;i=i+1;end
    // check if all the inputs are finished then the layer is OK
    if(i==nofinputs)
      begin FinishedTanh=1;end
    end 
    
    end
    endmodule

如图所示：

![在这里插入图片描述][ecefa5ec8ae9470487a38c876e180c26.png_pic_center]

### 2.2.2 分析与综合 ###

将UsingTheTanh16设置为顶层：

![在这里插入图片描述][e62a0f8d60994ee3871b59c8819106ca.png_pic_center]

关闭上次的分析文件：

![在这里插入图片描述][f468b8c2a2d64b4398380c34a9299ca3.png_pic_center]

对设计进行分析，操作如图：

![在这里插入图片描述][8c3296f529cf4978baf9bdd0ccd25c86.png_pic_center]  
分析后的设计，Vivado自动生成原理图，如图：

![在这里插入图片描述][76cff09725494f1f900fb272bf250f85.png_pic_center]  
对设计进行综合，操作如图：

![在这里插入图片描述][8d5168cebeeb4b74a21ebcb4ff351dcd.png_pic_center]

### 2.2.3 功能仿真 ###

创建TestBench，操作如图所示：

![在这里插入图片描述][28e56a75622a40ef9b93e8d122a4a5fc.png_pic_center]

双击打开，输入激励代码：

module tb_UsingTheTanh16();
    reg clk, resetExternal;
    reg [63:0]x;
    wire [63:0]Output;
    wire FinishedTanh;
    
    localparam PERIOD = 100;
    
    always
    	#(PERIOD/2) clk = ~clk;
    
    initial begin
    	#0 //starting the tanh 
    	clk = 1'b1;
    	resetExternal = 1'b1;
    	// trying a random input(0.6,3)     where tanh(0.600000023842)=0.53704958, tanh(3)~=1
    	x=64'b00111111000110011001100110011010_01000000010000000000000000000000;
    
    	#(PERIOD/2)
    	resetExternal = 1'b0;	
    	
    	#800
    	// waiting for The final output which will be (0.57,1)
    	if(Output==64'b00111111000010010110111101111011_00111111100000000000000000000000 && FinishedTanh==1'b1)begin 
    	 $display("Result is right (for the full array of inputs)");	  
    	  end
    	  else begin 
    	 $display("Result is Wrong");	 
    	    end	 
    	$stop;
    end
    
    UsingTheTanh16 UUT
    (
      .x(x),
    	.clk(clk),
    	.Output(Output),
    	.resetExternal(resetExternal),	
    	.FinishedTanh(FinishedTanh)	
    );
    
    endmodule

如图所示：

![在这里插入图片描述][52f99eee4a374ab8adbea072b4a7d43c.png_pic_center]

将tb\_UsingTheTanh16文件设置为顶层：

![在这里插入图片描述][97ebd526ef1947148d19baa44b011e4f.png_pic_center]  
开始进行仿真，操作如下:

![在这里插入图片描述][d98e0ee4cdc248699b9fbf79ed1c826c.png_pic_center]

开始仿真，如图：

![在这里插入图片描述][2cd96482d7fb40808be4c247bbaea5e7.png_pic_center]

仿真波形，如图：

![在这里插入图片描述][c571545faa5149a6976a7af2dfce6abd.png_pic_center]

## 2.3 IntegrationConv ##

### 2.3.1 设计输入 ###

双击打开integrationConv文件，修改代码为：

module integrationConv (clk,reset,CNNinput,Conv,ConvOutput);
    
    parameter DATA_WIDTH = 16;
    parameter ImgInW = 32;
    parameter ImgInH = 32;
    parameter ConvOut = 28;
    parameter Kernel = 5;
    parameter DepthC = 6;
    
    
    input clk, reset;
    input [ImgInW*ImgInH*DATA_WIDTH-1:0] CNNinput;
    input [Kernel*Kernel*DepthC*DATA_WIDTH-1:0] Conv;
    output [ConvOut*ConvOut*DepthC*DATA_WIDTH-1:0] ConvOutput;
    
    reg TanhReset;
    wire TanhFlag;
    
    wire [ConvOut*ConvOut*DepthC*DATA_WIDTH-1:0] Cout;
    wire [ConvOut*ConvOut*DepthC*DATA_WIDTH-1:0] CoutTanH;
    
    convLayerMulti C1
    (
    	.clk(clk),
    	.reset(reset),
    	.image(CNNinput),
    	.filters(Conv),
    	.outputConv(ConvOutput)
    );
    UsingTheTanh16
    #(.nofinputs(ConvOut*ConvOut*DepthC))
    Tanh1(
          .x(Cout),
          .clk(clk),
          .Output(CoutTanH),
          .resetExternal(TanhReset),
          .FinishedTanh(TanhFlag)
          );
    endmodule

如图所示：

![在这里插入图片描述][d6b372b42a3f45e1a61542b7256c60f3.png_pic_center]

### 2.3.2 分析与综合 ###

将integrationConv设置为顶层：

![在这里插入图片描述][ce67f57a9c9241749d4b6956bde86944.png_pic_center]

关闭上次的分析文件：

![在这里插入图片描述][879dcefd18794fb194a3e451510dbbf2.png_pic_center]

对设计进行分析，操作如图：

![在这里插入图片描述][3c423fe6d6db4557963368397ca64fd1.png_pic_center]

分析后的设计，Vivado自动生成原理图，如图：

![在这里插入图片描述][d990c2990ae94c39b639519632d6d283.png_pic_center]

对设计进行综合，操作如图：

![在这里插入图片描述][bab3f7306cef4d0abbeba44c99d20706.png_pic_center]

`纪念一下，2022卡塔尔世界杯，阿根廷捧起大力神杯，梅西圆梦`

![在这里插入图片描述][f52a49e7b3ed4c70b2f7aa112dc1bbac.jpeg_pic_center]

**希望本文对大家有帮助，上文若有不妥之处，欢迎指正**

**分享决定高度，学习拉开差距**

[02bb6bfbea354d63992f44d27755ea58.png_pic_center]: https://img-blog.csdnimg.cn/02bb6bfbea354d63992f44d27755ea58.png#pic_center
[c8c83b70b35646e986b61ccddec8ad18.png_pic_center]: https://img-blog.csdnimg.cn/c8c83b70b35646e986b61ccddec8ad18.png#pic_center
[Verilog_FPGA_CNN----]: https://blog.csdn.net/qq_42078934/article/details/128258185
[ba024527a6e44d88a996b94613802265.png_pic_center]: https://img-blog.csdnimg.cn/ba024527a6e44d88a996b94613802265.png#pic_center
[26e4af44fbf44dddbb884ef313ee8955.png_pic_center]: https://img-blog.csdnimg.cn/26e4af44fbf44dddbb884ef313ee8955.png#pic_center
[f7b17f75830440eda975e1ca4c004ab8.png_pic_center]: https://img-blog.csdnimg.cn/f7b17f75830440eda975e1ca4c004ab8.png#pic_center
[3ee33d68e1ca4c03b79a4c88c7ebe79c.png_pic_center]: https://img-blog.csdnimg.cn/3ee33d68e1ca4c03b79a4c88c7ebe79c.png#pic_center
[eebdce110cab48918e6222e1c6fdb893.png_pic_center]: https://img-blog.csdnimg.cn/eebdce110cab48918e6222e1c6fdb893.png#pic_center
[33f6beb787c74e408ec4a16510e9fb9b.png_pic_center]: https://img-blog.csdnimg.cn/33f6beb787c74e408ec4a16510e9fb9b.png#pic_center
[494aa783a14e4890b475858c54e15358.png_pic_center]: https://img-blog.csdnimg.cn/494aa783a14e4890b475858c54e15358.png#pic_center
[be81d59119734b09a3c628f5c03c54b5.png_pic_center]: https://img-blog.csdnimg.cn/be81d59119734b09a3c628f5c03c54b5.png#pic_center
[f67ccebeb5e544978f0db4e6b7e33713.png_pic_center]: https://img-blog.csdnimg.cn/f67ccebeb5e544978f0db4e6b7e33713.png#pic_center
[2e4ffbad3c7b4e088dde09c2d35f78d9.png_pic_center]: https://img-blog.csdnimg.cn/2e4ffbad3c7b4e088dde09c2d35f78d9.png#pic_center
[d65c3212dee944f8996a9848d616df61.png_pic_center]: https://img-blog.csdnimg.cn/d65c3212dee944f8996a9848d616df61.png#pic_center
[8ce0477d20a14340bf2f8050f4a3ee1d.png_pic_center]: https://img-blog.csdnimg.cn/8ce0477d20a14340bf2f8050f4a3ee1d.png#pic_center
[ef602b4c49524d98b77c12e08934aa1b.png_pic_center]: https://img-blog.csdnimg.cn/ef602b4c49524d98b77c12e08934aa1b.png#pic_center
[acf6a5b544fe4355a6725809d13a5e87.png_pic_center]: https://img-blog.csdnimg.cn/acf6a5b544fe4355a6725809d13a5e87.png#pic_center
[b220784cde764d23a942d5d15248c007.png_pic_center]: https://img-blog.csdnimg.cn/b220784cde764d23a942d5d15248c007.png#pic_center
[a3738a7658b441e2b3ae46031533bf41.png_pic_center]: https://img-blog.csdnimg.cn/a3738a7658b441e2b3ae46031533bf41.png#pic_center
[fbf1ad897de849c596f9bb2aaffa3e29.png_pic_center]: https://img-blog.csdnimg.cn/fbf1ad897de849c596f9bb2aaffa3e29.png#pic_center
[710b88e0eabf4df5ad12db6b7a802a57.png_pic_center]: https://img-blog.csdnimg.cn/710b88e0eabf4df5ad12db6b7a802a57.png#pic_center
[ecefa5ec8ae9470487a38c876e180c26.png_pic_center]: https://img-blog.csdnimg.cn/ecefa5ec8ae9470487a38c876e180c26.png#pic_center
[e62a0f8d60994ee3871b59c8819106ca.png_pic_center]: https://img-blog.csdnimg.cn/e62a0f8d60994ee3871b59c8819106ca.png#pic_center
[f468b8c2a2d64b4398380c34a9299ca3.png_pic_center]: https://img-blog.csdnimg.cn/f468b8c2a2d64b4398380c34a9299ca3.png#pic_center
[8c3296f529cf4978baf9bdd0ccd25c86.png_pic_center]: https://img-blog.csdnimg.cn/8c3296f529cf4978baf9bdd0ccd25c86.png#pic_center
[76cff09725494f1f900fb272bf250f85.png_pic_center]: https://img-blog.csdnimg.cn/76cff09725494f1f900fb272bf250f85.png#pic_center
[8d5168cebeeb4b74a21ebcb4ff351dcd.png_pic_center]: https://img-blog.csdnimg.cn/8d5168cebeeb4b74a21ebcb4ff351dcd.png#pic_center
[28e56a75622a40ef9b93e8d122a4a5fc.png_pic_center]: https://img-blog.csdnimg.cn/28e56a75622a40ef9b93e8d122a4a5fc.png#pic_center
[52f99eee4a374ab8adbea072b4a7d43c.png_pic_center]: https://img-blog.csdnimg.cn/52f99eee4a374ab8adbea072b4a7d43c.png#pic_center
[97ebd526ef1947148d19baa44b011e4f.png_pic_center]: https://img-blog.csdnimg.cn/97ebd526ef1947148d19baa44b011e4f.png#pic_center
[d98e0ee4cdc248699b9fbf79ed1c826c.png_pic_center]: https://img-blog.csdnimg.cn/d98e0ee4cdc248699b9fbf79ed1c826c.png#pic_center
[2cd96482d7fb40808be4c247bbaea5e7.png_pic_center]: https://img-blog.csdnimg.cn/2cd96482d7fb40808be4c247bbaea5e7.png#pic_center
[c571545faa5149a6976a7af2dfce6abd.png_pic_center]: https://img-blog.csdnimg.cn/c571545faa5149a6976a7af2dfce6abd.png#pic_center
[d6b372b42a3f45e1a61542b7256c60f3.png_pic_center]: https://img-blog.csdnimg.cn/d6b372b42a3f45e1a61542b7256c60f3.png#pic_center
[ce67f57a9c9241749d4b6956bde86944.png_pic_center]: https://img-blog.csdnimg.cn/ce67f57a9c9241749d4b6956bde86944.png#pic_center
[879dcefd18794fb194a3e451510dbbf2.png_pic_center]: https://img-blog.csdnimg.cn/879dcefd18794fb194a3e451510dbbf2.png#pic_center
[3c423fe6d6db4557963368397ca64fd1.png_pic_center]: https://img-blog.csdnimg.cn/3c423fe6d6db4557963368397ca64fd1.png#pic_center
[d990c2990ae94c39b639519632d6d283.png_pic_center]: https://img-blog.csdnimg.cn/d990c2990ae94c39b639519632d6d283.png#pic_center
[bab3f7306cef4d0abbeba44c99d20706.png_pic_center]: https://img-blog.csdnimg.cn/bab3f7306cef4d0abbeba44c99d20706.png#pic_center
[f52a49e7b3ed4c70b2f7aa112dc1bbac.jpeg_pic_center]: https://img-blog.csdnimg.cn/f52a49e7b3ed4c70b2f7aa112dc1bbac.jpeg#pic_center