Tensorflow：tensor数据类型转换、计算和变换

Myth丶恋晨 2022-06-12 13:51 1661阅读 0赞

# 数据类型及转换 #

### **tensorflow中的数据类型列表** ###

数据类型    Python 类型    描述  
DT\_FLOAT    tf.float32    32 位浮点数.  
DT\_DOUBLE    tf.float64    64 位浮点数.  
DT\_INT64    tf.int64    64 位有符号整型.  
DT\_INT32    tf.int32    32 位有符号整型.  
DT\_INT16    tf.int16    16 位有符号整型.  
DT\_INT8    tf.int8    8 位有符号整型.  
DT\_UINT8    tf.uint8    8 位无符号整型.  
DT\_STRING    tf.string    可变长度的字节数组.每一个张量元素都是一个字节数组.  
DT\_BOOL    tf.bool    布尔型.  
DT\_COMPLEX64    tf.complex64    由两个32位浮点数组成的复数:实数和虚数.  
DT\_QINT32    tf.qint32    用于量化Ops的32位有符号整型.  
DT\_QINT8    tf.qint8    用于量化Ops的8位有符号整型.  
DT\_QUINT8    tf.quint8    用于量化Ops的8位无符号整型.

### **转换函数** ###

tf.string\_to\_number(string\_tensor, out\_type=None, name=None)  
tf.to\_double(x, name='ToDouble')  
tf.[to\_float][to_float](x, name='ToFloat')  
tf.to\_bfloat16(x, name='ToBFloat16')  
tf.to\_int32(x, name='ToInt32')  
tf.to\_int64(x, name='ToInt64')  
tf.cast(x, dtype, name=None)

\[[TensorFlow 数据类型转换][TensorFlow]\]

**cast(x, dtype, name=None)**  
示例：int32转换为float32：

t1 = tf.Variable(\[1,2,3,4,5\])  
t2 = tf.cast(t1,dtype=tf.float32)

\[[tf.cast()数据类型转换][tf.cast]\]

### tf.string类型 ###

tf.string 张量可以保存不同长度的字节串，因为字节串被视为原子单位。字符串长度不包括在张量尺寸中。即不管用py2还是py3，tf中的str都是bytes类型，有编码的。

编码转换参考\[[TensorFlow支持Unicode，中文NLP终于省心了][TensorFlow_Unicode_NLP]\]

不过建议还是直接用bytes，不要变成unicode了，在tf内部还是会转成bytes。

# 数学计算 #

### **矩阵点乘** ###

tf.multiply(a, b)   a\*b

multiply这个函数实现的是元素级别的相乘，也就是两个相乘的数元素各自相乘，这个时候要求两个矩阵必须同样大小。

### **矩阵乘法** ###

tf.matmul(a, b)

tf.linalg.[matmul][] The inputs must, following any transpositions, be tensors of rank >= 2 where the inner 2 dimensions specify valid matrix multiplication arguments, and any further outer dimensions match. 一个跟张量a和张量b类型一样的张量且最内部矩阵是a和b中的相应矩阵的乘积。 三维矩阵乘法应该是二维一 一对应矩阵乘。如\[32, ?, ?\]矩阵乘，相当于32个不同的二维矩阵一一对应矩阵乘。

### 二维和三维tensor矩阵乘法 ###

使用map\_fn

squares = map_fn(lambda x: x * x, elems)

lstm\_outputs = tf.map\_fn(lambda o: tf.matmul(o, weights), lstm\_outputs)

### 矩阵运算的简便方法einsum ###

tf.[einsum][](  
    equation,  
    \*inputs,  
    \*\*kwargs  
)

Many common operations can be expressed in this way. For example:

\# Matrix multiplication  
>>> einsum('ij,jk->ik', m0, m1)  \# output\[i,k\] = sum\_j m0\[i,j\] \* m1\[j, k\]

\# Dot product  
>>> einsum('i,i->', u, v)  \# output = sum\_i u\[i\]\*v\[i\]

\# Outer product  
>>> einsum('i,j->ij', u, v)  \# output\[i,j\] = u\[i\]\*v\[j\]

\# Transpose  
>>> einsum('ij->ji', m)  \# output\[j,i\] = m\[i,j\]

\# Trace  
>>> einsum('ii', m)  \# output\[j,i\] = trace(m) = sum\_i m\[i, i\]

\# Batch matrix multiplication  
>>> einsum('aij,ajk->aik', s, t)  \# out\[a,i,k\] = sum\_j s\[a,i,j\] \* t\[a, j, k\]

\# Batch(soft\_align\_attention) Note:lz  
>>> einsum('abd,acd->abc', s, t)  \# out\[a,b,c\] = s\[a,b,d\] \* t\[a, d, c\]

# x1: [b, s1, d]
    # x2: [b, s2, d]
    # att: [b, s1, s2]
    att = tf.einsum("abd,acd->abc", x1, x2)

**示例**

\# 初始化矩阵  
x = tf.constant(\[\[1., 2., 3.\],  
                 \[1., 2., 3.\]\])  
y = tf.constant(\[\[2., 3., 4.\],  
                 \[2., 3., 4.\]\])  
z = tf.constant(\[\[3., 4.\],  
                 \[3., 4.\],  
                 \[3., 4.\]\])  
\# 外积  
out = tf.multiply(x, y)  
out1 = tf.einsum('ij,ij->ij', x, y)  
\# 点积  
dot = tf.matmul(x, z)  
dot1 = tf.einsum('ij,jk->ik', x, z)  
\# 转置  
trans = tf.transpose(x, \[1, 0\])  
trans1 = tf.einsum('ij->ji', x)  
with tf.Session() as sess:  
    print('外积')  
    print('out\\n\{\}'.format(sess.run(out)))  
    print('out1\\n\{\}'.format(sess.run(out1)))  
    print('点积')  
    print('dot\\n\{\}'.format(sess.run(dot)))  
    print('dot1\\n\{\}'.format(sess.run(dot1)))  
    print('转置')  
    print('trans\\n\{\}'.format(sess.run(trans)))  
    print('trans1\\n\{\}'.format(sess.run(trans1)))

外积  
out  
\[\[ 2.  6. 12.\]  
 \[ 2.  6. 12.\]\]  
out1  
\[\[ 2.  6. 12.\]  
 \[ 2.  6. 12.\]\]  
点积  
dot  
\[\[18. 24.\]  
 \[18. 24.\]\]  
dot1  
\[\[18. 24.\]  
 \[18. 24.\]\]  
转置  
trans  
\[\[1. 1.\]  
 \[2. 2.\]  
 \[3. 3.\]\]  
trans1  
\[\[1. 1.\]  
 \[2. 2.\]  
 \[3. 3.\]\]

### 矩阵操作tf.reduce\_() ###

#对于2-D
    #所有的reduce_...，如果不加axis的话，都是对整个矩阵进行运算
    tf.reduce_sum(a, 1） #对axis1
    tf.reduce_mean(a,0) #每列均值

第二个参数是axis，如果为0的话，*r**e**s*\[*i*\]=∑*j**a*\[*j*,*i*\]即（*r**e**s*\[*i*\]=∑*a*\[:,*i*\]）， 如果是1的话，*r**e**s*\[*i*\]=∑*j**a*\[*i*,*j*\]  
NOTE:返回的都是行向量,（axis等于几，就是对那维操作,i.e.:沿着那维操作）

### tf.reduce\_mean/tf.nn.moments ###

**tf.math.reduce\_mean(input\_tensor, axis=None, keepdims=None, name=None, reduction\_indices=None, keep\_dims=None)**

Aliases:tf.math.reduce\_mean  tf.[reduce\_mean][reduce_mean]  
Computes the mean of elements across dimensions of a tensor.

参数：axis为哪一维，哪一维就没有了，变成mean（对应于rnn中取所有hidden的平均）。如果只想取axis最后一个，可以使用\[:, -1, :\]（对应于rnn中取最后一个hidden）。当然如果取所有hidden且不平均，可以使用output = tf.keras.layers.Flatten()(output)。

**tf.nn.moments(x, axes)**

同时计算均值和方差

**示例：计算列的均值和方差**

import tensorflow as tf

doc\_score = tf.constant(\[\[1.0, 2, 3\], \[3, 4, 5\]\])  
doc\_score\_mean = tf.reduce\_mean(doc\_score, axis=0) \#, keepdims=True 可要可不要  
doc\_score\_mean = tf.Print(doc\_score\_mean, \['doc\_score\_mean: ', doc\_score\_mean\], summarize=-1)  
doc\_score\_mean\_, doc\_score\_var = tf.nn.moments(doc\_score, axes=0)  
doc\_score\_mean\_ = tf.Print(doc\_score\_mean\_, \['doc\_score\_mean\_: ', doc\_score\_mean\_\], summarize=-1)  
doc\_score\_var = tf.Print(doc\_score\_var, \['doc\_score\_var: ', doc\_score\_var\], summarize=-1)  
doc\_score1 = doc\_score / doc\_score\_mean  
doc\_score1 = tf.Print(doc\_score1, \['doc\_score1: ', doc\_score1\], summarize=-1)  
doc\_score2 = doc\_score / doc\_score\_mean\_  
doc\_score2 = tf.Print(doc\_score2, \['doc\_score2: ', doc\_score2\], summarize=-1)  
with tf.Session() as sess:  
  sess.run(\[doc\_score\_var, doc\_score1, doc\_score2\])  
\# \[doc\_score\_mean: \]\[\[2 3 4\]\]  
\# \[doc\_score\_mean\_: \]\[2 3 4\]  
\# \[doc\_score\_var: \]\[1 1 1\]  
\# \[doc\_score1: \]\[\[0.5 0.666666687 0.75\]\[1.5 1.33333337 1.25\]\]  
\# \[doc\_score2: \]\[\[0.5 0.666666687 0.75\]\[1.5 1.33333337 1.25\]\]

\[[tensorflow 计算均值和方差][tensorflow]\]

### tf.cumprod张量沿轴的累积积 ###

import tensorflow as tf

doc\_score = tf.constant(\[\[1, 2, 3\],  
                         \[2, 3, 4\]\])  
doc\_score\_cum = tf.cumprod(doc\_score, axis=1)  
with tf.Session() as sess:  
    print(sess.run(doc\_score))  
    print(sess.run(doc\_score\_cum))  
\# \[\[1 2 3\]  
\#  \[2 3 4\]\]  
\#  
\# \[\[ 1  2  6\]  
\#  \[ 2  6 24\]\]

\[[计算张量沿轴的累积积][Link 1]\]

### tf.cumsum张量的累积和 ###

cumsum (    
    x ,    
    axis = 0 ,    
    exclusive = False ,    
    reverse = False ,    
    name = None  
  )

### tf.argmax ###

Returns the index with the largest value across axes of a tensor.

tf.math.[argmax][](  
    input,  
    axis=None,  
    name=None,  
    output\_type=tf.dtypes.int64  
)

**示例**

labels = tf.constant(\[\[3, 2, 7\],  
                      \[2, 9, 8\]\])  
\# shape = (batch\_size, num\_labels)  
print(labels)  
with tf.Session() as sess:  
    print(tf.argmax(labels))  \# shape=(3,)=(num\_labels,)  
    print(tf.argmax(labels).eval())  
    print(tf.argmax(labels, 0).eval())  
    print(tf.argmax(labels, 1).eval()) \# shape=(2,)=(batch\_size,)

Tensor("Const:0", shape=(2, 3), dtype=int32)  
Tensor("ArgMax:0", shape=(3,), dtype=int64)  
\[0 1 1\]  
\[0 1 1\]  
\[2 1\]

### 逻辑操作 ###

### tf.logical\_and(),tf.logical\_or(),tf.logical\_not(),tf.logical\_xor() ###

示例：dataset中过滤包含nan的某些特征

def filter\_nan(line):  
    return tf.logical\_not(tf.logical\_or(  
        tf.logical\_or(tf.reduce\_any(tf.is\_nan(line\['user\_feature'\])),  
                      tf.reduce\_any(tf.is\_nan(line\['goods\_feature'\]))),  
        tf.reduce\_any(tf.is\_nan(line\['label'\])))  
    )

dataset = dataset.filter(filter\_nan)

\[[tf.logical\_and(),tf.logical\_or(),tf.logical\_not(),tf.logical\_xor()][tf.logical_and_tf.logical_or_tf.logical_not_tf.logical_xor]\]

[\-柚子皮-][-_-]

# tensor变换 #

### reshape和transpose ###

result=... #[1000,30,1]

result=tf.reshape(result,\[1000,10,3\])

result=tf.transpose(result,perm=\[1,0,2\]) \#\[10,1000,3\]

transpose: 返回数组的 dimension（尺寸、维度） i与输入的 perm\[i\]的维度相一致。如果未给定perm，默认设置为 (n-1...0)，这里的 n 值是输入变量的 rank 。因此默认情况下，这个操作执行了一个正规（regular）的2维矩形的转置。

transpose示例

with tf.Session() as sess:
        c = tf.transpose(x, perm=[0, 2, 1]).eval()
        d = tf.transpose(x, perm=[1, 0, 2]).eval()
        e = tf.reshape(x, [-1, 3]).eval()
    
        print('*' * 20)
        print(c)
        print('*' * 20)
        print(d)
        print(d.shape)
        print('*' * 20)
        print(e)
        print(e.shape)

\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*  
\[\[\[ 1  4\]  
  \[ 2  5\]  
  \[ 3  6\]\]

\[\[ 7 10\]  
  \[ 8 11\]  
  \[ 9 12\]\]\]  
\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*  
\[\[\[ 1  2  3\]  
  \[ 7  8  9\]\]

\[\[ 4  5  6\]  
  \[10 11 12\]\]\]  
(2, 2, 3)  
\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*  
\[\[ 1  2  3\]  
 \[ 4  5  6\]  
 \[ 7  8  9\]  
 \[10 11 12\]\]  
(4, 3)

x原始的perm是\[0, 1, 2\]。

\[0, 2, 1\]相当于是将1和2维转了一下。

\[1, 0, 2\]相当于是将1和0维转了一下。示例：如果\[1 2 3\]表示词（embedding），\[\[ 1  2  3\]  \[ 4  5  6\]\]代表句子，整个代表batch，那么\[1, 0, 2\]转换相当于将本来输入是将一个句子的所有词作为batch输入，转换成了将不同句子的对应位置词作为batch。即将(batch\_size, word\_num\_in\_sent, word\_emb\_size)转成了(word\_num\_in\_sent, batch\_size, word\_emb\_size)。

### tf.concat ###

tf.[concat][]( values,    axis,    name='concat')

其中参数axis为-1时，表示对最后一维进行concat。理解：axis=1即1维的维度会变化。

使用：values = \[(batch, sent, word\_emb1), (batch, sent, word\_emb2)\]，通过tf.concat(values, axis=-1)可以变成\[(batch, sent, word\_emb1+word\_emb2)\]，这样就合并不同word\_emb到一起了。

with tf.Session() as sess:
        g = tf.concat([x, y], axis=-1).eval()

\[\[\[  1   2   3  11  12  13\]  
  \[  4   5   6  14  15  16\]\]

\[\[  7   8   9  17  18  19\]  
  \[ 10  11  12 110 111 112\]\]\]  
(2, 2, 6)

t1 = [[1, 2, 3], [4, 5, 6]]
    t2 = [[7, 8, 9], [10, 11, 12]]
    tf.concat([t1, t2], 0)  # [[1, 2, 3], [4, 5, 6], [7, 8, 9], [10, 11, 12]]
    tf.concat([t1, t2], 1)  # [[1, 2, 3, 7, 8, 9], [4, 5, 6, 10, 11, 12]]
    
    # tensor t3 with shape [2, 3]
    # tensor t4 with shape [2, 3]
    tf.shape(tf.concat([t3, t4], 0))  # [4, 3]
    tf.shape(tf.concat([t3, t4], 1))  # [2, 6]
    
    t1 = [[[1, 2], [2, 3]], [[4, 4], [5, 3]]]
    t2 = [[[7, 4], [8, 4]], [[2, 10], [15, 11]]]
    tf.concat([t1, t2], -1)
    [[[ 1,  2,  7,  4],
      [ 2,  3,  8,  4]],
    
     [[ 4,  4,  2, 10],
      [ 5,  3, 15, 11]]]

### tensor交换两列（或者两行）之间的值 ###

先split再concat

**示例：多列之间值交换**

import tensorflow as tf

doc\_score = tf.constant(\[\[1, 2, 3\],  
                         \[2, 3, 4\]\])

\# \# for tf > 1.15  
\# rows = tf.gather(doc\_score, \[0, 2\], axis=0)  
\# doc\_score\_dst = tf.tensor\_scatter\_nd\_update(doc\_score, \[\[2\], \[0\]\], rows)

src\_split = tf.split(doc\_score, doc\_score.shape\[1\], axis=1)  
\# src\_split\[0\], src\_split\[2\] = src\_split\[2\], src\_split\[0\]  
\# doc\_score\_dst = tf.concat(src\_split\[0\], axis=1)  
doc\_score\_dst = tf.concat(\[src\_split\[1\], src\_split\[2\], src\_split\[0\]\], axis=1)

with tf.Session() as sess:  
    print(sess.run(doc\_score))  
    print(sess.run(doc\_score\_dst))  
\# \[\[1 2 3\]  
\#  \[2 3 4\]\]  
\#  
\# \[\[2 3 1\]  
\#  \[3 4 2\]\]

\[[tensorflow中，一个tensor如何交换两列（或者两行）之间的值？][tensorflow_tensor]\]

### 修改tensor中某些元素 ###

方案1：先修改再concate

示例：doc\_score除了最后一列，其它的都求e的幂

doc\_score = tf.concat(\[tf.exp(doc\_score\[:, -1\]), doc\_score\[:, -1:\]\], axis=1)

示例2：doc\_score只对最后一列数据求power(如=0.4)次方

coff\_power = tf.concat(\[tf.ones(doc\_score.shape\[1\] - 1), tf.constant(\[power\])\], axis=0)

或者coff\_power = tf.constant(self.coff\_power) \#self.coff\_power直接设置为eval('1.0,1.0,0.4')=(1.0,1.0,0.4')

doc\_score = tf.pow(doc\_score, coff\_power)

示例2：

import tensorflow as tf

price\_power = 0.4  
doc\_score = tf.constant(\[\[0.055, 0.061, 3050.0\], \[0.051, 0.058, 1024.1\]\])

coff\_power0 = tf.constant(eval('1.0,1.0,0.4'))  
coff\_power = tf.concat(\[tf.ones(doc\_score.shape\[1\] - 1), tf.constant(\[price\_power\])\], axis=0)  
coff\_power = tf.Print(coff\_power, \['coff\_power: ', coff\_power\], summarize=-1)

doc\_score = tf.pow(doc\_score, coff\_power)  
doc\_score = tf.Print(doc\_score, \['doc\_score2: ', doc\_score\], summarize=-1)

with tf.Session() as sess:  
  sess.run(\[coff\_power0, doc\_score\])  
\# \[coff\_power: \]\[1 1 0.4\]  
\# \[doc\_score2: \]\[\[0.055 0.061 24.7582512\]\[0.051 0.058 16.0006256\]\]

\[[修改tensor中某些元素][tensor]\]

### tf.layers.flatten() ###

Flattens an input tensor while preserving the batch axis (axis 0).

deprecated。可以使用output = tf.keras.layers.Flatten()(output)代替。

使用：将(batch\_size, ?, ..., ?)转成(batch\_size, ?)，这样就可以输入到mlp层得到(batch\_size, label\_size)。

with tf.Session() as sess:
        f = tf.layers.flatten(x).eval()

\[\[ 1  2  3  4  5  6\]  
 \[ 7  8  9 10 11 12\]\]  
(2, 6)

### tf.slice() ###

`tf.slice(input_, begin, size, name=None)`

`现在假设input的shape是[a1, a2, a3], begin的值是[b1, b2, b3],size的值是[s1, s2, s3],那么tf.slice()返回的值就是 input[b1:b1+s1, b2:b2+s2, b3:b3+s3]。 如果 si=−1 ，那么 返回值就是 input[b1:b1+s1,..., bi: ,...]`

`示例`

import tensorflow as tf
    import numpy as np
    sess = tf.InteractiveSession()
    a = np.array([[1,2,3,4,5],[4,5,6,7,8],[9,10,11,12,13]])
    tf.slice(a,[1,2],[-1,2]).eval()
    
    #array([[ 6,  7],
    #       [11, 12]])

### `插入维度`tf.expand\_dims ###

tf.expand\_dims用来在tensor中插入一个维度。

tf.[expand\_dims][expand_dims](  
    input,  
    axis=None,  
    name=None  
)

注意：expand\_dims will not add or reduce elements in tensor, it just change the shape by adding 1 to dimensions. For example, a vector with 10 elements could be treat as a 10x1 matrix.

The situation I have met to use `expand_dims` is when I tried to build a ConvNet to classify grayscale images. The grayscale images will be loaded as matrix of size `[320, 320]`. However, `tf.nn.conv2d` require input to be `[batch, in_height, in_width, in_channels]`, where the `in_channels` dimension is missing in my data. So I used `expand_dims` to add one more dimension. \[[Tensorflow: When use tf.expand\_dims?][Tensorflow_ When use tf.expand_dims]\]

**示例1：**

import tensorflow as tf

z = tf.constant(\[11, 12\])

with tf.Session() as sess:  
    print(z.shape)  
    z\_exp = tf.expand\_dims(z, 0)  
    print(z\_exp.eval())  
    print(z\_exp.shape)  
    z\_exp1 = tf.expand\_dims(z, 1) \# = tf.expand\_dims(z, -1)  = tf.expand\_dims(z, 0).transpose()  
    print(z\_exp1.eval())  
    print(z\_exp1.shape)

(2,)  
\[\[11 12\]\]  
(1, 2)  
\[\[11\]  
 \[12\]\]  
(2, 1)

**示例2：**

# 't' is a tensor of shape [2]
    tf.shape(tf.expand_dims(t, 0))  # [1, 2]
    tf.shape(tf.expand_dims(t, 1))  # [2, 1]
    tf.shape(tf.expand_dims(t, -1))  # [2, 1]

**示例3：**

下面的维度插入目的是把这两个tensor从2维转换成3维，使得大小匹配可以进行减法：

tf.expand\_dims在每一个tensor中插入一个维度，在vector的tensor中第一维度（D0）插入，在centroides tensor中第二维度（D1）插入。从图片来看，扩展后的tensor中各维度有了同样的含义：

![20190704172136840.jpg][]  
对于左图，在0维插入，则之前的0维变成1维，之前的1维变成2维。

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly95b3V6aXBpLmJsb2cuY3Nkbi5uZXQ_size_16_color_FFFFFF_t_70][]

### tf.tile() ###

tensorflow中的tile()函数是用来对张量(Tensor)进行扩展的，其特点是对当前张量内的数据进行一定规则的复制。最终的输出张量维度不变。

函数定义：

tf.tile(  
    input,  
    multiples,  
    name=None  
)  
input是待扩展的张量，multiples是扩展方法。如果multiples的某一个数据为1，则表示该维数据保持不变。  
假如input是一个3维的张量。那么mutiples就必须是一个1x3的1维张量。这个张量的三个值依次表示input的第1，第2，第3维数据扩展几倍。

import tensorflow as tf

a = tf.constant(\[\[1, 2\], \[3, 4\], \[5, 6\]\], dtype=tf.float32)  
a1 = tf.tile(a, \[2, 3\])  
with tf.Session() as sess:  
    print(sess.run(a))  
    print(sess.run(tf.shape(a)))  
    print(sess.run(a1))  
    print(sess.run(tf.shape(a1)))  
=======  
\[\[1. 2.\]  
 \[3. 4.\]  
 \[5. 6.\]\]  
\[3 2\]  
\[\[1. 2. 1. 2. 1. 2.\]  
 \[3. 4. 3. 4. 3. 4.\]  
 \[5. 6. 5. 6. 5. 6.\]  
 \[1. 2. 1. 2. 1. 2.\]  
 \[3. 4. 3. 4. 3. 4.\]  
 \[5. 6. 5. 6. 5. 6.\]\]  
\[6 6\]  
tf.tile()具体的操作过程如下：（原始矩阵应为3\*2）

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3BpcGlzb3JyeQ_size_16_color_FFFFFF_t_70][]  
每一维数据的扩展都是将前面的数据进行复制然后直接接在原数据后面。

### tf.stack() ###

tf.stack(values, axis=0, name=’stack’)

将 a list of R 维的`Tensor`堆成 `R+1`维的`Tensor`。  
Given a list of length N of tensors of shape (A, B, C);  
if axis == 0 then the output tensor will have the shape (N, A, B, C)

这时 res[i,:,:,:] 就是原 list中的第 i 个 tensor

. if axis == 1 then the output tensor will have the shape (A, N, B, C).

这时 res[:,i,:,:] 就是原list中的第 i 个 tensor

Etc.

# 'x' is [1, 4]
    # 'y' is [2, 5]
    # 'z' is [3, 6]
    stack([x, y, z]) => [[1, 4], [2, 5], [3, 6]]  # Pack along first dim.
    stack([x, y, z], axis=1) => [[1, 2, 3], [4, 5, 6]]

### tf.gather() ###

`tf.gather(params, indices, validate_indices=None, name=None)`

`indices must be an integer tensor of any dimension (usually 0-D or 1-D). Produces an output tensor with shape indices.shape + params.shape[1:]`

# Scalar indices, 会降维
    output[:, ..., :] = params[indices, :, ... :]
    
    # Vector indices
    output[i, :, ..., :] = params[indices[i], :, ... :]
    
    # Higher rank indices，会升维
    output[i, ..., j, :, ... :] = params[indices[i, ..., j], :, ..., :]

### tf.pad ###

tf.pad(tensor, paddings, mode="CONSTANT", name=None)

*  `tensor`: 任意`shape`的`tensor`，维度 `Dn`
 *  `paddings`: `[Dn, 2]` 的 `Tensor`, `Padding`后`tensor`的某维上的长度变为`padding[D,0]+tensor.dim_size(D)+padding[D,1]`
 *  `mode`: `CONSTANT`表示填`0`, `REFLECT`表示反射填充,`SYMMETRIC`表示对称填充。

\[[Tensor Transformations][]\]

### tf.one\_hot ###

tf.[one\_hot][one_hot](  
    indices,  
    depth,  
    on\_value=None,  
    off\_value=None,  
    axis=None,  
    dtype=None,  
    name=None  
)

示例：

tf.one\_hot(0, depth=2) = \[1. 0.\]

所以两类时input\_func需要这样处理label变成向量

label\_id\_vec = \[0.0\] \* params\['class\_num'\]  
    label\_id\_vec\[int(label)\] = 1.0

Note: tf.one\_hot的坑：tf.one\_hot(10, depth=2) = \[0. 0.\]，数值总是从0开始算，10只能表示成\[0\]\*9+\[1\],depth<10时就会是\[0\]\*depth。

### tensor保留其中一个最大值变为1，其它变成0 ###

how to convert max value in  tensor to 1 and others to 0 argmax

**示例1**

输入shape = (batch\_size, num\_labels)

logits = tf.constant(\[\[0.3, 0.0, 0.5, 0.2\],  
                      \[0.0, 1.0, 0.0, 0.0\],  
                      \[0.1, 0.1, 0, 0.8\],  
                      \[0.44, 0.32, 0.23, 0.01\],  
                      \[-0.2, 0.6, 0.5, 0.1\]\])  
num\_labels = logits.shape\[1\]  
pred\_ids = tf.one\_hot(tf.argmax(logits, 1), depth=num\_labels)  
pred\_ids1 = tf.one\_hot(tf.argmax(logits, 1), depth=num\_labels + 1)

with tf.Session() as sess:  
    print(pred\_ids.eval())  
    print(pred\_ids1.eval())

\[\[0. 0. 1. 0.\]  
 \[0. 1. 0. 0.\]  
 \[0. 0. 0. 1.\]  
 \[1. 0. 0. 0.\]  
 \[0. 1. 0. 0.\]\]  
\[\[0. 0. 1. 0. 0.\]  
 \[0. 1. 0. 0. 0.\]  
 \[0. 0. 0. 1. 0.\]  
 \[1. 0. 0. 0. 0.\]  
 \[0. 1. 0. 0. 0.\]\]

**示例2**

输入shape = (num\_labels,)

**方法1**

label1 = tf.one\_hot(tf.argmax(label1), depth=label1.shape\[0\])

**方法2**

不同于方法1，这个方法中多个max都会变成1。

tf.where(
            tf.equal(tf.reduce_max(x, axis=1, keep_dims=True), x), 
            tf.constant(1, shape=x.shape), 
            tf.constant(-1, shape=x.shape)
        )

\[[tensorflow function that projects max value to 1 and others -1 without using zeros][]\]

### tf.sigmoid ###

示例：tensor先sigmoid变换，再min\_max归一化  
import tensorflow as tf

doc\_score = tf.constant(\[\[0.0\], \[0.2\], \[0.3\],  
                         \[0.2\], \[1.0\], \[4.0\]\])

doc\_score = tf.Print(doc\_score, \['doc\_score: ', doc\_score\], summarize=10)  
doc\_score\_sig = tf.sigmoid(doc\_score)  
doc\_score\_sig = tf.Print(doc\_score\_sig, \['doc\_score\_sig: ', doc\_score\_sig\], summarize=10)  
doc\_score\_scale = doc\_score\_sig \* 2.0 - 1.0  
doc\_score\_scale = tf.Print(doc\_score\_scale, \['doc\_score\_scale: ', doc\_score\_scale\], summarize=10)

with tf.Session() as sess:  
  sess.run(doc\_score\_scale)  
\# \[doc\_score: \]\[\[0\]\[0.2\]\[0.3\]\[0.2\]\[1\]\[4\]\]  
\# \[doc\_score\_sig: \]\[\[0.5\]\[0.549833953\]\[0.574442506\]\[0.549833953\]\[0.731058598\]\[0.982013762\]\]  
\# \[doc\_score\_scale: \]\[\[0\]\[0.0996679068\]\[0.148885012\]\[0.0996679068\]\[0.462117195\]\[0.964027524\]\]

### tf.nn.softmax ###

Computes softmax activations. (deprecated arguments)

tf.nn.[softmax][](  
    logits,  
    axis=None,  
    name=None,  
    dim=None  
)

其中参数：axis: The dimension softmax would be performed on. The default is -1 which indicates the last dimension.

### 构建tensor的mask ###

数据处理时输出每个字符串的长度得到shape=(batch\_size,)的nwords，再通过tf.sequence\_mask获取对应的mask。

示例：batch\_size=3，每个字符串长度分别为3，2，5的处理。

nwords = tf.constant(\[3, 2, 5\])  
weights = tf.sequence\_mask(nwords)  
with tf.Session() as sess:  
    print(weights.eval())

\[\[ True  True  True False False\]  
 \[ True  True False False False\]  
 \[ True  True  True  True  True\]\]

### 自定义变换 ###

可以使用tf.py\_func\[[Tensorflow函数映射：py\_func和map\_fn][Tensorflow_py_func_map_fn]\]。

[\-柚子皮-][-_-]

## tensor操作 ##

### tf.assign(ref, value, validate\_shape=None, use\_locking=None, name=None) ###

将　value 赋值给　ref，并输出 ref，即 ref = value；这使得需要使用复位值的连续操作变简单。

update = tf.assign(ref, new\_value) \# 平时的使用写法

只有当这个赋值被完成时，该旧值`ref`才会被修改成`new_value`。所谓的赋值被完成其实指得是需要对`tf.assign()`函数的返回值执行一下`sess.run()`操作后，才能保证正常更新。

\[[Tensorflow：tf.assign()函数的使用方法及易错点][Tensorflow_tf.assign]\]

###  `tf.assign_add(ref,value,use_locking=None,name=None)`  ###

Update 'ref' by adding 'value' to it.

更新ref的值，通过增加value，即：ref = ref + value；

### **`tf.identity(``input,``name=None``)`** ###

Return a tensor with the same shape and contents as input.

返回一个具有相同形状张量和内容作为输入；

# tf控制类函数 #

### `tf.control_dependencies(control_inputs)` ###

`tf.control_dependencies()`设计是用来控制计算流图的，给图中的某些计算指定顺序。

在有些机器学习程序中我们想要指定某些操作执行的依赖关系，这时我们可以使用tf.control\_dependencies()来实现。  
control\_dependencies(control\_inputs)返回一个控制依赖的上下文管理器，使用with关键字可以让在这个上下文环境中的操作都在control\_inputs 执行。  
\[[tf.control\_dependencies()作用及用法][tf.control_dependencies]\]

### tf.group( \*inputs, \*\*kwargs ) ###

ops = tf.group(tensor1, tensor2,...)

其中\*inputs是0个或者多个用于组合tensor，一旦ops完成了，那么传入的tensor1,tensor2,...等等都会完成了，经常用于组合一些训练节点。

注意的是，tf.group()返回的是个操作，而不是值，是一个None，就是因为这个是一个操作，而不是一个张量。如果需要返回结果，请参考tf.tuple()。

\[[tf.group()用于组合多个操作][tf.group]\]

from:[\-柚子皮-][-_-]

ref:

[to_float]: http://tensorflow.biotecan.com/python/Python_1.8/tensorflow.google.cn/api_docs/python/tf/to_float.html
[TensorFlow]: https://hacpai.com/article/1498556150279
[tf.cast]: https://blog.csdn.net/dcrmg/article/details/79747814
[TensorFlow_Unicode_NLP]: https://www.jishuwen.com/d/2Csy#tuit
[matmul]: https://www.tensorflow.org/api_docs/python/tf/linalg/matmul
[einsum]: https://www.tensorflow.org/api_docs/python/tf/einsum
[reduce_mean]: https://www.tensorflow.org/api_docs/python/tf/math/reduce_mean
[tensorflow]: https://www.cnblogs.com/demo-deng/p/10207694.html
[Link 1]: https://www.w3cschool.cn/tensorflow_python/tensorflow_python-ug6f2df4.html
[argmax]: https://www.tensorflow.org/api_docs/python/tf/math/argmax
[tf.logical_and_tf.logical_or_tf.logical_not_tf.logical_xor]: https://blog.csdn.net/weixin_43374508/article/details/113530811
[-_-]: https://me.csdn.net/pipisorry
[concat]: https://www.tensorflow.org/api_docs/python/tf/concat
[tensorflow_tensor]: https://www.zhihu.com/question/384051840
[tensor]: https://www.cxyzjd.com/article/qq_34418352/106399327
[expand_dims]: https://www.tensorflow.org/api_docs/python/tf/expand_dims
[Tensorflow_ When use tf.expand_dims]: https://stackoverflow.com/questions/39008821/tensorflow-when-use-tf-expand-dims
[20190704172136840.jpg]: /images/20220612/ad1b61d1dc3f47da96f045801f9627c3.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly95b3V6aXBpLmJsb2cuY3Nkbi5uZXQ_size_16_color_FFFFFF_t_70]: /images/20220612/e6342342dc89495e9d1ccd24635cf95d.png
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3BpcGlzb3JyeQ_size_16_color_FFFFFF_t_70]: /images/20220612/84ecc18fb3a047a195fa1ccbad41a156.png
[Tensor Transformations]: https://www.tensorflow.org/versions/r0.12/api_docs/python/array_ops/
[one_hot]: https://www.tensorflow.org/api_docs/python/tf/one_hot
[tensorflow function that projects max value to 1 and others -1 without using zeros]: https://stackoverflow.com/questions/46368815/tensorflow-function-that-projects-max-value-to-1-and-others-1-without-using-zer
[softmax]: https://www.tensorflow.org/api_docs/python/tf/nn/softmax
[Tensorflow_py_func_map_fn]: https://blog.csdn.net/pipisorry/article/details/94639393
[Tensorflow_tf.assign]: https://blog.csdn.net/Invokar/article/details/89041501
[tf.control_dependencies]: https://blog.csdn.net/PKU_Jade/article/details/73498753
[tf.group]: https://blog.csdn.net/LoseInVain/article/details/81703786