pytorch基础操作学习笔记（autograd，Tensor）

以你之姓@ 2022-04-14 03:50 1158阅读 0赞

## 简述 ##

简单讲讲关于torch.autograd内容（因为我也有点菜）

### 文章目录 ###

*   *  简述
     *  简单讲讲
     *  Tensor
     *   *  介绍Tensor
         *  创建Tensor
         *  获取Tensor数据规模
         *  将tensor转成其他数据类型
         *  改变Tensor形状
         *  Tensor的切片操作
         *  Tensor的比较
         *  Tensor的数据筛选
         *  Tensor其他常用函数
         *   *  index\_select(input, dim, index)函数
             *  masked\_select(input, mask)
             *  nonzero(input)操作
             *  gather 根据index，在dim上选取数据，输出size与index一直
         *  clamp截断
     *  后记

## 简单讲讲 ##

*  创建数据

>>> import torch
    >>> from torch.autograd import Variable
    >>> x = Variable(torch.ones(2, 2), requires_grad=True)
    >>> x
    tensor([[1., 1.],
            [1., 1.]], requires_grad=True)

*  Variable中有三个东西，就是下面的三个：`data`，`grad`，`grad_fn`

>>> x.data
    tensor([[1., 1.],
            [1., 1.]])
    >>> x.grad
    >>> x.grad_fn

*  这时候下面两个其实是NoneType的

>>> x.grad_fn.data
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    AttributeError: 'NoneType' object has no attribute 'data'
    >>> x.grad_fn.data
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    AttributeError: 'NoneType' object has no attribute 'data'

*  尝试做反向传播

>>> x.backward()
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "C:\Users\sean\AppData\Local\Programs\Python\Python36\lib\site-packages\torch\tensor.py", line 93, in backward
        torch.autograd.backward(self, gradient, retain_graph, create_graph)
      File "C:\Users\sean\AppData\Local\Programs\Python\Python36\lib\site-packages\torch\autograd\__init__.py", line 84, in backward
        grad_tensors = _make_grads(tensors, grad_tensors)
      File "C:\Users\sean\AppData\Local\Programs\Python\Python36\lib\site-packages\torch\autograd\__init__.py", line 28, in _make_grads
        raise RuntimeError("grad can be implicitly created only for scalar outputs")
    RuntimeError: grad can be implicitly created only for scalar outputs

**意思很简单，就是说，这里如果是做反向传播，东西必须是标量的输出。**

*  对x求和

>>> y = x.sum()
    >>> y
    tensor(4., grad_fn=<SumBackward0>)

*  关于y做反向传播  
    有反向传播的函数。

>>> y.backward()
    >>> y
    tensor(4., grad_fn=<SumBackward0>)
    >>> y.grad
    >>> y.grad_fn
    <SumBackward0 object at 0x0000026BADBFF278>
    >>> a = y.backward()
    >>> a
    >>> a == None
    True

*  这时候再去检查x

>>> x
    tensor([[1., 1.],
            [1., 1.]], requires_grad=True)
    >>> x.grad
    tensor([[2., 2.],
            [2., 2.]])
    >>> x.grad_fn

**x的梯度不再是None**  
这里的话，其实x的梯度本来应该全是1的，但是做了两次之后，有了累积。

*  再做一次y的反向传播，验证猜想

>>> y.backward()
    >>> x.grad
    tensor([[3., 3.],
            [3., 3.]])

验证正确。所以这告诉我们每次计算前都需要把梯度归零。这里不归零，是因为有些算法需要用到这个操作。

*  梯度清0

>>> x.grad.data.zero_()
    tensor([[0., 0.],
            [0., 0.]])
    >>> x
    tensor([[1., 1.],
            [1., 1.]], requires_grad=True)
    >>> x.grad
    tensor([[0., 0.],
            [0., 0.]])

*  再算一次y的反向传播

>>> y.backward()
    >>> x.grad
    tensor([[1., 1.],
            [1., 1.]])

*  关于之前的求梯度结果全是1的解释

y = ∑ X i y = \\sum\{X\_i\} y=∑Xi

y y y关于 X i X\_i Xi来求导，得到的结果都是1。所以每一个数值都是1。

*  验证无关a的数值

由于我们看到下面的数值当中，a的数值都变成了2。但是 求导的结果任然是这个。

>>> a = Variable(torch.ones(2,2)*2, requires_grad=True)
    >>> a
    tensor([[2., 2.],
            [2., 2.]], requires_grad=True)
    >>> y = a.sum()
    >>> a
    tensor([[2., 2.],
            [2., 2.]], requires_grad=True)
    >>> y
    tensor(8., grad_fn=<SumBackward0>)
    >>> y.backward()
    >>> a.grad
    tensor([[1., 1.],
            [1., 1.]])

*  验证和算法有关

y = ∑ cos ⁡ X i y = \\sum\{\\cos\{X\_i\}\} y=∑cosXi  
关于X\_i求导有

d y d X i = sin ⁡ X i \\frac\{dy\}\{dX\_i\} =\\sin\{X\_i\} dXidy=sinXi

>>> a = Variable(torch.ones(2,2)*2, requires_grad=True)
    >>> a
    tensor([[2., 2.],
            [2., 2.]], requires_grad=True)
    >>> a.grad
    >>> b = a.cos()
    >>> b
    tensor([[-0.4161, -0.4161],
            [-0.4161, -0.4161]], grad_fn=<CosBackward>)
    >>> c = b.sum()
    >>> b.grad
    >>> c.backward()
    >>> b.grad
    >>> a.grad
    tensor([[-0.9093, -0.9093],
            [-0.9093, -0.9093]])

**通过数值来验算一下：**

>>> import numpy as np
    >>> np.sin(2)
    0.9092974268256817
    >>>

发现，保留4位有效数值的结果是一样的。

## Tensor ##

### 介绍Tensor ###

> Tensor，又名张量。从工程角度不妨认为是一个数组。  
> 和numpy中的ndarray类似，但是tensor支持GPU加速
> 
> –部分摘取于《深度学习框架Pytorch入门与实践》

### 创建Tensor ###

> 只有使用torch.Tensor(\*size)的方式创建的数组，才是创建数组的时候并不会分配内存空间，只有当使用到的时候，才会分配。

*  直接Tensor创建

>>> a = torch.Tensor(1)
    >>> a
    tensor([5.6052e-45])

发现数值其实是接近0的（准确说，在计算机里这个跟0也没太多区别了，一般来说）

*  直接Tensor创建（二）

**这个是将另外的类似于数组的结构转变成Tensor**

>>> a = torch.Tensor((2, 3))
    >>> a
    tensor([2., 3.])

类似的操作

>>> a = torch.Tensor((2, 3, 4, 5))
    >>> a
    tensor([2., 3., 4., 5.])

>>> a = torch.Tensor([2, 3, 4, 5])
    >>> a
    tensor([2., 3., 4., 5.])

**不得不说，这其实跟numpy的array构造法类似的**

>>> a = torch.Tensor(np.array([2, 3, 4, 5]))
    >>> a
    tensor([2., 3., 4., 5.])

*  直接Tensor创建（三）

用size的维度来做输入

>>> a = torch.Tensor(1,2)
    >>> a
    tensor([[0.0000, 0.0000]])

>>> a = torch.Tensor(3,2)
    >>> a
    tensor([[0.0000, 0.0000],
            [0.0000, 0.0000],
            [0.0000, 0.0000]])

*  构造全1的张量

>>> a = torch.ones(3,2)
    >>> a
    tensor([[1., 1.],
            [1., 1.],
            [1., 1.]])
    >>> a = torch.ones((3,2))
    >>> a
    tensor([[1., 1.],
            [1., 1.],
            [1., 1.]])
    >>> a = torch.ones([3,2])
    >>> a
    tensor([[1., 1.],
            [1., 1.],
            [1., 1.]])
    >>> a = torch.ones(np.array((3,2)))
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: ones(): argument 'size' (position 1) must be tuple of ints, not numpy.ndarray
    >>>

除了numpy不能使用的，其他的上面提到的，都可以作为**size**

*  构造全0的张量

跟全1的一模一样。

>>> a = torch.zeros(3,2)
    >>> a
    tensor([[0., 0.],
            [0., 0.],
            [0., 0.]])
    >>> a = torch.zeros((3,2))
    >>> a
    tensor([[0., 0.],
            [0., 0.],
            [0., 0.]])
    >>> a = torch.zeros([3,2])
    >>> a
    tensor([[0., 0.],
            [0., 0.],
            [0., 0.]])
    >>> a = torch.zeros(np.array([3,2]))
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: zeros(): argument 'size' (position 1) must be tuple of ints, not numpy.ndarray
    >>> a = torch.zeros(1)
    >>> a
    tensor([0.])
    >>> a = torch.zeros(2)
    >>> a
    tensor([0., 0.])
    >>>

*  对角元全是1，其他都是0的矩阵  
    只能是方阵

>>> a = torch.eye(2)
    >>> a
    tensor([[1., 0.],
            [0., 1.]])
    >>> a = torch.eye(1)
    >>> a
    tensor([[1.]])
    >>> a = torch.eye(3)
    >>> a
    tensor([[1., 0., 0.],
            [0., 1., 0.],
            [0., 0., 1.]])
    >>> a = torch.eye(4)
    >>> a
    tensor([[1., 0., 0., 0.],
            [0., 1., 0., 0.],
            [0., 0., 1., 0.],
            [0., 0., 0., 1.]])
    >>> a = torch.eye((1,2))
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: eye(): argument 'n' (position 1) must be int, not tuple

*  arange(s,e, step)
    
     *  step：步长

>>> a = torch.arange(0,1,10)
    >>> 1
    >>> a = torch.arange(0,10,1)
    >>> a
    tensor([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])

*  linspace(s, e, steps)
    
     *  steps：步数
     *  steps：大于等于2

>>> a = torch.linspace(0,1,10)
    >>> a
    tensor([0.0000, 0.1111, 0.2222, 0.3333, 0.4444, 0.5556, 0.6667, 0.7778, 0.8889,
            1.0000])
    >>> a = torch.linspace(0,10,1)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    RuntimeError: invalid argument 3: invalid number of points at d:\build\pytorch\pytorch-0.4.1\aten\src\th\generic\thtensormath.cpp:4408
    >>> a = torch.linspace(0,10,2)
    >>> a
    tensor([ 0., 10.])

*  rand(\*size) 均匀分布 (**类似的有randn标准正态分布**)

>>> a = torch.rand(1)
    >>> a
    tensor([0.5391])
    >>> a = torch.rand(1)
    >>> a
    tensor([0.7884])
    >>> a = torch.rand(1,2)
    >>> a
    tensor([[0.6491, 0.8738]])
    >>> a = torch.rand(2,1)
    >>> a
    tensor([[0.1860],
            [0.4988]])
    >>> a = torch.rand(2)
    >>> a
    tensor([0.9654, 0.6263])

>>> a = torch.randn(1)
    >>> a
    tensor([-1.2497])
    >>> a = torch.randn(1)
    >>> a
    tensor([-2.0466])
    >>> a = torch.randn(1)
    >>> a
    tensor([0.4620])
    >>> a = torch.randn(2)
    >>> a
    tensor([0.3337, 0.5260])
    >>> a = torch.randn(1,2)
    >>> a
    tensor([[-0.0197,  0.1966]])
    >>> a = torch.randn(2,1)
    >>> a
    tensor([[-0.3927],
            [-0.4279]])

*  normal(mean, std)正态分布 / (uniform(s, e))均匀分布

> 下面这个写的normal()还行  
> [https://blog.csdn.net/qq\_34690929/article/details/80029974][https_blog.csdn.net_qq_34690929_article_details_80029974]

*  固定mean，但不固定std

>>> torch.normal(mean=0,std=torch.rand(5),out=a)
    tensor([-0.1707,  0.6314,  0.2634, -0.2100, -1.3774])
    >>> torch.normal(mean=0,std=torch.rand(5))
    tensor([-0.1613, -0.0957,  0.2675,  0.3178, -0.6548])

*  固定std，但不固定mean

>>> torch.normal(mean=torch.rand(5), std=1)
    tensor([-0.7600,  1.1998,  0.9734,  3.5803,  0.8375])
    >>> torch.normal(mean=torch.rand(5), std=1,out=a)
    tensor([ 0.5822,  2.9761, -0.3904,  0.7321,  1.2330])

*  两个都不固定

>>> torch.normal(mean=torch.rand(5), std=torch.linspace(1,2,5),out=a)
    tensor([-0.2679,  0.1669,  3.7573, -1.9038, -0.2729])
    >>> torch.normal(mean=torch.rand(5), std=torch.linspace(1,2,5))
    tensor([ 0.7096,  1.0136, -2.5004, -0.2733,  3.6389])

*  如果想要两个都固定，只能重复来做
    
     *  下面的构造函数只有三个。

>>> torch.normal(mean=0,std=1,out=a)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: normal() received an invalid combination of arguments - got (mean=int, std=int, out=Tensor, ), but expected one of:
     * (Tensor mean, Tensor std, torch.Generator generator, Tensor out)
     * (Tensor mean, float std, torch.Generator generator, Tensor out)
     * (float mean, Tensor std, torch.Generator generator, Tensor out)

比如：  
下面的就是标准正态分布。

>>> torch.normal(mean=torch.ones(5), std=torch.ones(5)+1)
    tensor([ 0.6148, -3.3352,  3.0508,  0.4238,  1.4016])

**uniform就不一样了。必须另外操作**

我怀疑是改了版本了，我看有些书都是没这么写的。。感觉蛮奇怪的。

**但是在pytorch的官方文档中有这样的解释：**

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ExOTk5MDQxMg_size_16_color_FFFFFF_t_70]

> Fills `self` tensor with numbers sampled from the continuous uniform distribution:  
> 用采样自下面的连续均匀分布的数字来填充self tensor 。

**所以，虽然文档中没有写具体的实例代码，也能知道如何操作（如下：）**

>>> torch.Tensor(10).uniform_(0,1)
    tensor([0.3394, 0.8679, 0.3344, 0.6168, 0.6465, 0.7618, 0.3902, 0.5174, 0.0205, 0.6731])

*  randperm(m)随机排列

生成0到m的随机排列

>>> torch.randperm(10)
    tensor([1, 7, 4, 8, 9, 3, 0, 6, 2, 5])

### 获取Tensor数据规模 ###

*  其实只有下面的两种方法

>>> a
    tensor([-0.2679,  0.1669,  3.7573, -1.9038, -0.2729])
    >>> a.size()
    torch.Size([5])
    >>> a.shape
    torch.Size([5])

>>> a
    tensor([[-0.2679],
            [ 0.1669],
            [ 3.7573],
            [-1.9038],
            [-0.2729]])
    >>> a.size()
    torch.Size([5, 1])
    >>> a.shape
    torch.Size([5, 1])

### 将tensor转成其他数据类型 ###

*  转成list

很吃惊的是，居然精度有这么高！！

>>> a
    tensor([-0.2679,  0.1669,  3.7573, -1.9038, -0.2729])
    >>> a.tolist()
    [-0.2678675651550293, 0.16691356897354126, 3.757300853729248, -1.9038498401641846, -0.27292633056640625]

*  转成numpy

>>> a
    tensor([-0.2679,  0.1669,  3.7573, -1.9038, -0.2729])
    >>> a.numpy()
    array([-0.26786757,  0.16691357,  3.7573009 , -1.9038498 , -0.27292633],
          dtype=float32)

### 改变Tensor形状 ###

先创建一个数据

>>> a
    tensor([[-0.0752,  0.0072,  1.7456,  1.2480,  0.7506, -1.2426],
            [ 0.1641, -0.6519, -0.9540, -1.0443, -0.8130,  1.0243],
            [ 1.2052, -1.0993,  2.5415,  0.9572, -0.9123,  0.6194],
            [ 1.4059,  1.1456, -0.1732, -1.0271, -0.0565, -0.6258],
            [ 0.7262, -2.5908,  0.5556,  0.6691, -0.0912,  2.1089],
            [ 1.5669, -0.6453,  0.8954,  0.4817, -0.6550,  0.9734]])
    >>> a.size()
    torch.Size([6, 6])

*  reshape操作

>>> a.reshape(2, -1)
    tensor([[-0.0752,  0.0072,  1.7456,  1.2480,  0.7506, -1.2426,  0.1641, -0.6519,
             -0.9540, -1.0443, -0.8130,  1.0243,  1.2052, -1.0993,  2.5415,  0.9572,
             -0.9123,  0.6194],
            [ 1.4059,  1.1456, -0.1732, -1.0271, -0.0565, -0.6258,  0.7262, -2.5908,
              0.5556,  0.6691, -0.0912,  2.1089,  1.5669, -0.6453,  0.8954,  0.4817,
             -0.6550,  0.9734]])
    >>> a.reshape(2, -1).shape
    torch.Size([2, 18])

可以看出，其实是扁平压缩的。应该说，这个是类似于C++存放数组的方式，将高维数据转成一维数据来处理。之后，获取的时候，只需要用这种方式来进行获取就好了。

**但是注意，replace不会覆盖原来的版本**

*  view方法

>>> a.view(2, -1)
    tensor([[-0.0752,  0.0072,  1.7456,  1.2480,  0.7506, -1.2426,  0.1641, -0.6519,
             -0.9540, -1.0443, -0.8130,  1.0243,  1.2052, -1.0993,  2.5415,  0.9572,
             -0.9123,  0.6194],
            [ 1.4059,  1.1456, -0.1732, -1.0271, -0.0565, -0.6258,  0.7262, -2.5908,
              0.5556,  0.6691, -0.0912,  2.1089,  1.5669, -0.6453,  0.8954,  0.4817,
             -0.6550,  0.9734]])
    >>> a.reshape(2,-1)
    tensor([[-0.0752,  0.0072,  1.7456,  1.2480,  0.7506, -1.2426,  0.1641, -0.6519,
             -0.9540, -1.0443, -0.8130,  1.0243,  1.2052, -1.0993,  2.5415,  0.9572,
             -0.9123,  0.6194],
            [ 1.4059,  1.1456, -0.1732, -1.0271, -0.0565, -0.6258,  0.7262, -2.5908,
              0.5556,  0.6691, -0.0912,  2.1089,  1.5669, -0.6453,  0.8954,  0.4817,
             -0.6550,  0.9734]])

**但是a也不会覆盖掉**

*  `resize_`方法

>>> a.resize_(2,18)
    tensor([[-0.0752,  0.0072,  1.7456,  1.2480,  0.7506, -1.2426,  0.1641, -0.6519,
             -0.9540, -1.0443, -0.8130,  1.0243,  1.2052, -1.0993,  2.5415,  0.9572,
             -0.9123,  0.6194],
            [ 1.4059,  1.1456, -0.1732, -1.0271, -0.0565, -0.6258,  0.7262, -2.5908,
              0.5556,  0.6691, -0.0912,  2.1089,  1.5669, -0.6453,  0.8954,  0.4817,
             -0.6550,  0.9734]])
    >>> a
    tensor([[-0.0752,  0.0072,  1.7456,  1.2480,  0.7506, -1.2426,  0.1641, -0.6519,
             -0.9540, -1.0443, -0.8130,  1.0243,  1.2052, -1.0993,  2.5415,  0.9572,
             -0.9123,  0.6194],
            [ 1.4059,  1.1456, -0.1732, -1.0271, -0.0565, -0.6258,  0.7262, -2.5908,
              0.5556,  0.6691, -0.0912,  2.1089,  1.5669, -0.6453,  0.8954,  0.4817,
             -0.6550,  0.9734]])

这次就会发生改变了。  
**但是注意，这次不允许使用负数（来进行默认计算）**

>>> a.resize_(2,-1)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    RuntimeError: sizes must be non-negative

再变回去，内容也不会发生变化

>>> a.resize_(6,6)
    tensor([[-0.0752,  0.0072,  1.7456,  1.2480,  0.7506, -1.2426],
            [ 0.1641, -0.6519, -0.9540, -1.0443, -0.8130,  1.0243],
            [ 1.2052, -1.0993,  2.5415,  0.9572, -0.9123,  0.6194],
            [ 1.4059,  1.1456, -0.1732, -1.0271, -0.0565, -0.6258],
            [ 0.7262, -2.5908,  0.5556,  0.6691, -0.0912,  2.1089],
            [ 1.5669, -0.6453,  0.8954,  0.4817, -0.6550,  0.9734]])

*  **unsqueeze()方法**
    
     *  压缩
     *  其实就是扩充一个维度
     *  dim这个参数很关键。（从0开始，由于一开始没有dim=1，使用后多出这个维度来）
     *  不发生覆盖

>>> a = torch.rand(10)
    >>> a
    tensor([0.5478, 0.4366, 0.2502, 0.5778, 0.7834, 0.8406, 0.3881, 0.8908, 0.0255,
            0.4718])
    >>> a.unsqueeze(dim=1)
    tensor([[0.5478],
            [0.4366],
            [0.2502],
            [0.5778],
            [0.7834],
            [0.8406],
            [0.3881],
            [0.8908],
            [0.0255],
            [0.4718]])
    >>> a
    tensor([0.5478, 0.4366, 0.2502, 0.5778, 0.7834, 0.8406, 0.3881, 0.8908, 0.0255,
            0.4718])

但是如果维度只有1的话，我们dim=2就会出问题

>>> a.unsqueeze(dim=2)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    RuntimeError: Dimension out of range (expected to be in range of [-2, 1], but got 2)

但是这个报错很有趣，所以，我都试了下

>>> a
    tensor([0.5478, 0.4366, 0.2502, 0.5778, 0.7834, 0.8406, 0.3881, 0.8908, 0.0255,
            0.4718])
    >>> a.unsqueeze(dim=0)
    tensor([[0.5478, 0.4366, 0.2502, 0.5778, 0.7834, 0.8406, 0.3881, 0.8908, 0.0255,
             0.4718]])
    >>> a.unsqueeze(dim=-1)
    tensor([[0.5478],
            [0.4366],
            [0.2502],
            [0.5778],
            [0.7834],
            [0.8406],
            [0.3881],
            [0.8908],
            [0.0255],
            [0.4718]])
    >>> a.unsqueeze(dim=-2)
    tensor([[0.5478, 0.4366, 0.2502, 0.5778, 0.7834, 0.8406, 0.3881, 0.8908, 0.0255,
             0.4718]])

*  squeeze()反向操作

>>> a
    tensor([0.5478, 0.4366, 0.2502, 0.5778, 0.7834, 0.8406, 0.3881, 0.8908, 0.0255,
            0.4718])
    >>> a.squeeze()
    tensor([0.5478, 0.4366, 0.2502, 0.5778, 0.7834, 0.8406, 0.3881, 0.8908, 0.0255,
            0.4718])

把数据做一下简单的调整

>>> a
    tensor([[0.5478],
            [0.4366],
            [0.2502],
            [0.5778],
            [0.7834],
            [0.8406],
            [0.3881],
            [0.8908],
            [0.0255],
            [0.4718]])
    >>> a.squeeze()
    tensor([0.5478, 0.4366, 0.2502, 0.5778, 0.7834, 0.8406, 0.3881, 0.8908, 0.0255,
            0.4718])

先压缩一下：

>>> a = a.unsqueeze(dim=1)
    >>> a
    tensor([[[0.5478]],
    
            [[0.4366]],
    
            [[0.2502]],
    
            [[0.5778]],
    
            [[0.7834]],
    
            [[0.8406]],
    
            [[0.3881]],
    
            [[0.8908]],
    
            [[0.0255]],
    
            [[0.4718]]])
    >>> a.shape
    torch.Size([10, 1, 1])
    >>> a.squeeze()
    tensor([0.5478, 0.4366, 0.2502, 0.5778, 0.7834, 0.8406, 0.3881, 0.8908, 0.0255,0.4718])

直接就压缩到最简单的版本了。  
但是，通过下面的代码结果可以看出来，其实不是压缩到最简单的版本。估计只是考虑了中间只有1的操作

>>> a.resize_(2,5)
    tensor([[0.5478, 0.4366, 0.2502, 0.5778, 0.7834],
            [0.8406, 0.3881, 0.8908, 0.0255, 0.4718]])
    >>> a
    tensor([[0.5478, 0.4366, 0.2502, 0.5778, 0.7834],
            [0.8406, 0.3881, 0.8908, 0.0255, 0.4718]])
    >>> a.squeeze()
    tensor([[0.5478, 0.4366, 0.2502, 0.5778, 0.7834],
            [0.8406, 0.3881, 0.8908, 0.0255, 0.4718]])

只是把中间的维度为1的给全去掉了。

>>> a.resize_(2, 1, 1, 5, 1, 1)
    tensor([[[[[[0.5478]],
    
               [[0.4366]],
    
               [[0.2502]],
    
               [[0.5778]],
    
               [[0.7834]]]]],
    
    
    
    
            [[[[[0.8406]],
    
               [[0.3881]],
    
               [[0.8908]],
    
               [[0.0255]],
    
               [[0.4718]]]]]])
    >>> a.squeeze()
    tensor([[0.5478, 0.4366, 0.2502, 0.5778, 0.7834],
            [0.8406, 0.3881, 0.8908, 0.0255, 0.4718]])

### Tensor的切片操作 ###

创建数据

>>> a = torch.randn(6,6)
    >>> a
    tensor([[-0.9391,  1.4903, -1.4979,  1.4666,  0.1815, -0.6964],
            [-1.2770, -0.8761, -1.9706,  0.8806, -0.9304,  0.0181],
            [ 0.1157,  1.3184,  0.2521, -1.5565, -0.8318,  1.0100],
            [-0.2843,  0.3335, -0.1813, -1.3236,  2.2849, -0.0776],
            [-0.6731, -0.5142,  0.2758,  0.4677,  2.0181, -1.2722],
            [ 1.8404,  0.7929, -0.8389,  1.0610, -0.0790, -0.2701]])

*  简单提取和切片的区别
    
     *  后面两个为切片，第一个是直接index提取

>>> a[0]
    tensor([-0.9391,  1.4903, -1.4979,  1.4666,  0.1815, -0.6964])
    >>> a[:1]
    tensor([[-0.9391,  1.4903, -1.4979,  1.4666,  0.1815, -0.6964]])
    >>> a[0,:]
    tensor([-0.9391,  1.4903, -1.4979,  1.4666,  0.1815, -0.6964])

*  取第一行到第二行（数值从0开始取）
    
     *  看来默认是取行了。

>>> a[1:3]
    tensor([[-1.2770, -0.8761, -1.9706,  0.8806, -0.9304,  0.0181],
            [ 0.1157,  1.3184,  0.2521, -1.5565, -0.8318,  1.0100]])

*  取列的话操作类似（取第一列到第二列）

>>> a[:,1:3]
    tensor([[ 1.4903, -1.4979],
            [-0.8761, -1.9706],
            [ 1.3184,  0.2521],
            [ 0.3335, -0.1813],
            [-0.5142,  0.2758],
            [ 0.7929, -0.8389]])

*  最后一行和最后一列

>>> a[-1]
    tensor([ 1.8404,  0.7929, -0.8389,  1.0610, -0.0790, -0.2701])
    >>> a[:,-1]
    tensor([-0.6964,  0.0181,  1.0100, -0.0776, -1.2722, -0.2701])

*  取不连续的行列
    
     *  注意后面的逗号不能省去
     *  列是类似的。

>>> a
    tensor([[-0.9391,  1.4903, -1.4979,  1.4666,  0.1815, -0.6964],
            [-1.2770, -0.8761, -1.9706,  0.8806, -0.9304,  0.0181],
            [ 0.1157,  1.3184,  0.2521, -1.5565, -0.8318,  1.0100],
            [-0.2843,  0.3335, -0.1813, -1.3236,  2.2849, -0.0776],
            [-0.6731, -0.5142,  0.2758,  0.4677,  2.0181, -1.2722],
            [ 1.8404,  0.7929, -0.8389,  1.0610, -0.0790, -0.2701]])
    >>> a[(1,3),]
    tensor([[-1.2770, -0.8761, -1.9706,  0.8806, -0.9304,  0.0181],
            [-0.2843,  0.3335, -0.1813, -1.3236,  2.2849, -0.0776]])

### Tensor的比较 ###

*  Tensor > int

>>> a
    tensor([[-0.9391,  1.4903, -1.4979,  1.4666,  0.1815, -0.6964],
            [-1.2770, -0.8761, -1.9706,  0.8806, -0.9304,  0.0181],
            [ 0.1157,  1.3184,  0.2521, -1.5565, -0.8318,  1.0100],
            [-0.2843,  0.3335, -0.1813, -1.3236,  2.2849, -0.0776],
            [-0.6731, -0.5142,  0.2758,  0.4677,  2.0181, -1.2722],
            [ 1.8404,  0.7929, -0.8389,  1.0610, -0.0790, -0.2701]])
    >>> a > 0
    tensor([[0, 1, 0, 1, 1, 0],
            [0, 0, 0, 1, 0, 1],
            [1, 1, 1, 0, 0, 1],
            [0, 1, 0, 0, 1, 0],
            [0, 0, 1, 1, 1, 0],
            [1, 1, 0, 1, 0, 0]], dtype=torch.uint8)

得到的是一个非负整数的矩阵，同等规模。估计是pytorch为了更方便计算，直接用的是uint8类型。

*  显示的时候是有精度的，所以等号不太管用
    
     *  如果你还记得上面的tolist的操作的话，就理解了

>>> a
    tensor([[-0.9391,  1.4903, -1.4979,  1.4666,  0.1815, -0.6964],
            [-1.2770, -0.8761, -1.9706,  0.8806, -0.9304,  0.0181],
            [ 0.1157,  1.3184,  0.2521, -1.5565, -0.8318,  1.0100],
            [-0.2843,  0.3335, -0.1813, -1.3236,  2.2849, -0.0776],
            [-0.6731, -0.5142,  0.2758,  0.4677,  2.0181, -1.2722],
            [ 1.8404,  0.7929, -0.8389,  1.0610, -0.0790, -0.2701]])
    >>> a == -0.9391
    tensor([[0, 0, 0, 0, 0, 0],
            [0, 0, 0, 0, 0, 0],
            [0, 0, 0, 0, 0, 0],
            [0, 0, 0, 0, 0, 0],
            [0, 0, 0, 0, 0, 0],
            [0, 0, 0, 0, 0, 0]], dtype=torch.uint8)

### Tensor的数据筛选 ###

*  比如Tensor大于某个数值的才要之类的操作

>>> a[a>0]
    tensor([1.4903, 1.4666, 0.1815, 0.8806, 0.0181, 0.1157, 1.3184, 0.2521, 1.0100,
            0.3335, 2.2849, 0.2758, 0.4677, 2.0181, 1.8404, 0.7929, 1.0610])

这个跟numpy的结果类似

>>> a.numpy()[a.numpy() > 0]
    array([1.4902998 , 1.4666233 , 0.18153903, 0.8805863 , 0.01809631,
           0.11573527, 1.3184009 , 0.25213283, 1.0099975 , 0.3334596 ,
           2.2849436 , 0.2758326 , 0.46773553, 2.0181057 , 1.8403844 ,
           0.7928698 , 1.0610487 ], dtype=float32)

### Tensor其他常用函数 ###

#### index\_select(input, dim, index)函数 ####

在指定的维度上选取

>>> indices = torch.tensor([0,1])
    >>> torch.index_select(a, 1, indices)
    tensor([[-0.9391,  1.4903],
            [-1.2770, -0.8761],
            [ 0.1157,  1.3184],
            [-0.2843,  0.3335],
            [-0.6731, -0.5142],
            [ 1.8404,  0.7929]])
    >>> a
    tensor([[-0.9391,  1.4903, -1.4979,  1.4666,  0.1815, -0.6964],
            [-1.2770, -0.8761, -1.9706,  0.8806, -0.9304,  0.0181],
            [ 0.1157,  1.3184,  0.2521, -1.5565, -0.8318,  1.0100],
            [-0.2843,  0.3335, -0.1813, -1.3236,  2.2849, -0.0776],
            [-0.6731, -0.5142,  0.2758,  0.4677,  2.0181, -1.2722],
            [ 1.8404,  0.7929, -0.8389,  1.0610, -0.0790, -0.2701]])

**记住，这里的tensor不同于Tensor**

>>> indices
    tensor([0, 1])
    >>> torch.Tensor([0,1])
    tensor([0., 1.])

发现了吧，必须要是正整数。  
而且也必须是Tensor

>>> torch.index_select(a, 1, [0,1])
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: index_select(): argument 'index' (position 3) must be Tensor, not list

#### masked\_select(input, mask) ####

*  效果跟a\[a>0\]类似。
 *  只不过可以使用手动创建的更复杂的masked\_select操作。

>>> torch.masked_select(a, a>0)
    tensor([1.4903, 1.4666, 0.1815, 0.8806, 0.0181, 0.1157, 1.3184, 0.2521, 1.0100,
            0.3335, 2.2849, 0.2758, 0.4677, 2.0181, 1.8404, 0.7929, 1.0610])
    >>> a[a>0]
    tensor([1.4903, 1.4666, 0.1815, 0.8806, 0.0181, 0.1157, 1.3184, 0.2521, 1.0100,
            0.3335, 2.2849, 0.2758, 0.4677, 2.0181, 1.8404, 0.7929, 1.0610])

#### nonzero(input)操作 ####

*  返回非0点坐标

>>> a
    tensor([[1., 0., 0., 0., 0.],
            [0., 1., 0., 0., 0.],
            [0., 0., 1., 0., 0.],
            [0., 0., 0., 1., 0.],
            [0., 0., 0., 0., 1.]])
    >>> torch.nonzero(a)
    tensor([[0, 0],
            [1, 1],
            [2, 2],
            [3, 3],
            [4, 4]])

#### gather 根据index，在dim上选取数据，输出size与index一直 ####

*  其实就是一个多重映射

>>> t = torch.tensor([[1,2],[3,4]])
    >>> torch.gather(t, 1, torch.tensor([[0,0],[1,0]]))
    tensor([[1, 1],
            [4, 3]])
    >>> torch.gather(t, 0, torch.tensor([[0,0],[1,0]]))
    tensor([[1, 2],
            [3, 2]])

*  用下面的例子来区分dim=1和dim=0的区别

>>> t
    tensor([[1, 2],
            [3, 4]])
    >>> torch.gather(t, 0, torch.tensor([[0,1],[1,0]]))
    tensor([[1, 4],
            [3, 2]])
    >>> torch.gather(t, 1, torch.tensor([[0,1],[1,0]]))
    tensor([[1, 2],
            [4, 3]])

看到有这么一段公式  
3D-Tensor下的公式。（n维都是类似的）

out[i][j][k] = input[index[i][j][k]][j][k]  # if dim == 0
    out[i][j][k] = input[i][index[i][j][k]][k]  # if dim == 1
    out[i][j][k] = input[i][j][index[i][j][k]]  # if dim == 2

### clamp截断 ###

这个函数就是用来做截断的

>>> t
    tensor([[1, 2],
            [3, 4]])
    >>> torch.clamp(t, 2, 3)
    tensor([[2, 2],
            [3, 3]])

## 后记 ##

大概就这么多吧？以后再遇到再补充吧？大家有想法也可以提出来呢~

[https_blog.csdn.net_qq_34690929_article_details_80029974]: https://blog.csdn.net/qq_34690929/article/details/80029974
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L2ExOTk5MDQxMg_size_16_color_FFFFFF_t_70]: /images/20220414/c18cd9765a2a467688de48f4fb57b55e.png