Introduction to Tensors in Pytorch #1
Tensors are the building block of neural networks and are used to represent the data in form of numbers. In this post you will learn the basics of tensors like declaring, types and shapes
A neural network is the bunch of nodes that take one or more input, processes them and returns one or more output. The data flow is done in form of tensors which we will learn in this post. Understanding tensors in PyTorch is an important and lengthy topic because while optimizing weights (or training), the backpropagation algorithm will apply some methods on these tensor objects to achieve its goal so I have divided it into two parts.
I will keep this short and to the point discussing important things only. So let's dive deep into tensors
What are Tensors
Tensors are the building blocks of a neural network. A PyTorch tensor is the generalized form of arrays in \( n \) dimensions to run arbitrary computations on GPU. [READ MORE]. A tensor can be of any dimension. In this post, you will learn about the following dimensions only
- 0D -> Scalar numbers like
0
,10
,20
etc. are 0-Dimensional tensors - 1D -> Array of numbers like
[1, 2, 3]
or[20, 30, 10]
etc. - 2D -> Matrix of numbers like
[[1, 2, 3], [2, 3, 4]]
Before creating and using tensors in PyTorch, install it by following the instructions from here: https://pytorch.org/get-started/locally/
Creating Tensors
You can create the tensors by importing torch
and using torch.tensor(data)
method.
To convert it into NumPy array, simply use Tensor.numpy()
method and use it as you wish
Tensor Types and Their Conversions
PyTorch supports the following data types of the tensors
To get the data type used Tensor.dtype
property and Tensor.type()
to get the PyTorch tensor type
The difference between dtype
and type()
is that the first one will return you the type of data in the tensor and the second one will return the type of tensor. In short, dtype will give you the same name of the type as you will see in NumPy whereas the type method will provide its class name developed by the original authors of PyTorch
To explicitly create a tensor of a specific type, either pass dtype
while defining the tensor in the torch.tensor
method or use a tensor constructor. Here I am passing the integer values with dtype=tf.float32
. This will convert the integer data into floating numbers while creating tensors
You can pass in the datatype in the Tensor.type(<new data type>)
or Tensor.type_as(<new data type>)
method, to change the data type of the existing tensor.
Shapes and Dimensions
While training the model, you will deal with higher dimensions and the neural network only accept the dimension which is defined at the input layer while architecting the model. The dimension tells whether the tensor is 0-D or 1-D or 2-D or even higher than that. The dimension of a tensor is also called the rank of the tensor.
To get the size you can use Tensor.size()
method or Tensor.shape
property and to get the dimension of the tensor, use Tensor.ndimension()
method or Tensor.ndim
property.
Reshaping Tensors
Sometimes working with the neural network, you will face a situation when you are supposed to change the dimension of the tensor. This can be done by using Tensor.view(*shape)
or Tensor.reshape(*shape)
method of the tensor
For example, I have a tensor [[1, 2, 3], [2, 3, 4]]
and I want to convert it to 2D with 1 row and 6 columns
There are special values for the reshape and view methods. Suppose you don't know how many rows would be good while resizing your matrix, you can set it to -1
it will automatically adjust the rows for you. The same goes for columns.
Note – Both the rows and columns can't be set to -1
at the same time.