A neural network is the bunch of nodes that take one or more input, processes them and returns one or more output. The data flow is done in form of tensors which we will learn in this post. Understanding tensors in PyTorch is an important and lengthy topic because while optimizing weights (or training), the backpropagation algorithm will apply some methods on these tensor objects to achieve its goal so I have divided it into two parts.
I will keep this short and to the point discussing important things only. So let's dive deep into tensors
What are Tensors
Tensors are the building blocks of a neural network. A PyTorch tensor is the generalized form of arrays in \( n \) dimensions to run arbitrary computations on GPU. [READ MORE]. A tensor can be of any dimension. In this post, you will learn about the following dimensions only
- 0D -> Scalar numbers like
20etc. are 0-Dimensional tensors
- 1D -> Array of numbers like
[1, 2, 3]or
[20, 30, 10]etc.
- 2D -> Matrix of numbers like
[[1, 2, 3], [2, 3, 4]]
Before creating and using tensors in PyTorch, install it by following the instructions from here: https://pytorch.org/get-started/locally/
You can create the tensors by importing
torch and using
To convert it into NumPy array, simply use
Tensor.numpy() method and use it as you wish
Tensor Types and Their Conversions
PyTorch supports the following data types of the tensors
The difference between
type() is that the first one will return you the type of data in the tensor and the second one will return the type of tensor. In short, dtype will give you the same name of the type as you will see in NumPy whereas the type method will provide its class name developed by the original authors of PyTorch
To explicitly create a tensor of a specific type, either pass
dtype while defining the tensor in the
torch.tensor method or use a tensor constructor. Here I am passing the integer values with
dtype=tf.float32. This will convert the integer data into floating numbers while creating tensors
Shapes and Dimensions
While training the model, you will deal with higher dimensions and the neural network only accept the dimension which is defined at the input layer while architecting the model. The dimension tells whether the tensor is 0-D or 1-D or 2-D or even higher than that. The dimension of a tensor is also called the rank of the tensor.
Sometimes working with the neural network, you will face a situation when you are supposed to change the dimension of the tensor. This can be done by using
Tensor.reshape(*shape) method of the tensor
For example, I have a tensor
[[1, 2, 3], [2, 3, 4]] and I want to convert it to 2D with 1 row and 6 columns
There are special values for the reshape and view methods. Suppose you don't know how many rows would be good while resizing your matrix, you can set it to
-1 it will automatically adjust the rows for you. The same goes for columns.
Note – Both the rows and columns can't be set to
-1 at the same time.