pathterminuspages/machine learning/aboutcontactabout me


20.12.2020 | PyTorch/nn


@1. Linear

nn.Linear applies a linear transformation to the input vector $x$ obtaining $$ y = x A^{\top} + b $$ The transformation is done based on the weight parameter (matrix) $A$ and the bias (vector) $b$. Both these are initialized randomly. We can illustrate with the code

import torch as ts import torch.nn as nn f = nn.Linear(5,3) x = ts.tensor([float(x) for x in range(0,5)]) y = f(x) A = f.weight b = f.bias y0 = A @ x + b # these two are the same print(y) print(y0)

This function is often used to initialize a map/hidden layer that can be learned on.

CommentsGuest Name:Comment: