Template

import torch
import torch.nn as nn
import torch.optim as optim
import torch.nn.functional as F

import torchvision
import torchvision.datasets as dset
import torchvision.transforms as T

import numpy as np

torch.autograd.set_detect_anomaly(True) # for debug stacktrace

Tutorial

Quickstart - PyTorch Tutorials 1.13.0+cu117 documentation

Learning PyTorch with Examples - PyTorch Tutorials 1.13.0+cu117 documentation

Concept

Dynamic Computation Graphs

build computation graph on the fly

• PyTorch methods that end in underscore modify the Tensor in-place(don’t return a new Tensor)

Static Computation Graphs

Caffe2: https://caffe2.ai/

Concepts

Tensor: Like a numpy array, but can run on GPU

nn.parameter.Parameter: subclass of Tensor that can be registered into Modules’ parameters() iterator when being assigned as attributes. If no any, then all Tensors attributes will be registered.

Optim

Autograd

Automatic Differentiation with torch.autograd - PyTorch Tutorials 1.13.0+cu117 documentation