ResNet network construction and model training using Pytorch

network building

ResNet consists of a series of stacked residual blocks whose main function is to make the network more powerful by increasing its depth without limit. Before we build the ResNet model, let's define 4 layers, each consisting of multiple residual blocks. The purpose of these layers is to reduce the spatial size while increasing the number of channels.

Using ResNet50 as an example, we can use the following code to define a ResNet network:

class ResNet():
    def __init__(self, num_classes=1000):
        super().__init__()
        self.conv1 = nn.Conv2d(3, 64, kernel_size=7, stride=2, padding=3, bias=False)
        self.bn1 = nn.BatchNorm2d(64)
         = (inplace
（replenish）
That is, the model needs to add some input layers to the normalization and activation layer。
```python
import  as init
class Flatten():
    def __init__(self):
        super().__init__()
    def forward(self, x):
        return ((0), -1)
class ResNet():
    def __init__(self, num_classes=1000):
        super().__init__()
        self.conv1 = nn.Conv2d(3, 64, kernel_size=7, stride=2, padding=3, bias=False)
        self.bn1 = nn.BatchNorm2d(64)
         = (inplace=True)
        self.layer1 = (
            ResidualBlock(64, 256, stride=1),
            *[ResidualBlock(256, 256) for _ in range(1, 3)]
        )
        self.layer2 = (
            ResidualBlock(256, 512, stride=2),
            *[ResidualBlock(512, 512) for _ in range(1, 4)]
        )
        self.layer3 = (
            ResidualBlock(512, 1024, stride=2),
            *[ResidualBlock(1024, 1024) for _ in range(1, 6)]
        )
        self.layer4 = (
            ResidualBlock(1024, 2048, stride=2),
            *[ResidualBlock(2048, 2048) for _ in range(1, 3)]
        )
         = nn.AdaptiveAvgPool2d((1, 1))
         = Flatten()
         = (2048, num_classes)
        for m in ():
            if isinstance(m, nn.Conv2d):
                init.kaiming_normal_(, mode="fan_out", nonlinearity="relu")
            elif isinstance(m, (nn.BatchNorm2d, )):
                init.constant_(, 1)
                init.constant_(, 0)
    def forward(self, x):
        x = self.conv1(x)
        x = self.bn1(x)
        x = (x)
        x = self.layer1(x)
        x = self.layer2(x)
        x = self.layer3(x)
        x = self.layer4(x)
        x = (x)
        x = (x)
        x = (x)
        return x

Improvement points are listed below:

We usecomponent that combines multiple residual blocks into a single functional block (layer). This allows for easy modification of the network depth and separates it from the other layers ix easier to get started, for example when retraining the top classifier in migration learning.
We added normalization and activation functions to the output layer of ResNet. They help to increase the convergence speed of the model and improve the performance.
insofar asnn.Conv2dand neural network components such as batch normalization layers, we use the built-in initialization functions in PyTorch. They automatically set up the parameters for each layer for us.
We also added aFlattenlayer, which spreads the 4-dimensional output into a 2-dimensional tensor for classification by the next fully connected layer.

training model

We have now implemented the ResNet50 model, and next we will explain how to train and test the model.

First we need to define the loss function and optimizer. Here, we use the cross-entropy loss function, and the Adam optimizer.

import  as optim
device = ("cuda" if .is_available() else "cpu")
model = ResNet(num_classes=1000).to(device)
criterion = ()
optimizer = ((), lr=0.001)

When training with PyTorch, we typically create a loop that calculates the loss and updates the model parameters for each batch of input data. Below is the code for that loop:

def train(model, optimizer, criterion, train_loader, device):
    ()
    train_loss = 0
    correct = 0
    total = 0
    for batch_idx, (inputs, targets) in enumerate(train_loader):
        inputs, targets = (device), (device)
        optimizer.zero_grad()
        outputs = model(inputs)
        loss = criterion(outputs, targets)
        ()
        ()
        train_loss += ()
        _, predicted = (1)
        total += (0)
        correct += (targets).sum().item()
    acc = 100 * correct / total
    avg_loss = train_loss / len(train_loader)
    return acc, avg_loss

In the training loop above, we first pass the()The delegate enters the training mode. Then use theoptimizer.zero_grad()removals

Above is the use of Pytorch to realize the details of ResNet network construction and model training, more information about Pytorch ResNet construction network model training please pay attention to my other related articles!