An example of implementing a LeNet network to learn how to build a neural network using pytorch.
The structure of the LeNet network is shown below.
I. Using Classes to Build Network Models
To build our own network model, we need to create a new class, have it inherit from the class, and must override the __init__() and forward() functions in the Module class.
init() function is used to assert the definition of the layers in the model, and the forward() function is used to describe the connectivity between the layers and define the process of forward propagation computation.
That is, the __init__() function is only used to define layers, but not to connect them, and the forward() function serves to connect these defined layers into a network.
The code to implement the LeNet network using the above method is as follows.
import as nn class LeNet(): def __init__(self): super().__init__() self.C1 = nn.Conv2d(1, 6, 5) = () self.S2 = nn.MaxPool2d(2, 2) self.C3 = nn.Conv2d(6, 16, 5) self.S4 = nn.MaxPool2d(2, 2) self.C5 = nn.Conv2d(16, 120, 5) self.C6 = (120, 84) self.C7 = (84, 10) def forward(self, x): x1 = self.C1(x) x2 = (x1) x3 = self.S2(x2) x4 = self.C3(x3) x5 = (x4) x6 = self.S4(x5) x7 = self.C5(x6) x8 = self.C6(x7) y = self.C7(x8) return y net = LeNet() print(net)
prove
In the __init__() function, pytorch-encapsulated classes like (), nn.Conv2d() are instantiated to define network layers such as fully-connected layers, convolutional layers, etc., and to specify their parameters.
For example, self.C1 = nn.Conv2d(1, 6, 5) means defining a convolutional layer that has a convolutional kernel with an input channel of 1, an output channel of 6, and a size of 5 × 5.
The real input data to this convolutional layer is in the forward() function. x1 = self.C1(x) means that the input x is fed to the convolutional layer and the output x1 is obtained.
II. Introduction of operations at the realization layer
Introducing a function in the module simplifies the content in the __init__() function.
In the __init__() function, we can define only layers with parameters that need to be learned, such as convolutional layers and linear layers, whose weights need to be learned.
For layers that don't need to learn parameters, we don't need to define them in the __init__() function, we just need to introduce calls to the relevant functions in the class in the forward() function.
For example, in LeNet, we only define the convolutional and fully connected layers in __init__(). The pooling layer and activation function only need to be implemented in the forward() function by calling the function in.
import as nn import as F class LeNet(): def __init__(self): super().__init__() self.C1 = nn.Conv2d(1, 6, 5) self.C3 = nn.Conv2d(6, 16, 5) self.C5 = nn.Conv2d(16, 120, 5) self.C6 = (120, 84) self.C7 = (84, 10) def forward(self, x): x1 = self.C1(x) x2 = (x1) x3 = F.max_pool2d(x2) x4 = self.C3(x3) x5 = (x4) x6 = F.max_pool2d(x5) x7 = self.C5(x6) x8 = self.C6(x7) y = self.C7(x8) return y net = LeNet() print(net)
The result of the run is
Of course, in also implements layers that require learning parameters, including the convolutional layer conv2d() and the linear layer linear(), but pytorch officially recommends that we only use the functions in for layers that do not require learning parameters.
For a layer, the difference between using an implementation and using a () implementation is:
is a class, inherited from, so internally there will be many properties and methods such as train(), eval(),load_state_dict, state_dict, etc.
() is simply a function. As a class, it needs to be instantiated and passed parameters, and then fed input data into the instantiated object as a function call.
conv = nn.Conv2d(in_channels, out_channels, kernel_size, padding) output = conv(input)
() is called with both input data and setup parameters passed in.
output = .conv2d(input, weight, bias, padding)
There is no need to define and manage your own weights, but () requires you to define your own weights and pass them in manually each time you call it.
III. Sequential classes
1. Basic use
The Sequential class inherits from the Module class. For a simple sequential model, instead of writing an additional class to inherit from the Module class, you can use the Sequential class provided by pytorch to package several layers or submodules directly into one large module.
For example, in LeNet, we can easily build a neural network by directly arranging the layers in order and wrapping them in the Sequential class.
import as nn net = ( nn.Conv2d(1, 6, 5), (), nn.MaxPool2d(2, 2), nn.Conv2d(6, 16, 5), (), nn.MaxPool2d(2, 2), nn.Conv2d(16, 120, 5), (120, 84), (84, 10) ) print(net) print(net[2]) # Layers are available through the index
The result of the run is
The above method does not assign a name to each layer, but uses the index number of the layer, 0, 1, or 2, by default. We can use the index value to get the information of the corresponding layer directly.
Of course, we can also assign a name to the layer, but we don't get the layer by name, we still have to use the index number if we want to get the layer.
import as nn
from collections import OrderedDict
net = (OrderedDict([
('C1', nn.Conv2d(1, 6, 5)),
('Sig1', ()),
('S2', nn.MaxPool2d(2, 2)),
('C3', nn.Conv2d(6, 16, 5)),
('Sig2', ()),
('S4', nn.MaxPool2d(2, 2)),
('C5', nn.Conv2d(16, 120, 5)),
('C6', (120, 84)),
('C7', (84, 10))
]))
print(net)
print(net[2]) # Layers are available through the index
The result of the run is
Layers can also be added to Sequential() using the add_module function.
import as nn
net = ()
net.add_module('C1', nn.Conv2d(1, 6, 5))
net.add_module('Sig1', ())
net.add_module('S2', nn.MaxPool2d(2, 2))
net.add_module('C3', nn.Conv2d(6, 16, 5))
net.add_module('Sig2', ())
net.add_module('S4', nn.MaxPool2d(2, 2))
net.add_module('C5', nn.Conv2d(16, 120, 5))
net.add_module('C6', (120, 84))
net.add_module('C7', (84, 10))
print(net)
print(net[2])
outputs
2. Using the Sequential class to wrap layers into submodules
The Sequential class can also be applied to the methods of a custom Module class to wrap several layers into one large layer (block).
Of course there are still three ways to use Sequential, so we'll just use the first one as an example.
import as nn class LeNet(): def __init__(self): super().__init__() = ( nn.Conv2d(1, 6, 5), (), nn.MaxPool2d(2, 2), nn.Conv2d(6, 16, 5), (), nn.MaxPool2d(2, 2) ) = ( nn.Conv2d(16, 120, 5), (120, 84), (84, 10) ) def forward(self, x): x1 = (x) y = (x1) return y net = LeNet() print(net)
outputs
IV. ModuleList and ModuleDict classes
The ModuleList class and ModuleDict class are both subclasses of the Modules class, and similarly to Sequential, it allows for a number of layers or submodules to be packaged and tabulated to construct a network.
But unlike the Sequential class, these two classes just define and arrange these layers into a List or Dictionary, but do not connect them, i.e., they do not implement the forward() function.
Therefore, these two classes do not require the input and output dimensions of neighboring layers to match, nor can they directly feed input data directly into ModuleList and ModuleDict.
ModuleList is accessed in a similar way to a normal List.
net = ([
(784, 256),
()
])
((256, 20)) # ModuleList can be appended like a normal List.
print(net[-1]) # ModuleList access methods are also similar to List
print(net)
# X = (1, 784)
# net(X) # Error. Entering data into the ModuleList will be an error because the ModuleList is only meant to store the
# the modules of the network, but does not connect them, i.e. does not implement forward()
outputs
ModuleDict is also used in a similar way to a regular dictionary.
net = ({
'linear': (784, 256),
'act': (),
})
net['output'] = (256, 10) # Add
print(net['linear']) # Access
print()
print(net)
# net((1, 784)) # will report NotImplementedError
outputs
ModuleList and ModuleDict are used to provide more flexibility in defining forward propagation. Here is an example of ModuleList usage from the official website.
class MyModule():
def __init__(self):
super(MyModule, self).__init__()
= ([(10, 10) for i in range(10)])
def forward(self, x):
# ModuleList can act as an iterable, or be indexed using ints
for i, l in enumerate():
x = [i // 2](x) + l(x)
return x
In addition, in ModuleList and ModuleDict, the parameters of all submodules are automatically added to the neural network, which is different from the normal List and Dict.
An example.
class Module_ModuleList():
def __init__(self):
super(Module_ModuleList, self).__init__()
= ([(10, 10)])
class Module_List():
def __init__(self):
super(Module_List, self).__init__()
= [(10, 10)]
net1 = Module_ModuleList()
net2 = Module_List()
print("net1:")
for p in ():
print(())
print("net2:")
for p in ():
print(p)
outputs
V. Inputting data into the model
Assuming that we input data into the model as INPUT and the forward propagation result from the model is OUTPUT, the method of inputting the data is
output = net(input)
net is the name of the object, and we have completed the forward propagation calculation by passing the input directly as an argument to the object name without showing a call to the forward() function.
The above writeup is actually equivalent to
output = (input)
This is because in the class, the __call__() function is defined, which includes a call to the forward() method.
The __call__() method in the python syntax allows an instance of a class to be used as an "object name()" as if it were a normal function, and executes the contents of the __call__() function body.
summarize
The above is a personal experience, I hope it can give you a reference, and I hope you can support me more.