I. What is argparse?
argparse is a standard Python library for parsing command-line arguments, which means we don't have to manually assign values to variables in our code.
Instead, you can pass the appropriate parameters directly to the program on the command line, and then have the variables read those parameters.
If you don't have theargparseIf you want to install it, you can do so by executing the following command
pip install argparse
To use it, simply import the
import argparse
1.1 An example
Let's look at the simplest example of understanding the use of theargparse The details of each API are described in more detail after the general steps of the
""" Solve for the sum of two numbers """
twoSum = lambda x, y: x + y
parser = ()
parser.add_argument('--a', type=int, required=True, help="first number")
parser.add_argument('--b', type=int, required=True, help="second number")
args = parser.parse_args()
first_num, second_num = ,
print(twoSum(first_num, second_num))Save the above to in. Let's start by executing the command line
python3 -h
You can see the corresponding help information
usage: [-h] --a A --b B options: -h, --help show this help message and exit --a A first number --b B second number
through (a gap)usage it can be seen Receive two mandatory options:--a cap (a poem)--b(with)[] are optional, and those without are mandatory), which represent the first and second numbers, respectively, whereA cap (a poem)B represent the actual incoming parameters, respectively.
Execute it on the command line
python3 --a 12 --b 19
The return result is31This means that we have done the operation of solving for the sum of two numbers via the command line.
II. ArgumentParser
utilizationargparse The first step is to create aArgumentParser object, which contains all the information needed to parse the command line into Python datatypes, with the following common parameters
(prog=None, usage=None, description=None, epilog=None)
2.1 prog
prog The default value is([0]), which is the name of the file where the program is located.
For example, in the first chapter, we created theArgumentParser object without specifying theprogThe default value is therefore used。
Let's start with an example
parser = () parser.print_help() # and on the command line by invoking the python3 -h the same effect as the（There will be some nuances.）
Helpful information about the output:
usage: [-h] options: -h, --help show this help message and exit
You can see that if you don't specifyprogThe help message will be displayed as the program name. Now specifyprog
parser = (prog="My Program")
corresponding help information:
usage: My Program [-h] options: -h, --help show this help message and exit
You can see that the original The place becameMy Program。
2.2 usage
By default, theArgumentParser Constructs usage messages based on the options it contains.
The example from Chapter 1 is still used here:
usage: [-h] --a A --b B options: -h, --help show this help message and exit --a A first number --b B second number
Because we didn't specifyusageSo.ArgumentParser will use the three options it contains:-h、--a、--b to build usage messagesusageIt is located on the first line of the help message.
If you feel that the defaultusage There is some redundancy (because the loweroptions (the individual options have been described in detail), we can customize theusage
parser = (usage="See the options below for usage")
The corresponding help message becomes
usage: See the options below for usage options: -h, --help show this help message and exit --a A first number --b B second number
It is important to note that after specifying theusage Later.prog will be overwritten, even if theArgumentParser is specified in theprog It didn't work either.
2.3 description
description parameter is used to briefly describe what this program does and how it does it. Not specifying thedescription The help message will not be displayed.
Still using the example from the first chapter, here we specify that thedescription
parser = (description="This is my program.")
Corresponding help information
usage: [-h] --a A --b B This is my program. options: -h, --help show this help message and exit --a A first number --b B second number
can be seenusage cap (a poem)options There's an extra line in the middle, which is what we specified for thedescription。
2.4 epilog
This parameter is the same as thedescription Similarly, the difference is thatdescription Put it in.options before, andepilog Put it in.options After.
Still using the example from the first chapter, here we specify both thedescription cap (a poem)epilog
parser = (description="This is my program.", epilog="The end.")
Corresponding help information
usage: [-h] --a A --b B This is my program. options: -h, --help show this help message and exit --a A first number --b B second number The end.
Generally speaking, the most commonly used of the above four parameters is the
prog
III. add_argument
add_argument() method is used to add an option (positional parameter) to the parser.
ArgumentParser.add_argument( name or flags..., nargs, default, type, choices, required, help )
The above only lists theadd_argument() A few of the most commonly used parameters in the method.
3.1 name or flags
name or flags For the option (options) or positional parameters (positional arguments）。
If it's an option you can pass in a series of flags (e.g. the help that comes with it has two:-h、--help), or in the case of a positional parameter only aname。
for example
parser = ()
parser.add_argument('-i', '--install')
args = parser.parse_args()corresponding help information:
usage: [-h] [-i INSTALL] options: -h, --help show this help message and exit -i INSTALL, --install INSTALL
This means that our command line call topython3 -i INSTALL cap (a poem)python3 --install INSTALL is equivalent.
Unlike the options, the position parameter cannot be preceded by the-and can only be passed in one at a time, e.g.
parser = ()
parser.add_argument('param1')
parser.add_argument('param2')
args = parser.parse_args()corresponding help information:
usage: [-h] param1 param2 positional arguments: integer options: -h, --help show this help message and exit
through (a gap)usage You can see that the position parameter is mandatory to pass in when calling the command line.
Above is to add the location parameter firstparam1 re-addparam2 If we switch the order, the position of the two parameters in the help message will also be switched, which also explains the meaning of "position".
As you can see, the option and position parameters, the former being equivalent to the keyword passing parameter and the latter to the position passing parameter.
3.2 type & default
As the name suggests.type refers to the type of data that the option or positional parameter will be converted to (parameters passed in on the command line default tostr (type present).
for example
parser = ()
parser.add_argument('--a')
parser.add_argument('--b', type=int)
args = parser.parse_args()
print(type(), type())fulfillmentpython3 --a 3 --b 3 After getting the result
<class 'str'> <class 'int'>
default refers to the default value of an option or location parameter, such as
parser = ()
parser.add_argument('--a', type=int, default=5)
args = parser.parse_args()
print()direct executionpython3 will output5, because default values are used.
If the execution of thepython3 --a x then it will outputx（x (is any integer and cannot be omitted).
If there is no provision for
--aSpecify the default value, and there is no default value to the--apass the reference, thenbecause ofNone。
3.3 required & help
Because the location parameter is required to be passed in on the command line, therequired Can only be used for options.
required set up asTrue means that this option is mandatory, otherwise it is optional, and the default isFalse。
for example
parser = ()
parser.add_argument('--a')
args = parser.parse_args()At this point the help message is
usage: [-h] [--a A] options: -h, --help show this help message and exit --a A
usage first line--a A bracketed by a pair of square brackets[] bracketed to show that--a is optional. Now specify therequired=True
parser.add_argument('--a', required=True)At this point the help message changes to
usage: [-h] --a A options: -h, --help show this help message and exit --a A
can be seen[] It's gone. That means--a It's become a must-have option.
help Used to describe an option or location parameter, the description will be displayed in the help message
parser = ()
parser.add_argument('--lr', type=float, default=1e-3, help="learning rate")
args = parser.parse_args()Corresponding help information
usage: [-h] [--lr LR] options: -h, --help show this help message and exit --lr LR learning rate
3.4 nargs & choices
supposing that the option--a need to receive 5 parameters, then you need to usenargs to specify
parser = ()
parser.add_argument('--a', type=int, nargs=5)
args = parser.parse_args()
print()fulfillmentpython3 --a 1 2 3 4 5
available
[1, 2, 3, 4, 5]
Attention is required.nargs=1 You end up with a list containing only one element, not the element itself.
Further.nargs='?' representing the number of incoming parameters as 0 or 1.nargs='+' representing the number of incoming parameters at least 1.nargs='*' The delegate can pass in any number of parameters.
There are times when the option--a can only take on a fixed number of values, such as--a can only be selected from the integers 1, 3, and 5, which is when you need to use thechoices to specify a list
parser = ()
parser.add_argument('--a', type=int, choices=[1, 3, 5])
args = parser.parse_args()Corresponding help information
usage: [-h] [--a {1,3,5}]
options:
-h, --help show this help message and exit
--a {1,3,5}in the event that--a If the number that follows is not one of 1, 3, or 5, an error will be reported.
IV. parse_args
Let's look at an example first.
import argparse
import sys
parser = ()
parser.add_argument('--a', type=int)
parser.add_argument('--b', type=int)
print()At the command line, executepython3 --a 3 --b 5 get the result
['', '--a', '3', '--b', '5']
As can be seen from this.[0] is the name of the file.[1:] is the option we passed in on the command line.
As you may have noticed during your previous studies, every time we finish adding the appropriate option/position parameter to the parser, we have to execute theparser.parse_args()。
By default, theparse_args() adoption[1:] as its argument and returns a namespace (similar to a dictionary).
give an example
parser = ()
parser.add_argument('--a', type=int, nargs=3)
parser.add_argument('--b', type=str)
parser.add_argument('--c', type=float)
args = parser.parse_args()
print(type(args))
print(args)fulfillmentpython3 --a 1 3 5 --b 'k' --c 3.14 capture
<class ''>
Namespace(a=[1, 3, 5], b='k', c=3.14)
If only thepython3 --a 1 3 5 --b 'k'and then we get
<class ''>
Namespace(a=[1, 3, 5], b='k', c=None)
As can be seen, if no corresponding option is provided on the command line and there is no default value for the option, the value of the option in the namespace isNoneThis is a point we made back in section 3.2.
parser.parse_args() returns a namespace object, which we usually use withargs to store.
To accessargs middle buttonk corresponding valuevJust Ready to go.
V. Avoiding error reporting
in implementingargs = parser.parse_args() In this step, an error may be reported, for example
parser = ()
parser.add_argument('--a', type=int)
args = parser.parse_args()If we run the command linepython3 --a 'abc' This is because strings cannot be converted to integers.
usage: [-h] [--a A] : error: argument --a: invalid int value: 'abc'
Generally we will use the following code block to avoid seeing unfriendly errors straight away
try:
args = parser.parse_args()
except:
parser.print_help()
(0)VI. Using shell scripts for parameterization
Deep learning often requires tuning parameters, and if you use the IDE directly to open the.py File de-tuning is inevitably a bit clunky and can become poorly maintained.
If you use theargparseAlthough you don't have to modify the.py file, but it's a bit of a hassle to change it over and over again on the command line, so it's time to combine it with a shell script.
For simplicity, let's assume that our project is structured as follows:
myproject
├── __init__.py
├── model
│ ├──
│ ├──
│ └──
├── scripts
│ └──
├──
└── utils
├──
└──
included among these The contents of the
import argparse
import sys
# Import other packages...
# Assume there are only two hyperparameters to tune
parser = ()
parser.add_argument('--bs', type=int, default=128, help="batch size")
parser.add_argument('--lr', type=float, default=0.001, help="learning rate")
try:
args = parser.parse_args()
except:
parser.print_help()
(0)
# Hyperparameter settings
BATCH_SIZE =
LEARNING_RATE =
# Other codes...At this point we just need to add the script directoryscripts Create a new file underThe following is a list of the contents of this file.
# Used to ensure that wherever the script is executed, the directory in which it is located is returned correctly, so that it can be used to locate the relative position of the program to be run. cd "$(dirname $0)" python3 ../ \ --bs 256 \ --lr 0.005 \
Then execute it on the command line (assuming it is currently in themyproject (under the table of contents)
cd scripts && bash
You can start training. Subsequently, if you need to adjust the parameter, modify the The number in the
This article on the Python command-line parameter parsing of the argparse module is introduced to this article, more related to Python's argparse module content, please search for my previous posts or continue to browse the following related articles I hope you will support me in the future more!