Preface
Logs are very important for programs and programmers. How important is it? If you want to work in a healthy company for a long time, you have to learn to do a phased stroke. One of the keys to phased strokes is to work faster than expected but pretend. . . No, this start is wrong, so start over.
Logs are very important for both programs and programmers. In addition to experience, programmers can solve problems by checking whether the log can effectively record the scene and context of the problem, etc.
In addition to making the logs in the program as accurate as possible, a competent Logger is also required to allow the program to record effective logs. A good Logger (logger) must provide the following capabilities:
- It supports writing logs to multiple output streams, such as selectively allowing the test and development environment to output logs to the console and log files at the same time, and the production environment only outputs to the files.
- Supports multiple levels of log levels, such as common ones:
TRACE，DEBUG，INFO，WARN，ERRORwait. - Support structured output, the commonly used structured output is now
JSONIn this way, this allows the unified logging platform to directly aggregate logs onto the log platform through components such as logstash. - Need to support log cutting --
log rotation, cut the logs according to date, time interval or file size. - In Log Entry (that is, records per line) in addition to the information that is actively recorded, it also includes functions that print the log, file, line number, recording time, etc.
Today I will take you to see how to create a competent Logger in a project developed using Go. Before that, let’s go back to 2009 and see the built-in Logger that Go has provided us since its inception.
Logger native to Go
The Go language comes with a built-in log package, which provides us with a default Logger that can be used directly. The detailed usage of this library can be found in the official documentation:/log
Logs are logged and will be output to the console by default. For example, the following example:
package main
import (
"log"
"net/http"
)
func main() {
simpleHttpGet("")
simpleHttpGet("")
}
func simpleHttpGet(url string) {
resp, err := (url)
if err != nil {
("Error fetching url %s : %s", url, ())
} else {
("Status Code for %s : %s", url, )
()
}
return
}
In this routine, GET requests are made to two URLs respectively, and then the return status code/request error is recorded. After executing the program, there will be similar output:
2022/05/15 15:15:26 Error fetching url : Get "": unsupported protocol scheme "" 2022/05/15 15:15:26 Status Code for : 200 OK
Because the protocol header is missing in the URL of the first request, the request cannot be initiated successfully, and the log also records the error message well.
The built-in log package of Go also supports outputting logs to files, throughCan put anyThe implementation is set to the output of the log. Next, we modify the above routine to output logs to the file.
You can try the running effect yourself, and you won’t do too many demonstrations here.
Disadvantages of Go native Logger
The advantages of native Logger are obvious, simple and out of the box, without citing external three-party libraries. We can follow the five standards for a Logger proposed at the beginning to see if the default Logger can be used in the project.
- Basic log levels only
- Only one
PrintOptions. Not supportedINFO/DEBUGetc.
- Only one
- For the error log, it has
FatalandPanic- Fatal logs are called
(1)To end the program - Panic log throws a panic after writing a log message
- But it's missing one
ERRORLog level, this level can be thrown without throwingpanicor log errors when exiting the program
- Fatal logs are called
- Lack of the ability to structured log format-only support simple text output, cannot format log records into
JSONFormat. - No log cutting capability is provided.
Zap Log Library
In the Go ecosystem, there are many log libraries that can be selected. We have briefly introduced it beforelogrusUse of this library:Click me to view, it is compatible with Go's built-in log library at the API level and directly implements itInterface, supports switching the system-level logger of the program to it.
However, logrus is not as good as performance-sensitive scenarios, and uses more Uber's open source zap log library. Because Uber has a high contribution to today's Go ecosystem, coupled with its own business - online ride-hailing performance-sensitive scenarios, Uber's open source libraries are very popular. Nowadays, many people use Zap to do logging loggers. The programmer's inner OS should be that no matter whether my concurrency is high or not, it will be done. What if one day I can suddenly get from 2 concurrency to 2W concurrency.
One of the main reasons for high performance of Zap is that there is no need to reflect, and each field to be written in the log must carry a type.
(
"Success..",
("statusCode", ),
("url", url))
A record is written to the log, and the Message is "Success...." In addition, two string key-value pairs are written. Zap For fields to be written in the log, each type has a corresponding method to convert the field intotype . for example:
('key', 123)
('key', true)
('err', err)
('arbitraryType', &User{})
There are many other methods of this type, so I won't list them one by one. This way of recording logs causes a little worse in user experience, but it is acceptable to consider the performance benefits.
Let’s first learn how to use Zap, and then make some custom configurations and encapsulations for Zap in the project to make it better. The most important thing is to match the five standards we proposed at the beginning about a good Logger.
How to use Zap
Install zap
First of all, let’s talk about the installation method of zap. Run the following command directly to download zap into the local dependency library.
go get -u /zap
Setting up Logger
Let’s first talk about the configured Logger provided by zap, and we will customize it later.
- By calling
()、()、()These three methods can create a Logger. - All the above three methods can create a Logger. They all make different configurations for Logger, such as
()When recording logs, the created Logger will automatically record the information of calling the function, the time of logging, etc., without any worries about these three, just use them.(), and when using it in the project, we will not directly use the zap configured Logger, and we need to make more detailed customization.
ZAP's Logger provides methods to record logs of different levels. For example, from low to high log levels, there are generally: Debug, Info, Warn, and Error. They are all used the same way, and the following is the method signature of the Info method.
func (log *Logger) Info(msg string, fields ...Field) {
if ce := (InfoLevel, msg); ce != nil {
(fields...)
}
}
The first parameter of the method is in the logmsgThe information to be recorded in the field,msgIt is a fixed field in the log line record. Other fields should be added to the log and passed directly.Type parameters, we have already mentioned aboveThe field of type is from("key", "value")This method is created. Since the Info method signaturefiledsParameter declarations are variable parameters, so it supports adding any multiple fields to the log line record, such as in routines:
("Success..", ("statusCode", ), ("url", url))
In the log line record, exceptmsgField, also addedstatusCode，urlTwo custom fields. Used in the above routine()The created Logger will output to the consoleJSONFormat log lines, such as those used aboveInfoAfter the method, the console will have an output similar to the following.
{"level":"info","ts":1558882294.665447,"caller":"basiclogger/:31","msg":"Success..","statusCode":"200 OK","url":""}
Customize Zap's Logger
Next, we will make further custom configuration of zap so that logs can be output not only to the console, but also to files, and then change the log time from timestamp format to a more accessible way for humans toDateTimeTime format.
Let’s talk less below, just put the code directly, and put the necessary explanations in the comments.
var logger *
func init() {
encoderConfig := ()
// Set the format of time in logging = zapcore.ISO8601TimeEncoder
// Log Encoder or JSONEncoder, format log lines into JSON format encoder := (encoderConfig)
file, _ := ("/tmp/", os.O_CREATE|os.O_APPEND|os.O_WRONLY, 644)
fileWriteSyncer = (file)
core := (
// Write logs to the console and files at the same time. Remember to write and remove the console in the production environment. The log records are basically Debug or above. Remember to change the production environment to Info. (encoder, (), ),
(encoder, fileWriteSyncer, ),
)
logger = (core)
}
Log Cutting
Zap itself does not support log cutting, and can be used to assist in cutting with another library lumberjack.
func getFileLogWriter() (writeSyncer ) {
// Use lumberjack to implement log rotate lumberJackLogger := &{
Filename: "/tmp/",
MaxSize: 100, // A maximum of 100M per file MaxBackups: 60, // Clean older logs after more than 60 log files MaxAge: 1, // Everything is cut in one day Compress: false,
}
return (lumberJackLogger)
}
Encapsulation Logger
We cannot set this way every time we use logs, so it is best to initialize these configurations in a separate package so that it can be initialized once in the project.
In addition to the above configurations, our configuration also lacks information about the log caller, such as function name, file location, line number, etc. In this way, when troubleshooting problems and looking at logs, the timeliness of positioning problems will be greatly improved.
Let's encapsulate Logger.
// Send a private message go-logger to the official account "Network Manager Daobi"// Get complete code and use demopackage zlog
// Simple encapsulation of the use of zap log library//How to use:// ("hello", ("name", "Kevin"), ("arbitraryObj", dummyObject))
// ("hello", ("name", "Kevin"), ("arbitraryObj", dummyObject))
// ("hello", ("name", "Kevin"), ("arbitraryObj", dummyObject))
var logger *
func init() {
......
}
func getFileLogWriter() (writeSyncer ) {
......
}
func Info(message string, fields ...) {
callerFields := getCallerInfoForLog()
fields = append(fields, callerFields...)
(message, fields...)
}
func Debug(message string, fields ...) {
callerFields := getCallerInfoForLog()
fields = append(fields, callerFields...)
(message, fields...)
}
func Error(message string, fields ...) {
callerFields := getCallerInfoForLog()
fields = append(fields, callerFields...)
(message, fields...)
}
func Warn(message string, fields ...) {
callerFields := getCallerInfoForLog()
fields = append(fields, callerFields...)
(message, fields...)
}
func getCallerInfoForLog() (callerFields []) {
pc, file, line, ok := (2) // Go back to the two layers and get the function information of the caller who writes the log if !ok {
return
}
funcName := (pc).Name()
funcName = (funcName) //Base function returns the last element of the path, only retaining the function name callerFields = append(callerFields, ("func", funcName), ("file", file), ("line", line))
return
}
Why not(core, ())In this way, how to add the caller's information to the log line? The main thing is to be more flexible and be able to formulate the corresponding log fields yourself, soCallerPut several information in separate fields. After collecting the logs on the log platform, it is more conducive to retrieval when querying the logs.
In the following routine, try to use our encapsulated log Logger to do a simple test.
package main
import (
"/utils/zlog"
)
type User strunct {
Name stirng
}
func main() {
user := &User{
"Name": "Kevin"
}
("test log", ("user", user))
}
The output is similar to the following output.
{"level":"info","ts":"2022-05-15T21:22:22.687+0800","msg":"test log","res":{"Name":"Kevin"},"func":"","file":"/Users/Kevin/go/src//demo/","line":84}
Summarize
Regarding the customization and packaging of Zap Logger, here are just some basic and necessary entry-level customizations. After everyone understands them, you can refer to the interface provided by the official documents to make more customizations.
Source code link/go-study-lab/go-http-server/blob/master/utils/zlog/
For more information about the custom packaging of the Go language Zap library Logger, please follow my other related articles!