Elevating Your Application Monitoring: Golang’s Logging Best Practices

Exploring the right blend of structure, correlation and simplicity

Alright, let’s talk about logging! It is like the GPS for software developers, helping us find our way when things get messy. It helps us understand what’s happening within the application, troubleshoot issues, and monitor performance. In the world of GoLang, an efficient and well-structured logging system is essential for building robust and maintainable applications.

While basic logging gets the job done, there’s a whole world of advanced logging techniques waiting to take your app monitoring to the next level. In this post, we’ll explore these techniques in GoLang. We’re talking about adding context to your logs, crafting custom log levels, diving into structured logging, and even getting started with the power of distributed tracing. Ready to embark on this logging journey? Let’s roll!

The Standard Library Log Package

GoLang comes with a built-in log package that is straightforward and easy to use. Here's how you can get started with it:

import "log"

func main() {
    log.Println("Hello, this is the standard library Logging!")
}

Third-Party Logging Libraries

While the standard log package is sufficient for simple logging needs, you may find third-party logging libraries more feature-rich and flexible for more complex applications. Some popular logging libraries in the GoLang ecosystem include:

1. Uber-Zap: Zap is a high-performance, structured logger for Go. It’s particularly well-suited for production environments and can be configured for high-performance logging.
2. Logrus: Logrus is a structured logger for Go, completely API-compatible with the standard library logger. It offers support for hooks, formatters, and log levels
3. ZeroLog: ZeroLog is a lightweight and performant logger for Go, designed for high throughput and minimal allocation, making it an excellent choice for applications with strict performance requirements.

The Uber-Zap logger provides a powerful and highly efficient solution for advanced logging. In this post, we’ll explore logging in GoLang with Uber-Zap and delve into advanced logging techniques such as log correlation, distributed tracing, and event-driven logging using code snippets.

Setting the Stage with Uber’s Zap Logging Library

Before we dive into advanced logging, let’s set up the Uber-Zap logger. First, install the package:

go get go.uber.org/zap

Now, let’s create a basic Uber-Zap logger:

package main

import (
    "go.uber.org/zap"
)

func main() {
    logger, _ := zap.NewProduction()
    defer logger.Sync()

    logger.Info("Hello, Uber-Zap Logging!")
}

Brewing an extensible logger

Creating a wrapper for the Uber-Zap logger can help you encapsulate common logging functionality and apply consistent configurations throughout your application. Below is a basic example of how to create an extensible logging interface using Uber-Zap:

package main

import (
 "context"

 "go.uber.org/zap"
)

type Fields map[string]interface{}

type Logger interface {
 Info(ctx context.Context, msg string, fields Fields)
 Error(ctx context.Context, msg string, fields Fields)
}

type logger struct {
 logger *zap.Logger
}

func InitLogger() Logger {
 zapLogger, _ := zap.NewProduction()
 return &logger{
  logger: zapLogger,
 }
}

Here, we have declared Info and Error functions which take in a log message and optional

Let’s now implement the interface:

func (l *logger) Info(ctx context.Context, msg string, fields Fields) {
  zapFields := l.getFields(ctx, fields)
  l.logger.Info(msg, zapFields...)

}

func (l *logger) Error(ctx context.Context, msg string, fields Fields) {
  zapFields := l.getFields(ctx, fields)
  l.logger.Error(msg, zapFields...)
}

func (l *logger) getFields(ctx context.Context, fields Fields) []zap.Field {
  zapFields := []zap.Field{}
  
  if fields == nil {
    return zapFields 
  }

  for key, value := range fields {
    zapFields = append(zapFields, zap.Any(key, value))
  }

 return zapFields
}

We will be using this wrapper to delve deeper into logging in the rest of this post

Advanced Logging Techniques

1. Structured Logging

Structured logging is a technique that transforms log messages into a well-defined format, typically using JSON or key-value pairs. This structured format provides several advantages for log management and analysis

1. Search and Filter: Structured logs are easily searchable and filterable.
2. Consistency: With structured logging, log messages follow a consistent format, enhancing clarity and predictability
3. Machine-Readable: Structured logs are machine-readable. Their format is designed to be processed by automated log analysis too

package main

import "context"

func main() {
    logger := InitLogger()

  ctx := context.Background()
    url := "/login"
    userId := 15

    logger.Info(ctx, "Hello, Uber-Zap Logging!",
        Fields{
        "url": url,
        "userId": userId,
        }
    )
}

2. Custom Log Levels

Custom log levels allow you to define and use log levels beyond the standard levels like “Info,” “Debug,” and “Error.” This can be particularly useful when you need to categorize log messages according to their importance and significance in your application, and also while managing access control of logs.

Let’s delve into defining custom log levels using Uber-Zap, using the wrapper interface we’ve defined

import (
    "go.uber.org/zap"
    "go.uber.org/zap/zapcore"
)

const (
    CriticalLevel = zapcore.Level(10)
    SecretLevel = zapcore.Level(100)
)

func registerCustomLevels() {
    zap.RegisterLevel("CRITICAL", zapcore.Level(CriticalLevel))
    zap.RegisterLevel("SECRET", zapcore.Level(SecretLevel))
}

We will now modify the interface we’ve earlier defined to use these custom levels:

type Logger interface {
 Info(ctx context.Context, msg string, fields Fields)
 Secret(ctx context.Context, msg string, fields Fields)
 Error(ctx context.Context, msg string, fields Fields)
 Critical(ctx context.Context, msg string, fields Fields)
}

//... logger declarations

func InitLogger() Logger {
 registerCustomLevels()

 cfg := zap.NewProductionConfig()
 customLevels := map[string]zapcore.Level{
  "CRITICAL": zapcore.Level(CriticalLevel),
  "SECRET":   zapcore.Level(SecretLevel),
 }

 cfg.Level.SetLevel(customLevels)
 zapLogger, _ := cfg.Build()

 return &logger{
  logger: zapLogger,
 }
}

Let’s now define the Secret and Critical functions:

//... implement the Info and Error methods

func (l *logger) Secret(ctx context.Context, msg string, fields Fields) {
 zapFields := l.getFields(ctx, fields)
 l.logger.WithLevel(zapcore.Level(Secret)).Info(msg, zapFields...)
}

func (l *logger) Critical(ctx context.Context, msg string, fields Fields) {
 zapFields := l.getFields(ctx, fields)
 l.logger.WithLevel(zapcore.Level(Critical)).Error(msg, zapFields...)
}

//... implement the getFields method

3. Contextual Logging

Contextual logging involves associating additional context with log entries, such as request IDs, user information, or application-specific data. This technique is invaluable for tracing and debugging requests in distributed systems and microservices.

Data such as request ids trace, inter service trace ids, url of the request, etc can be stored within the context object which is passed to all functions within the service. We can log these fields as default fields as follows:

//... rest of the wrapper implementation

func (l *logger) getFields(ctx context.Context, fields Fields) []zap.Field {
 zapFields := []zap.Field{}
  
  if fields == nil {
     fields = Fields{} 
  }

  fields["trace_id"] = ctx.Value("trace_id")
  fields["url"] = ctx.Value("url")

  for key, value := range fields {
     zapFields = append(zapFields, zap.Any(key, value))
  }

 return zapFields
}

4. Log Correlation

Log correlation is a technique that links related log entries across different components of an application or services in a distributed system. It allows you to trace the flow of a particular request or action as it moves through various parts of your system.

An easy way to correlate logs is to add a unique Trace ID to the context, and log it by default on all logs, as shown previously.

Let us look at setting this trace id on every request to a server using a middleware. We will be using the Gin framework to implement the server and the middleware:

package main

import (
    "github.com/gin-gonic/gin"
    "github.com/google/uuid"
    "go.uber.org/zap"
    "net/http"
)

func TraceIDMiddleware() gin.HandlerFunc {
    return func(ctx *gin.Context) {
        traceID := uuid.New().String()
    
    // Let us set the trace id and url in the context
        ctx.Set("trace_id", traceID)
        ctx.Set("url", ctx.FullPath())

        ctx.Next()
    }
}

func main() {
    router := gin.Default()
  logger := InitLogger()

    // Use the trace ID middleware
    router.Use(TraceIDMiddleware())

    // Define a route
    router.GET("/test", func(ctx *gin.Context) {
       logger.Info(ctx, "Hey!", nil)
    })

    router.Run(":8080")
}

The above code will initialise a server and log the url and a unique trace id along with the rest of the info whenever the logger is called.

Best Practices

While implementing the advanced logging techniques, keep these best practices in mind:

1. Log Only What You Need: Avoid over-logging, which can lead to information overload. Log only the essential data needed for monitoring and debugging.
2. Log Levels and Context: Use appropriate log levels (INFO, DEBUG, ERROR) and include context information in your logs for better traceability.
3. Error Handling: Properly handle and log errors to understand the root causes of issues.
4. Security: Be mindful of security concerns and avoid logging sensitive information like passwords or API keys.
5. Log Rotation: Implement log rotation and retention policies to manage log file sizes effectively.

Conclusion

In summary, advanced logging techniques in GoLang, including structured logging, custom log levels, and log correlation, are powerful tools for enhancing application monitoring and troubleshooting. By using third-party logging libraries like Zap, you can log in a structured format, making it easier to analyze your logs.

These techniques are invaluable, particularly in distributed systems. So, dive in and make the most of these techniques for effective, efficient, and insightful logging. Happy coding!

Stackademic

Thank you for reading until the end. Before you go:

• Please consider clapping and following the writer! 👏
• Follow us on Twitter(X), LinkedIn, and YouTube.
• Visit Stackademic.com to find out more about how we are democratizing free programming education around the world.