How to create a custom log format to include more details?
Jul 20, 2025 am 03:57 AMCustom log formats can improve log readability and analysis efficiency. The specific steps include: 1. Determine the fields to be recorded, such as client IP, request time, response time, status code, User-Agent, custom identifier, etc.; 2. Configure in different services, such as Nginx using log_format, Apache using LogFormat, Node.js using morgan middleware; 3. Pay attention to maintaining format consistency, avoiding sensitive information, adapting to log parsing tools, enabling log rotation and testing format validity.
When analyzing servers or application logs, the standard default format tends to be limited info. If you want to track requests, troubleshoot problems, or do data analysis more comprehensively, custom log formats are a practical approach. This not only makes the log more readable, but also provides structured data for subsequent log processing and monitoring tools.
Let's take a look at how to create a custom log format with more information.
1. Determine which fields you need to record
The first step is to clarify the data points you care about. Common extension fields include:
- Client IP (real IP, especially when a proxy is used)
- Request time, response time
- HTTP method, path, protocol version
- Response status code, return size
- User-Agent and Referer
- Specific fields in the request header, such as X-Forwarded-For, Authorization
- Custom identifiers such as trace ID or session ID
For example, when you debug performance issues, you may need to record the request processing time; if you pay attention to source traffic, Referer is very important.
2. Configure custom log formats in common services
Different services have different configuration methods. Here are some examples of common scenarios:
Nginx Example
Nginx supports custom log formats through the log_format
directive. For example:
log_format custom '$remote_addr - $remote_user [$time_local] "$request" ' '$status $body_bytes_sent "$http_referer" ' '"$http_user_agent" "$http_x_forwarded_for" ' 'rt=$request_time ut="$upstream_response_time"'; access_log /var/log/nginx/access.log custom;
Here we add $request_time
and $upstream_response_time
to help analyze request delays.
Apache Example
Apache uses LogFormat
and CustomLog
:
LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-agent}i\" %D %T" custom CustomLog "/var/log/apache2/access.log" custom
where %D
is the microsecond processing time and %T
is the second processing time.
Node.js/Express Example
Use middleware such as morgan
to flexibly define log formats:
const morgan = require('morgan'); app.use(morgan(':remote-addr - :remote-user [:date[clf]] ":method :url HTTP/:http-version" :status :res[content-length] ":referrer" ":user-agent" :response-time ms'));
You can also customize the token to add additional information, such as trace ID.
3. Precautions and suggestions
- Keep consistency : If you have multiple service nodes, make sure they use the same log format for easy unified processing.
- Avoid sensitive information : Do not write sensitive content such as passwords and tokens into the log.
- Consider log parsing tools : If you are using log systems such as ELK, Fluentd, Loki, etc., it is best to make the log format easily parsed by these tools.
- Compression and rotation : Turn on logrotate to prevent excessive log files from affecting performance.
- Test your format : After modifying the format, manually trigger several requests to see if the output meets expectations.
Basically that's it. The custom log format itself is not complicated, but to truly bring out its value, the key lies in what field you choose and how to use it to assist in troubleshooting and analysis. As long as you set it reasonably according to your business needs, you can greatly improve the value of the log.
The above is the detailed content of How to create a custom log format to include more details?. For more information, please follow other related articles on the PHP Chinese website!

Hot AI Tools

Undress AI Tool
Undress images for free

Undresser.AI Undress
AI-powered app for creating realistic nude photos

AI Clothes Remover
Online AI tool for removing clothes from photos.

Clothoff.io
AI clothes remover

Video Face Swap
Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Article

Hot Tools

Notepad++7.3.1
Easy-to-use and free code editor

SublimeText3 Chinese version
Chinese version, very easy to use

Zend Studio 13.0.1
Powerful PHP integrated development environment

Dreamweaver CS6
Visual web development tools

SublimeText3 Mac version
God-level code editing software (SublimeText3)

Hot Topics

The main Nginx configuration file is usually located in the conf directory under /etc/nginx/nginx.conf (Ubuntu/Debian, CentOS/RHEL), /usr/local/etc/nginx/nginx.conf (macOSHomebrew) or the source code installation path; you can view the loaded configuration path through nginx-t, ps-ef|grepnginx check the path specified by the startup parameters, or use find/-namenginx.conf and locatenginx.conf to quickly find; the configuration file structure includes global settings, events blocks and http blocks, and common site configurations are common.

When Nginx experiences a "Toomyopenfiles" error, it is usually because the system or process has reached the file descriptor limit. Solutions include: 1. Increase the soft and hard limits of Linux system, set the relevant parameters of nginx or run users in /etc/security/limits.conf; 2. Adjust the worker_connections value of Nginx to adapt to expected traffic and ensure the overloaded configuration; 3. Increase the upper limit of system-level file descriptors fs.file-max, edit /etc/sysctl.conf and apply changes; 4. Optimize log and resource usage, and reduce unnecessary file handle usage, such as using open_l

Enabling Gzip compression can effectively reduce the size of web page files and improve loading speed. 1. The Apache server needs to add configuration in the .htaccess file and ensure that the mod_deflate module is enabled; 2.Nginx needs to edit the site configuration file, set gzipon and define the compression type, minimum length and compression level; 3. After the configuration is completed, you can verify whether it takes effect through online tools or browser developer tools. Pay attention to the server module status and MIME type integrity during operation to ensure normal compression operation.

The stub_status module displays the real-time basic status information of Nginx. Specifically, it includes: 1. The number of currently active connections; 2. The total number of accepted connections, the total number of processing connections, and the total number of requests; 3. The number of connections being read, written, and waiting. To check whether it is enabled, you can check whether the --with-http_stub_status_module parameter exists through the command nginx-V. If not enabled, recompile and add the module. When enabled, you need to add location blocks to the configuration file and set access control. Finally, reload the Nginx service to access the status page through the specified path. It is recommended to use it in combination with monitoring tools, but it is only available for internal network access and cannot replace a comprehensive monitoring solution.

The "Addressalreadyinuse" error means that another program or service in the system has occupied the target port or IP address. Common reasons include: 1. The server is running repeatedly; 2. Other services occupy ports (such as Apache occupying port 80, causing Nginx to fail to start); 3. The port is not released after crash or restart. You can troubleshoot through the command line tool: use sudolsof-i:80 or sudolnetstat-tulpn|grep:80 in Linux/macOS; use netstat-ano|findstr:80 in Windows and check PID. Solutions include: 1. Stop the conflicting process (such as sudos

The main difference between NginxPlus and open source Nginx is its enhanced functionality and official support for enterprise-level applications. 1. It provides real-time monitoring of the dashboard, which can track the number of connections, request rate and server health status; 2. Supports more advanced load balancing methods, such as minimum connection allocation, hash-based consistency algorithm and weighted distribution; 3. Supports session maintenance (sticky sessions) to ensure that user requests are continuously sent to the same backend server; 4. Allow dynamic configuration updates, and adjust upstream server groups without restarting the service; 5. Provides advanced cache and content distribution functions to reduce backend pressure and improve response speed; 6. Automatic configuration updates can be achieved through APIs to adapt to Kubernetes or automatic scaling environments; 7. Includes

The method to enable HSTS is to configure the Strict-Transport-Security response header in the HTTPS website. The specific operations are: 1.Nginx adds the add_header directive in the server block; 2.Apache adds the header directive in the configuration file or .htaccess; 3.IIS adds customHeaders in web.config; it is necessary to ensure that the site fully supports HTTPS, parameters include max-age (valid period), includeSubDomains (subdomains are effective), preload (preload list), and the prereload is the prerequisite for submitting to the HSTSPreload list.

A/B testing can be implemented through Nginx's split_clients module, which distributes traffic proportionally to different groups based on user attribute hashing. The specific steps are as follows: 1. Use the split_clients instruction to define the grouping and proportions in the http block, such as 50%A and 50%B; 2. Use variables such as $cookie_jsessionid, $remote_addr or $arg_uid as hash keys to ensure that the same user is continuously allocated to the same group; 3. Use the corresponding backend through if conditions in the server or location block; 4. Record the grouping information through a custom log format to analyze the effect; 5. Track the performance of each group with the monitoring tool
