twobee - Fotolia
Logs can produce massive amounts of data, so IT professionals must construct a logging strategy focused on business goals to get the most from logs and log analytics.
Collecting all the possible data from every system, service or application in an environment might not be an effective logging strategy for every context. An organization that keeps everything will consume enormous amounts of storage, and much of that log data might not even provide benefits to the business.
Build a logging strategy with business objectives in mind
It's certainly possible to gather and retain logs across all the system components, but IT professionals should only commit to this logging strategy if it adds tangible value to the business. For example, if the goal is to monitor and enhance application performance or user experience, it might be worthwhile to collect and aggregate data that includes network latency, database transaction performance and webpage loading times.
An effective logging strategy starts with a review of the log information that is important to the business. Decide which logs to keep, what information those logs should contain, where to retain and protect those log files, and what tools to use for search and analytics. These decisions will all contribute to more meaningful logs and a more efficient logging strategy.
If a log isn't easy to read and understand, it can't provide useful information to the enterprise. Select a common log format that is suitable for the log aggregation and analysis tools, as well as for the IT staff that must ultimately read the log entries.
Store and aggregate logs
Logs must be stored, but it's an important part of a logging strategy to select an appropriate location for storage. In most cases, all the logs should be stored in a single, central, dedicated location that is separate from the production environment. Organizing all the logs together in the same place makes it easier for tools to locate logs for analysis. IT staff will also have a single log repository with which to work.
A single consolidated storage location can further simplify log management, data protection, replication and backup because it enables protection for all the logs at the same time in the same volumes. Separating log files from the production environment also enables nonproduction staff, such as software developers and help desk teams, to access logs without accessing the production environment. This logging strategy adds more security and integrity to the data center.
Log aggregation enables the IT staff to correlate various log data with a comprehensive picture of the infrastructure's behavior. Aggregation can show cause-and-effect relationships that will speed troubleshooting and reveal opportunities for improvement. For example, aggregation can reveal a connection between resource use and application errors, which can lead to configuration changes or upgrades that can resolve problems before applications, services or users are seriously affected.
A well-designed logging strategy takes into account the needs of the business and IT staff, as well as requirements for effective storage and aggregation.
Dig Deeper on Virtual machine monitoring, troubleshooting and alerting
Related Q&A from Stephen J. Bigelow
Learn how load balancing in the cloud differs from a traditional network traffic distribution, and explore services available from AWS, Google and ... Continue Reading
Access management is critical to securing the cloud. Understand the differences between AWS IAM roles and users to properly restrict access to AWS ... Continue Reading
Containers have rapidly come into focus as a popular option for deploying applications, but they have limitations and are fundamentally different ... Continue Reading