Analyzing syslog data with VMware's Log Insight tool

Follow these step-by-step instructions to view and monitor syslog data using VMware vCenter Log Insight.

VMware Inc. joined a crowded analysis market with the release of its vCenter Log Insight tool, but the product stands out from the crowd thanks to vSphere and vCenter Operations Manager integration. Let's walk through the process of deploying Log Insight and how to use it on a daily basis.


Before you deploy Log Insight, download the virtual appliance -- currently in beta -- from the Log Insight community or Log Insight product page using your free VMware website log-in.

Once you have the OVF virtual appliance, you can deploy it from the vSphere Windows Client or the vSphere Web Client using the Deploy OVF Template option as seen in Figure 1.

The Deploy OVF Template Wizard walks you through the following processes:

Figure 1 - OVF template
Figure 1
  • Accepting the end-user license agreement
  • Selecting a deployment location for Log Insight
  • Selecting the type of storage for the Log Insight VMDKs -- thick lazy zeroed, thick eager zeroed, or thin provisioning
  • Configuring the IP address for Log Insight -- static or dynamic with DHCP

The entire process of download and deployment took me less than 30 minutes. Figure 2 shows the powered-on Log Insight virtual machine.

Figure 2 - virtual machine
Figure 2

By default, Log Insight deploys with three VMDK files (virtual disks) attached: One is for the Log Insight OS, one is for the Log Insight application logs and one is for the Log Insight consolidated syslog databases and indexes. The VMDK used for consolidated syslog databases and indexes is set at 140 gigabyte (GB).

Once Log Insight fills that space, your consolidated syslog data will be rotated out unless you either increase the size of the logging VMDK or enable archiving. To expand the log data capacity, add another VMDK and restart the Log Insight VM. Log Insight automatically recognizes the new VMDK and adds that capacity to the existing log capacity, effectively expanding the consolidated log storage space. To add an archiving location, enter the URL of a NFS file share.

Read more on syslog data analysis

Viewing and filtering syslog data

Analyzing firewall logs and antivirus data using Excel

What to do with server log data

You'll also want to keep an eye on VM size: By default, the Log Insight VM has 2 virtual CPUs (vCPUs), 8 GB of vRAM and ~140 GB of virtual disk space. VMware says this configuration can support roughly 20 hosts and 3 GB of log data per day. If you have more hosts or more log data communicating with Log Insight each day, you should resize it. The large size in the Log Insight sizing guide supports up to 750 hosts and 112 GB of log data per day, provided by 16 vCPUs and 32 GB of vRAM. To achieve this size Log Insight VM, you must power off the VM and convert the VM hardware from version 7 to 8 to support the 16 vCPUs. You could also run vCenter Log Insight on a laptop or desktop for lab testing and change the vRAM and the memory heap size in the Log Insight Web interface. The heap size should be half the size of the configured memory.

Initial configuration

Once you deploy Log Insight, point your Web browser to the Log Insight IP address or hostname. You'll then see the VMware login screen where the default username is admin and the default password is blank. From here, begin the initial Log Insight configuration wizard, as seen in Figure 3.

Figure 3 - configuration wizard
Figure 3

This wizard walks you through configuring the following parameters:

  • The admin password and email address
  • The Log Insight license key (not needed during the beta)
  • Notification email addresses
  • NTP time configuration
  • SMTP server configuration
  • VMware vCenter and vCenter Operations Manager integration
  • Data archiving (optional)
  • SSL certificate(optional)

When you restart Log Insight, reconnect and log in with the admin username and password, you'll see the message in Figure 4.

Figure 4 – error message
Figure 4

You now need to configure your ESXi hosts (at minimum) to send their syslog data to Log Insight. You can configure this using any of the following options:

  • Configure-esxi (recommended) -- Run this new Log Insight command from the Log Insight console and configure syslog on all ESXi hosts managed by your vCenter server.
  • Esxcli system syslog config -- You must run this command on each host, specifying the syslog server.
  • vSphere Client GUI -- As shown in Figure 5, you need to go into the configuration tab for each host, under advanced settings for software, and set the syslog.global.loghost.
Figure 5 – configuration tab
Figure 5

PowerCLI -- Use the set –vmhostsyslogserver command.

For ESXi hosts to send traffic to Log Insight, you must enable the outbound firewall rule in the service profile to allow syslog traffic outbound.

Insight and analysis

Once Log Insight starts receiving your syslog traffic, you can start analyzing the data in one of two ways, but you will use both methods depending on the information you need. The vSphere Dashboard contains the overview of the virtual infrastructure logs. From the dashboard, you can perform a detailed interactive analysis.

Interactive analysis offers a Google-like search box, as shown in Figure 6.

Figure 6 – search box
Figure 6

You can filter the data using constraints, reducing the number of fields or reducing the time range. Once you have tweaked your search and filtered results, you can use the save query option, which allows you to put it on a dashboard, set an alert for the query and even share the query by sending a URL to a coworker.

The user-friendly interface of VMware vCenter Log Insight makes it a welcome addition to VMware's management portfolio. If adopted by third-party vendors, and if used by the VMware community, the content pack sharing could make Log Insight popular and powerful. With support for physical servers, ESXi hosts, vCenter servers, vCenter Operations Manager, network devices and storage devices, Log Insight could make life easier for many vSphere managers.

Dig Deeper on VMware management tools