Tag Archives: vRLI

vRLI Header

Using vRealize Log Insight Cloud to archive on-premise Log Insight Data

vRealize Log Insight 8.6 brings the ability to build a hybrid log management platform, utilizing the functionality of an on-premises deployment of vRLI and vRLI Cloud.

From the release notes, in this blog post we’ll be looking at how to configure the following:

  • Simplify Log Archival with Non-Indexed Partitions: Use vRealize Log Insight Cloud to archive logs to meet your long-term retention requirements. vRealize Log Insight Cloud provides a no-limit logging solution at a low cost and eliminates any storage management overheads of the past. This enables easy accessibility to archived logs through on-demand queries.

For this, you will need access to a vRealize Log Insight Cloud Instance, with a cloud proxy deployed to your environment that can be accessed by the on-premises vRealize Log Insight platform.

The expectation is that you would forward you vRealize Log Insight on-premises logs to the vRealize Log Insight Cloud instance storing them only in a Non-Indexed Partition (discussed below). As your on-premises deployment act as your easy to analyse near time (within 30 days) copy of your logs. 

In this blog post I also explore the configuration and use of Index Partitions which essentially offers that near time usability and analysing of logs as well.

The high-level steps for the configuration discussed in this blog post are:

  • Send infrastructure or application logs to your on-premises vRealize Log Insight deployment
  • Setup the cloud proxy (if not already done)
  • Setup log forwarding from the on-premises Log Insight instance
  • In vRealize Log Insight Cloud, configure Non-Index Partition to receive the forwarded logs
What are Log Partitions?

Log Partitions are a feature that allows you to ingest logs based on user-defined filters. This feature is available as a paid subscription (or Trial).

There are two types of log Log Partitions:

  • Indexed Partitions
    • Stores logs for up to 30 days
    • Billed only for volume of logs ingested into the partition
    • Search and analyse logs in this partition without additional costs
  • Non-Indexed Partitions
    • Stores logs for up to 7 years
    • Billed for the volume of logs ingested into the partition, and for searching the logs.
    • If you need to query logs frequently, you can move logs to a recall partition for 30 days.
      • No additional cost for searching and analysing logs in the recall partition

Logs that do not match a query criteria in any of the configured partitions, will be stored in the Default Indexed Partition. This is read only and stores logs for 30 days.

Note:  

- Alerts and dashboard widgets are not operational in non-indexed partitions.
- Log partitions store logs ingested in the last 24 hours only.
- You can create a maximum of 10 log partitions in an organization.
Video Walk-through

Example Logs

In my Log Insight environment, I have setup the FluentD configuration to forward the Tanzu Kubernetes Grid logs from two clusters to vRealize Log Insight (on-premises deployment).

You can find the configuration settings for this within vRealize Log Insight, under the Sources Tab > Containers > Tanzu Kubernetes Grid.

vRLI Log Archive - Configure Fluentd for Tanzu Kubernetes Grid

vRLI Log Archive - Tanzu Kubernetes Grid Logs

Setup the Cloud Proxy

Continue reading Using vRealize Log Insight Cloud to archive on-premise Log Insight Data