IBM Cloud Log Analysis is a service which plugs directly into the IBM Cloud Kubernetes Service, allowing for cluster-level log aggregation of Kubernetes clusters on IBM Cloud™ as well as log aggregation for your applications that are running in your containers.

It is incredibly easy to set up for your clusters, offering both a limited graphical user interface approach and a fully-fledged command-line counterpart accessible through the IBM Cloud CLI. Once IBM Cloud Log Analysis is activated for your cluster, an industry-standard Kibana dashboard will be viewable through the IBM Cloud Kubernetes Service user portal.

Table of contents

Requirements

Here is what you’ll need to start using the service:

• An IBM Cloud™ account – sign up here if you don’t have one
• A Kubernetes cluster on the IBM Cloud Kubernetes Service
• An IBM Cloud CLI installation

To sign up for an IBM Cloud account and start using the IBM Kubernetes Service Service, visit the

Once your account is set up and you have created a Kubernetes cluster, it is a good idea to install the IBM Cloud CLI. Visit the getting started page for a tutorial on installing this utility.

Log types

There are five different types of logs that you can aggregate for your Kubernetes cluster by using IBM Cloud Log Analysis:

1. Container: Turning this on will enable automatic collection of your containers’ stdout and stderr streams. These logs will only appear once you have deployed containers to your cluster.
2. Worker: Turning this on will enable automatic collection of the worker node logs in your cluster, which is limited to the contents of /var/log/syslog and /var/log/auth.log in each of your worker nodes.
3. Kubernetes: Turning this on will enable automatic collection of the Kubernetes system component logs, which includes the contents of /var/log/kubelet.log and /var/log/kube-proxy.log for your cluster.
4. Ingress: Turning this on will enable automatic collection of Kubernetes Ingress Controller logs, which include the /var/log/alb/ids/.log, /var/log/alb/ids/.err, /var/log/alb/customerlogs/.log and /var/log/alb/customerlogs/.err* log files of your cluster’s ingress controller(s).
5. Application: Turning this on will enable automatic collection of application logs for application containers deployed to your cluster. Note that this log type is highly specific to the application you choose to deploy, and is only configurable via the IBM Cloud CLI.

Setup

Now that you have the basics out of the way, it is time to turn on IBM Cloud Log Analysis. There are two ways this can be achieved: through the IBM Kubernetes Service portal and through the IBM Cloud CLI. Both of these methods will be covered below.

GUI setup

• Click on the hamburger menu in the top left:

  

• Click on Containers:

  

• Navigate to your cluster and click on it. This will load the Overview page for your cluster. Click on the Enable logging button to enable the IBM Cloud Log Analysis service for this cluster:

  

• This will bring up the Create Logging Configuration dialogue, which allows you to configure which log types you want to automatically forward from your cluster to IBM Cloud Log Analysis. An explanation of each log type is available in the Log Types section above. The Cloud Foundry Org and Cloud Foundry Space fields should populate automatically:

  

CLI Setup

• Using the previously installed IBM Cloud CLI, login to IBM Cloud. For help on accomplishing this, refer to these instructions.

• Install the Container Service Plugin:

    bx plugin install container-service -r Bluemix
  

  Show more

• Initialize the Container Service Plugin:

    bx cs init
  

  Show more

• Set your terminal context to your Kubernetes cluster:

    $(bx cs cluster-config --export YOUR_CLUSTER_NAME)
  

  Show more

• Configure the log types you want for your cluster:

  • To enable Container log forwarding, run the following command:

      bx cs logging-config-create ClusterName --logsource container --namespace '*' --type ibm --hostname EndPoint --port 9091 --org OrgName --space SpaceName
    

    Show more

  • To enable Worker log forwarding, run the following command:

      bx cs logging-config-create ClusterName --logsource worker --type ibm --hostname EndPoint --port 9091 --org OrgName --space SpaceName
    

    Show more

  • To enable Kubernetes system component log forwarding, run the following command:

      bx cs logging-config-create ClusterName --logsource kubernetes --type ibm --hostname EndPoint --port 9091 --org OrgName --space SpaceName
    

    Show more

  • To enable Ingress Controller log forwarding, run the following command:

      bx cs logging-config-create ClusterName --logsource ingress --type ibm --hostname EndPoint --port 9091 --org OrgName --space SpaceName
    

    Show more

  • To enable Application log forwarding, run the following command:

      bx cs logging-config-create ClusterName --logsource application --type ibm --hostname EndPoint --port 9091 --org OrgName --space SpaceName --app-containers Containers --app-paths Paths
    

    Show more

  • Note that for application log forwarding, the --app-paths flag will be dependent on the application you choose to deploy to your cluster (and will vary based on your chosen application). The above command is just an example.

For further information on configuring your log forwarding via the IBM Cloud CLI, refer to the following documentation.

Kibana dashboard

Now that you have enabled log forwarding from your Kubernetes cluster to the IBM Cloud Log Analysis service, your chosen log types will be automatically forwarded to the service and viewable in the Kibana dashboard. Kibana is an open source, data visualization project created by Elastic. The combination of Elasticsearch, Kibana, and Logstash are referred to as the Elastic Stack, which makes up the backbone of the service. IBM Cloud Log Analysis allows users to utilize this stack and easily plug it in to their Kubernetes clusters without needing to stand it up on their own infrastructure, saving set-up and maintenance headaches and allowing for users to access their logs faster.

To access the Kibana dashboard, navigate to your cluster’s Overview page in a web browser and click on the View Logs button:

Once the IBM Cloud Logging page has opened, click on Kibana 5 at the top to bring up the Kibana dashboard. Here, you can use the search bar to filter your logs based on cluster name, set a time window, and create visualizations for your log data.