Nagios is a popular event management application. With it’s scriptable commands and plugins, it is very customizable for performing any action when a monitor alert is triggered. This document will demonstrate how to communicate directly with LiveTime’s CMDB and update CI’s in realtime with bi-directional feeds back to Nagios and LiveTime.

When a check fails and the monitor alerts, one common result can be that an incident is created whilst the failing service is investigated. With the event-handler capability of Nagios and LiveTime’s inbound web services, this is simple to setup.

Additionally, with LiveTime’s outbound web services and run book automation the event-handler can be paused and re-instated after the incident has been dealt with. This makes it easy to have the Nagios alert create just one request and then only create new ones once the issue with the service has been resolved. This is especially useful for services that are intermittently working/failing and alerting on the Nagios monitors.

Furthermore, if the CMDB in LiveTime is configured correctly, it can be possible to identify the item in question and take the item offline in LiveTime, restoring the item after the incident has been resolved.

Operational Workflow

1. Service fails and alerts in Nagios
2. The Nagios event handler calls a PHP script
3. The PHP script parses the host and service names from Nagios and utilizing LiveTime’s inbound web services, it determines the item in the CMDB in question. If the Item for the service is not found that is related to the host, then it falls back then it falls back to the default item configured in the script.
4. An incident is raised against the item established in step 3
5. If an exact item match is found in step 3, the item’s status is taken offline
6. If the incident is successfully created, to prevent further requests being created, the event handler is disabled for the alert in Nagios via a web request with the host and service parsed in step 3.
7. The Nagios check that triggered this is recorded as the first note on the incident.
8. Once the issue is resolved, the incident is moved into a status which has an outbound web services script attached to it.
9. Outbound script reads the first note on the incident to establish the Nagios check involved and re-enables the event handler on Nagios for this, via a web request.
10. It also updates the item on incident back to it’s available status, if an item was identified in step 3.

Setup

For the purposes of this document, the Nagios Virtual Machine was used. This is available for download from the Nagios website. If Nagios is already setup and running, then skip to the last step of these instructions as ensuring php-soap is installed is the most important step here.

Once running, the admin pages are at:

http://host/nagiosxi/

The user pages are at:

http://host/nagios/

To avoid having to rely on accessing the console output of the virtual machine, the firewall needs opening up to allow local users access. To do this, for all ports, on a 192.168.1.0/24 based network (for example), one would login as root using the default password of nagiosxi and run:

iptables -I INPUT -p tcp -s 192.168.1.0/24 -j ACCEPT
iptables-save
service iptables save

Some checks then need to be added via the admin pages. For the example in this document, we add an FTP check to a Windows server named Windows 101.

If the auto discovery wizard is to be used to quickly get some checks added, where you pick which machines you want to check after it completes, traceroute needs to be installed first.

To do this login as root and run:

yum install traceroute

To get files to/from the virtual machine, one method my be to use scp, and to do that, you would need to first run:

yum install openssh-clients

Finally, PHP is already installed on the virtual machine but for web services commands to run, php-soap also needs to be installed. This is done by running:

yum install php-soap

Creating the Request in LiveTime

A web services script is created to do this, and this is copied to the Nagios server. There is a scripts folder already setup on Nagios so in our example we copied the script as:

/usr/local/nagiosxi/scripts/lt_nagios_create_request.php

In Nagios go to the admin Configure > Core Configuration Manager (http://192.168.1.125/nagiosxi/config/nagioscorecfg/)
Select Commands drop down > Commands > Add new
Enter the following, making sure Active is ticked and click Save:

Command:

lt-create-request

Command Line:
/usr/local/nagiosxi/scripts/lt_nagios_create_request.php $SERVICESTATE$ $SERVICESTATETYPE$ $SERVICEATTEMPT$ $HOSTNAME$ $SERVICEDESC$

Command Type:
Misc

To complete this addition, click Apply Configuration.

Next, Under Monitoring, Click Services.

Find the service that needs to create a request on alert (in our example, the Config Name is ‘Windows 101’ and the Service Name is ‘FTP’) and click tools icon to edit it.

Under the Check Settings tab, use the drop down for Event handler to select lt-create-request and set the enabled option to on.

Click to save those settings and then click Apply Configuration to complete this modification.

Configuring the LiveTime CMDB

For the correct item to be used, there are a number of ways to do this, but in our example PHP script, we set it up so the host is an item in LiveTime, and the Nagios Service being tested is also an item in LiveTime.

For instance, you might have a server numbered ‘Windows 101’ as the host with and FTP service running on it, which for this we have assigned an item numbered ‘FTP101’, which is of the type ‘FTP’. The items are then setup in a relationship so that the ‘FTP101’ is installed on ‘Windows 101’.

The number of the actual service item doesn’t matter but in our example the number of the host item does, if items are not to be assigned to the default item.

The example script first attempts to locate the item numbered the same as the host, ‘Windows 101’ and then looks for a related item of the type matching the service being checked, ‘FTP’. It finds the item ‘FTP101’ so it picks that item for the new incident.

If it didn’t find an FTP type of item related to the host, it would just raise it would just raise it against the default item.

The script will also update the item status to an Offline one but only if it finds an exact match to the service. If not, it will not update it. This is purely in the example script so obviously can be configured as required.

Also, the example script is not strict on the relationship type, for the purposes of demonstrating this. It could be further amended to only look at services installed on the host, as opposed to those also used by the host, if other relationships existed in the CMDB in question. This could readily be added to the script.

Re-activating the Event Handler

LiveTime’s outbound web services can be used to call a script to reactivate the event handler in LiveTime once the request has reached a status where the service issues have been resolved.

Additionally, the script can call back to LiveTime’s inbound web services to update the item’s status as online, if an exact item was identified and placed into an offline state in the original script that created the request.

As a Java application, the scripts behind LiveTime’s outbound web services need to be in Java.

For the best performance, the script that performs these actions should ideally be coded in Java. However, for the purposes of this demonstration, it is also possible to code enough java to call another command, which can then be coded in any language. In our case, we are going to call a PHP script as we can then reuse some of the code used in the initial script too.

The script sends, as command line arguments, the status change involved (ie. if the status was Entered or Exited), the request number and the item number on the request. The PHP command then reads these and performs the required actions.

To install the provided code for this, jump to step 3 below and copy the included jar file over to the location specified. However, if making changes to the script, the following needs to be done:

1. To amend the code, copy this file to the folder with the java file in:

{LiveTime Install Folder}/LiveTime.woa/Contents/Resources/Java/livetime-listen.jar

2. After all required changes have been made, the following commands need to be entered:

javac lt_nagios_resolved.java
jar cf lt_nagios_resolved.jar lt_nagios_resolved.class -classpath livetime-listen.jar

3. The resulting lt_nagios_resolved.jar file then needs to be coped to the the following location on the LiveTime server:

{LiveTime Install Folder}/LiveTime/WEB-INF/lib/

After this, the LiveTime application server, needs to be restarted for this to be picked up.

By default, the java code is calling ‘php /root/lt_nagios_resolved.php’. As such, php and php-soap need to be installed on the LiveTime server, along with copying lt_nagios_resolved.php to the /root/ folder.

Note that the PHP script is set to change the status of the service to ‘Available’ in line with the online status provided by default for items of the Service category in LiveTime. This can be modified according to what is in use.

In fact, the script could be further developed to look for ‘Available’ and if that wasn’t present, try ‘Online’, etc, if multiple item category types were used. Obviously that is up to the user to add such coding.

Finally, to call this java code, the workflow used for dealing with the alert requires a status that is linked to the class in the jar file created above. To do this, ‘Outbound Web Services’ needs to be enabled under Setup > Privileges > System and the status in question is then updated so that it’s Listener Class is:

lt_nagios_resolved

Nagios Closed Loop Example Code - 9.04 kB zip

We have provided a link to the complete source code of a more complex application. Our Web Services are based on Apache Axis 1.x in these examples for the greatest compatibility with existing applications.

Feel free to use this code as the basis for you own applications.

In order to communicate with LiveTime’s web services you should have a thorough knowledge of SOAP and also LiveTime’s Web Services API’s. LiveTime has a developers guide which details each of the API calls and how to use them. What follows are scripts you can use to handle some of the more difficult parts of communication. Notably session management.

Java Web Services Example - 2.11 MB zip

package com.livetime.sample;

import com.livetime.ws.cust.Customer_PortType;
import com.livetime.ws.cust.Customer_Service;
import com.livetime.ws.cust.Customer_ServiceLocator;

public class CreateCustomer extends BaseClient {

	/* Customer Service URL */
	public static final String customerServiceURL = "http://10.0.1.11/LiveTime/WebObjects/LiveTime.woa/ws/Customer";

	/* Constructor - doesn't need to do anything coz this is just a sample */
	public CreateCustomer() {

	}

	private boolean createCustomer() {
		java.net.URL custURL = null;

		try {
			// Service endpoint
			custURL = new java.net.URL(customerServiceURL);

			// Get handle to the service
			Customer_Service service = new Customer_ServiceLocator();
			Customer_PortType port = service.getCustomer(custURL);

			// Call BaseClient method to populate persistent headers
			populateHeaders((javax.xml.rpc.Stub)port);

			// create properties you want to set for this person - properties you may want to set here:

			// custom1 (String 255), custom2(String 255), custom3(String 255), custom4(String 255), custom5(String 255),
			// aliases (comma separated list of email addresses), orgunit (String - case insensitive match to a LiveTime org unit),
			// phone(String 32), phone2(String 32), phone3(String 32), address(String 128), addressTwo(String 128), postalCode(String 32), city(String 64),
			// country (Integer as String - must be id from getCountries() call), state(Integer as String - must be id from getStatesForCountry(String countryId) call)

			java.util.HashMap<String, String> properties = new java.util.HashMap<String, String>();
			properties.put("phone", "+1 949 777 5800");
			properties.put("phone2", "+61 3 9620 7588");

			// So this is generally reusable without scrubbing database each time, we add a random number after the username & email
			// wouldn't do this in the real world obviously - but I'll mention it in case someone tries :p
			String suffix = Integer.toString((new Double(Math.random() * 1000000)).intValue());

			// username, email, first name, last name, properties (above)
			java.util.HashMap response = port.createCustomer("bob_" + suffix, "bob_" + suffix + "@mycompany.com", "Bob", "Nelson", properties);

			// Just output stuff from here on
			String successString = response.get("success").toString();
			boolean successful = Boolean.parseBoolean(successString);

			if(!successful) {
				System.out.println("Customer Creation failed: " + response);
			}
			else {
				System.out.println("Customer Creation succeeded: " + response);
			}

			return successful;
		}
		catch(Exception ex) {
			ex.printStackTrace();
			return false;
		}
	}

	// Entry point for execution
	public static void main(String[] args) {
		// create instance of this class
		CreateCustomer createCustomer = new CreateCustomer();
		// try and connect, if successful, create customer and logoug
		if(createCustomer.connect()) {
			// create the customer
			createCustomer.createCustomer();
			// logout
			createCustomer.disconnect();
		}
	}
}

With the widespread popularity of PHP for web programming, many people have expressed interest in writing scripts to communicate with LiveTime’s Web Services rather than using more complex languages like Java.

Many script languages now have plugins for working with Web Services. In this example we are going to discuss PHP. Equally, you can achieve the same thing using languages like Perl.

In order to communicate with LiveTime’s web services you should have a thorough knowledge of SOAP and also LiveTime’s Web Services API’s. LiveTime has a developers guide which details each of the API calls and how to use them. What follows are scripts you can use to handle some of the more difficult parts of communication. Notably session management.

The following code assumes that PHP is installed and running with the PHP SOAP extension. You will note how the cookie is used to manage session state. Feel free to use these functions in your own scripts to facilitate rapid protoyping of your own solutions to communicate directly with LiveTime.

<?php
	/*
	  LiveTime example PHP command line script for web services
	  Requires the PHP SOAP extension to operate
	*/

	$hostAddress = "www.host.com";
	$baseProduct = "LiveTime";
	$baseAddress = "/" . $baseProduct . "/WebObjects/" . $baseProduct . ".woa/ws/";
	$baseService = "http://" . $hostAddress . $baseAddress;

	function login($username,$password) {
		global $objClient;
		/*
    	  Setting "trace" will allow us to view the request that we are making, after we have made it.
		*/
		# Connect to LiveTime Authenticate?wsdl
		# Use only technicians or supervisors
		$results = $objClient->connect($username, $password);

		//This will return an Associative Array
		$success = $results['success'];
		if ($success=="false") {
			echo $results['message'] . "\n";
			exit();
		}
		echo "Login Successful\n";
		createSession();
	}

	function logout() {
		global $objClient;
		global $baseService;

		$objClient->SoapClient($baseService . "Authenticate?wsdl", array('trace' => true));

		$results = $objClient->disconnect();
		//This will return an Associative Array
		$success = $results['success'];
		if ($success=="false") {
			echo $results['message'] . "\n";
			exit();
		}
		echo "Logout Successful\n";
		exit();
	}

	function createSession() {
		global $objClient;
		#
		# Retrieve the Last Response Header from the LiveTime server

		$responseHeader = $objClient->__getLastResponseHeaders();

		$sessionPos=strpos($responseHeader, "JSESSIONID=");

		if ($sessionPos === false) {
			echo "No session id was found. Exiting.\n";
			exit();
		}
		//cookie will always be 32 bytes
		$cookie = substr($responseHeader,$sessionPos + 11,32);

		#
		# Set the Cookie name for the next request
		$objClient->__setCookie("JSESSIONID", $cookie);
	}

	function findUser($username,$email,$first,$last) {
		global $objClient;
		global $baseService;

		//switch to the customer api
		$objClient->SoapClient($baseService . "Customer?wsdl", array('trace' => true));

		echo "Searching for " . $username . $email . $first . $last . "\n";
		#
		# Call findCustomer, using an email address and echo out response
		# username, email, first, last
		$results = $objClient->findCustomer($username,$email,$first,$last);

		$success = $results['success'];
		if ($success=="false") {
			echo $results['message'] . "\n";
			return;
		}

		#if we have results lets show them
		#returns an array of arrays
		foreach(array_values($results) as $ess) {
			print_r($ess);
		}
	}

	//construct the soap client to create a connection and then login
	$objClient = new SoapClient($baseService . "Authenticate?wsdl", array('trace' => true));
	//if using AD or LDAP you need to enter the Fully Qualified AD/LDAP Domain
	//super@domain.mycompany.com
	login("super","super");

	findUser("","cbrown","","");

	//we are all done
	logout();

?>

In the example function above we first login using the connect method of LiveTime’s Authenticate wsdl and this subsequently calls the createSession function. To maintain session we grab the JSESSIONID from the response header and then extract the 32 byte cookie. We can now use this cookie in subsequent web service calls to LiveTime. The complete sample code is available for download. Do not forget to logout after completing all the tasks in your code.

Once you have developed your first few scripts you will see that it is possible to do anything you like with the data you obtain from LiveTime. This can be used for coordinating information from multiple data stores or live population of the CMDB from custom systems.

Enterprises currently faced with the challenge of integrating and extending their investments in sophisticated business applications such as ERP, CRM and Service Desk, are increasingly turning to Web services. As a result, many businesses looking to better leverage their IT assets and provide their organizations with the agility needed to stay competitive in today’s economy are now Web service enabling their IT infrastructure. The ultimate goal of these efforts is to increase enterprise efficiency, improve customer satisfaction and increase profitability.

This paper provides an overview of Web services technology and its ability to fulfill enterprise-wide Service Desk integration requirements.

Web Services and IT Service Management - 201.53 KB pdf

Authentication

Using the Authentication gateway, LiveTime products can integrate with any LDAP or Active Directory Server. Supporting Anonymous, SASL and SSL authentication and individual mappings, it is possible to control every aspect of user login and security.

This gateway makes it possible to use mixed login techniques for different users or groups, and connects with any 3rd party server to validate user access. LiveTime seamlessly supports all major directory servers including:

Single Sign-on (SSO) and SAML

In addition, LiveTime also provides integration points into SAML or SAML2 based single sign on (SSO) or identity management environments. This allows authenticated users to bypass the login page and land directly in the request queue. Details on this integration are discussed further in the LiveTime blog.

Asset Management and Discovery

The LiveTime Asset Management Integration Engine (AMIE) enables enterprises to quickly and easily synchronize with third party asset management systems. Using XML descriptor files embedded within the LiveTime application, AMIE is able to connect to the foreign host and transfer any type of Asset Management data into LiveTime’s Configuration Management Database (CMDB).

AMIE provides default integrations to most major asset management systems, and provides a simple mechanism to create your own in a matter of hours using simple XML descriptor files. Default adaptors include:

Monitoring and Event Management

LiveTime enables integration into event management and monitoring tools through multiple mechanisms. This can be achieved through direct web service calls from your tool of choice or using closed-loop email request management. This enables monitoring tools to directly communicate with LiveTime and leverage Configuration Items within the CMDB, Service Level agreements and Technician assignment with minimal setup.

Web Services

Web services are important to business because they enable systems from multiple vendors to communicate with one another, and reduce the risks associated with relying on one vendor for services. Web services give companies the ability to do more business electronically, with more potential business partners, in more ways than before, at a reasonable cost.

LiveTime provides rich web services for every major function within the application. Consisting of several hundred API calls across 6 classes from Authentication through Request management and the CMDB, there is no limit to what can be achieved. Customers have provided integrations to proprietary customer databases, home grown asset management systems and ticket exchange gateways.

Newport Beach, CA – January 22, 2004 - LiveTime Software announced the inclusion of Web Services for the LiveTime Support release scheduled for Q2 2004.

The LiveTime Web Services Gateway will enable organizations to integrate LiveTime Support 3.0 with in-house applications, extending its capabilities within an enterprise and streamlining the support process. The Gateway will allow organizations to incorporate LiveTime Support 3.0 into any Enterprise environment with integration points provided for ERP, Accounting, SFA and HRM systems including SAP, Siebel, PeopleSoft, Oracle.

LiveTime Support is renowned for its standards-based architecture using J2EE and vendor neutral data access layer. The Web Services Gateway will further unlock the power of LiveTime Support through the Simple Object Access Protocol (SOAP) programming interface. Using XML, SOAP is both language and platform independent, allowing any application to interface with LiveTime Support regardless of the operating system or platform.

The LiveTime Support Gateway is an extensible solution that allows enterprises to integrate and synchronize mission critical Support and Help Desk operations with other business activities. “By utilizing Web Services in LiveTime Support, organizations can truly leverage their knowledge infrastructure to provide real-time integration across the Enterprise”, said Dr. Darren Williams, CEO of LiveTime Software. “With the ability to link to other Enterprise systems, LiveTime Support is able to access customer records from SFA or HR systems and raise problem tickets immediately, significantly reducing the turn-around time in logging calls and resolving problems.”

LiveTime is working closely with key partners to ensure success in the development of the Web Services Gateway. “LiveTime has adhered to all the core Web services specifications in developing the SOAP interfaces interface to our flagship product”, said George Vossos, CTO at LiveTime Software, ” this ensures that our customers can confidently integrate their support workflow with other key business processes.”