Monitor SOAP services with Python

Monitoring a server is more than just being notified of disk, memory or CPU usage. It is more than inspecting the logs and notify when something goes wrong. Each application should have a heartbeat interface meaning there must be a way to check every five minutes for example that the application is up and running, responding to requests.

Python is a great language for scripting and I am showing here how to use it to monitor SOAP services.

I tested this to monitor Advent Geneva report engine which is exposed through SOAP.

This script can be used to monitor a SOAP web service and send an alert in case it does not work as expected. Note that we send a first request, we parse the response to get a session ID and then we send a second request using that session ID to assert there is a proper response (We assert the presence of certain node in that final response) It works with responses containing namespaces, a typical scenario in real life SOAP Web Services.

#!/usr/bin/python
#
# soap-geneva-monitor.py <url>
#
# Author: Nestor Urquiza
# Date: 04/07/2011
#
# Sends an email alert whenever the Advent Geneva report fails
# Used to monitor Geneva Report Engine through its SOAP interface
#
# Preconditions:
# Install httplib2 from http://code.google.com/p/httplib2/wiki/Install
# Install ElementTree from http://effbot.org/zone/element-index.htm
#


import sys
import socket
from smtplib import SMTPException
import httplib2
import elementtree.ElementTree as etree
from urllib2 import HTTPError
import smtplib

########
# Config
#######

sender = 'donotreply@nestorurquiza.com'
receivers = ['nurquiza@nestorurquiza.com']
host = 'mail.nestorurquiza.com'
thisHostname = socket.gethostname() 



########
# Functions
#######

def sendRequest( xml_str ):
 headers = {'SOAPAction': ''}
 headers['User-Agent'] = "Monitoring"
 headers['Content-length'] = str(len(xml_str))

 


 h = httplib2.Http(".cache")

 response, content = h.request(url, "POST", body=xml_str, headers=headers)

 if response.status == 500 and not \
 (response["content-type"].startswith("text/xml") and \
 len(content) > 0):
   raise HTTPError(response.status, content, None, None, None)

 if response.status not in (200, 500):
   raise HTTPError(response.status, content, None, None, None)

 doc = etree.fromstring(content)
 

 if response.status == 500:
   faultstring = doc.findtext(".//faultstring")
   detail = doc.findtext(".//detail")
   raise HTTPError(response.status, faultstring, detail, None, None)
 
 return doc

########
# Main
#######
if len(sys.argv) != 3:  
  sys.exit("Usage: soap-geneva-monitor.py <url> <hostname>")

#url = "http://genevatest:4640"
url = sys.argv[1]
subject = url + " monitored from " + thisHostname 
hostname = sys.argv[2]
print hostname
#Define the namespace for responses
namespace = "http://geneva.advent.com"

xml_str = """<?xml version="1.0" encoding="UTF-8"?>
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
 <soapenv:Body>
  <gen:StartCallableRunrep xmlns:gen="http://geneva.advent.com">
   <gen:portNumber>9000</gen:portNumber>
   <gen:hostname>""" + str(hostname) + """</gen:hostname>
   <gen:username>user</gen:username>
   <gen:password>password</gen:password>
   <gen:extraFlags></gen:extraFlags>
  </gen:StartCallableRunrep>
 </soapenv:Body>
</soapenv:Envelope>
"""

try:
  doc = sendRequest( xml_str )
  
  print etree.tostring(doc)
  runrepSessionId = doc.findtext(".//{%s}runrepSessionId" % namespace)
  #print "runrepSessionId:" + str(runrepSessionId)

  xml_str = """<?xml version="1.0" encoding="UTF-8"?>
  <soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
  <soapenv:Body>
    <gen:RunCallableRunrepRunReport xmlns:gen="http://geneva.advent.com">
      <gen:runrepSessionId>""" + str(runrepSessionId) + """</gen:runrepSessionId>
        <gen:runrepFileName>taxlotappacc.rsl</gen:runrepFileName>
        <gen:extraFlags>-p TREE -at Dynamic -ap "TREE" -ps 04/06/2011 -pe 04/06/2011 --SeparateLegalEntities 1 --FundLegalEntity "TREE MF"</gen:extraFlags>
    </gen:RunCallableRunrepRunReport>
  </soapenv:Body>
  </soapenv:Envelope>"""

  doc = sendRequest( xml_str )
  portfolioName = doc.findtext(".//{%s}portfolioName" % namespace)
  #print portfolioName

  if portfolioName != 'TREE':
    raise Exception("Data Error", "Portfolio name '" + portfolioName + "' != TREE") 
except Exception, err:
 error = "I could not get a successful response for the SOAP Service. Details: " + str( err )
 message = "From: %s\r\nTo: %s\r\nSubject: %s\r\n\r\n" % (sender, receivers, subject) + error
 try:
    smtpObj = smtplib.SMTP(host)
    smtpObj.sendmail(sender, receivers, message)         
 except SMTPException:
    print "Error: unable to send email"

The script runs every 5 minutes from an external machine (Not from Geneva)

*/10 * * * * /usr/sbin/soap-geneva-monitor.py http://genevatest:4640