Python Scripted API Bridge
Helper Script for RESTful API integrations with ServiceNow & other tools, using events, emails or incidents.
Tl;dr
At the very bottom of this script is the link to my GitHub repo for the latest version of the script template.
What:
Lately, I have been coming across a lot of requests to create integrations between disparate tools into ServiceNow (SN). Most of these tools have some sort of alerting mechanism but they don’t have an API integration into ServiceNow. I typically use a python script to execute as a script so events or alerts from an external system can be integrated into ServiceNow.
I recently used about 90% of the functions from this script to create a standalone monitoring script. This script would query a list of hosts and check if they are alive via ping, then if the response is OK, check to see if the appropriate web server port is open.
Another item to note, is that we can reuse these functions and with a few changes we can POST data via REST/JSON or create email-based integrations to other tools like BMC Remedy, JIRA, etc. or even other notifications tools like xMatters, PagerDuty, VictorOps, etc. to name a few.
Why:
I know this is all very simple stuff for python gurus but for me, it was a lot of time searching and trial/error to script this. Once I got the script working, I would typically rewrite a lot of code.
This is the exact use case for object-oriented programming (OOP) and I wanted to take a stab at creating templates, so I can bring this code into any future integrations, to reuse it easily.
I took all the most common functions that I have used over the last few months and built a “super” script. My plan is to keep this code as a template and just cut/paste additional functions as needed.
My hope is that this code helps you shorten your development work. Also, if you find a way to make this efficient, my hope is that you would kindly help others and myself. Next stop, code walk-through.
Code Walkthrough:
Start of the script. Notice that I am using Python 3 for this script (and most others that I work with).
#!/usr/bin/env python3
'''
This is a script with a list of useful functions and snippets for sending events and emails to ServiceNow. This works as a standalone script but you need to code your own "stuff".
Dan Tembe
dtembe@yahoo.com
06/03/2019
'''
Next comes the import. Note that you will need to use pip or pip3 to add all the python packages for this script to run properly. This is standard in any python development, so I am not going to explain each import.
import json
import datetime
from datetime import timedelta
import requests
from requests.auth import HTTPBasicAuth
import urllib3
import logging
import glob
import os
import smtplib, ssl
//TODO - write a "requirements.txt" to streamline imports.
Next part of script deals with removing files. This is just good housekeeping. I don’t have any date/time variables declared to remove files older than certain number of days, but that can be easily added to this code to make it work as you need it to.
#Remove files ( uncomment and add in log file path/name if you want to delete logs from last run)
for f in glob.glob('/tmp/dt_script.log'):
try:
os.remove(f)
except:
pass
for f in glob.glob('/var/log/scriptlog.log'):
try:
os.remove(f)
except:
pass
Next part deals with global variables. I am sure you will not need all these, but in my experience, I have had to manipulate date / time to ensure I had the right alerts or events from logs or tools that matched a certain date and time before I could post them or email them to ServiceNow.
**** Please Comment out any “print” statements before moving any code snippets code to production.
#Global Vars -
# Useful vars (not needed but I like to keep these in my script in case I need to manipulate date/time in any of my functions
day = datetime.date.today()
daystring = datetime.datetime.today().strftime('%Y-%m-%d-%H-%M-%S-%f')
today = datetime.datetime.today().strftime('%Y-%m-%d')
minus5days = datetime.date.today() - timedelta(5)
todayminus5days = minus5days.strftime('%Y-%m-%d')
#
print('variables - Day: ' , day, ' daystring: ' , daystring , ' today: ' , today , ' minus5days: ' , minus5days , ' todayminus5days: ' , todayminus5days , ' ' )
Next, we add in all the ServiceNOW specific details, so our data can be posted. This is needed only if you are using ITOM Event Management module or the ITSM Incident based integration. As it states in the script, make sure the user account you are using to post events or incidents to ServiceNOW has the API integration user role added to it. This will ensure the POST of events or incidents to the SN instance will not get rejected when the script attempts to post.
There are 2 distinct URL’s from ServiceNow listed below. The functions for posting events use “snowemurlevent” variable and for posting incidents uses “snowurlincident”.
In this code block, you will need to change “instanceName”, “SNUserName” and “SNPassword” with the appropriate for your SN instance.
#Vars Used to Post Data to ServiceNow -
snowemurlevent = "https://instanceName.service-now.com/api/now/table/em_event"
snowurlincident = "https://instanceName.service-now.com/api/now/table/incident"
#Make sure the user has API access to SN Instance
snowemuser = "SNUserName"
snowempassword = "SNPassword"
Next part is about adding in SMTP information. I use my personal GMAIL settings to test. You will need to use your internal or any SMTP server that will allow you to relay emails through. This is only relevant if you are going to use emails (SMTP) to send your notifications to SN; then process the email as an Inbound action.
In the below snippet you will change “user@company.com”, "emailPassword", 'smtp.gmail.com', ‘587’, 'instanceName@service-now.com' with correct values.
# smtplib module send mail to be able to create an inbound action in SN. I am using gmail for testing
mailuser = "user@company.com"
mailpass = "emailPassword"
mailsmtp = 'smtp.gmail.com'
#make sure you are aware of the correct port. Gmail SSL is 587 for testing
mailport = '587'
TO = 'instanceName@service-now.com'
Next part we will add in logging environment. This is a good practice since you will most likely run this script as a daemon so there will be no one watching the console for errors. It is good to log details.
Feel free to change where the script will log to. I am using /tmp or /var/tmp/ as some sample directories.
# Creating Logger Environment
logger = logging.getLogger(__name__)
logger.setLevel(logging.INFO)
# create a file handler
handler = logging.FileHandler('/tmp/dt_script.log')
handler.setLevel(logging.INFO)
# create a logging format
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
handler.setFormatter(formatter)
# add the handlers to the logger
logger.addHandler(handler)
logger.info('\n')
logger.info('Starting Script')
Next is the SSL Warning. I run into these numerous times, where the internal tools don’t have a valid SSL. This becomes an issue of you are using this script to “GET” data or events. We all know on the SN side this is not an issue as there is a valid SSL Cert.
#SSL Warnings Disabled when uncommented. Comment out to turn on SSL Warnings. urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
Now comes the good part. I am not going to copy each function in the script. Instead I will take one function of Incident, Event and Email and explain it as best as I can.
Let us start with the Incident creation function. The same we have below is called “SnCriticalIncident”. This function when used will create a new Incident with the values provided in the variables declared.
The function input is either a JSON Array with key / value pairs or individually declared variables. I have named all the input values to be like the Incident table field names, so it will be easy to provide these values into the function from any tool or process you are invoking to receive this data.
Once all the values are provided into the key:value pairs, in object called data. From this data object, we will create valid json using json.dumps(data). This is a valid JSON object called “postData” and is now ready to be posted to ServiceNow Incident table.
Lines starting with Try and catch show the exact method of posting “postData” object to ServiceNow. Ensure your headers are correct, using basic authentication. The variables for logging into SN were declared at the beginning of this script.
The Post will provide a response, we are logging this response into our log file. Just so we don’t break this script, we are using “try” and “except” to ensure all errors are logged and the script moves past any hiccups.
Also, keep in mind that you don’t have to pass all these variables into the function. You can create static values for some of these variables at the top of the script and pass those into the function if you don’t expect these to come from external tool or other functions.
def SnCriticalIncident(caller, categoryList, subcategoryList, businessServiceLookup, cmdb_ciLookup, contact_typeList, stateList, impact, urgency, assignmentGroupLookup, assigned_toLookup, short_description, description):
try:
#jsonarray = l
o_caller = (caller)
o_category = (categoryList)
o_subcategory = (subcategoryList)
o_business_service = (businessServiceLookup)
o_configuration_item = (cmdb_ciLookup)
o_contact_type = (contact_typeList)
o_state = (stateList)
o_impact = ('1') #this works
o_urgency = ('1')
o_assignment_group = (assignmentGroupLookup)
o_assigned_to = (assigned_toLookup)
o_short_description = (short_description)
o_description = (description)
print ("-" * 50)
print (o_caller, o_category, o_subcategory, o_business_service, o_configuration_item, o_contact_type, o_state, o_impact, o_urgency, o_assignment_group, o_assigned_to, o_short_description, o_description )
print ("-" * 50)
data = {"caller_id": o_caller, "category": o_category, "subcategory": o_subcategory, "business_service": o_business_service, "cmdb_ci": o_configuration_item, "contact_type": o_contact_type, "state": o_state, "impact": o_impact, "urgency": o_urgency, "assignment_group": o_assignment_group, "assigned_to": o_assigned_to, "short_description": o_short_description, "description": o_description}
postData = json.dumps(data)
try:
url = snowurlincident
auth = HTTPBasicAuth(snowemuser, snowempassword)
head = {'Content-type': 'application/json',
'Accept': 'application/json'}
payld = postData
ret = requests.post(url, auth=auth, data=payld, headers=head)
# sys.stdout.write(ret.text)
returned_data = ret.json()
logger.info(returned_data)
print(returned_data)
except IOError as e:
print(e)
logger.error(e)
pass
except IOError as e:
print(e)
logger.error(e)
pass
The " SnCriticalEvent” is a function to create a new SN Event with Critical Severity. The key differences are that we are posting to a different table in ServiceNow. The SN Events are posted to “em_event” table.
Secondly, we are also using different values and function needs different number of input values.
Beyond this the events are posted via RESTful API into ServiceNow. Using events instead of incidents for tools or any external events or alerts data is preferable because we can then use event rules and alert management rules to de-duplicate, suppress and enrich events and alerts. Another key topic worth mentioning is that with ITOM suite, we can utilize the Integration hub for going back out to the tools or endpoints and performing automated actions which could result in auto resolution of alerts.
Now we will go over the code for emails. In my script, I am using Gmail as my SMTP along with my personal auth for sending these emails. In your case, esp. for production or even for lab use, you will need to use an internal SMTP or SMTP relay server.
I am using “smtplib” import to send emails. Since I know once the email reaches ServiceNow it will need to be handled via an email Inbound Action, I am sending the values for each key in a key and value pair. In my example, I plan to create events from my inbound action rule, from the inbound emails. Therefore, the keys match the event table fields.
Without changing the format of the email, we can very easily use the values and create an inbound action to create an Incident, Change, problem, etc. The values in the email can automatically be accessed by referring to keys as long as they are separated by a colon ( : ). This makes writing inbound action a very straightforward task.
The email function process is very similar to the function for sending events or incidents at the beginning, since we are using JSON to create the key and value pairs. The process to send the email is different as you will see the difference in the try / except lines of code.
The variables for SMTP server, the SMTP server port, recipient, sender email and password were all declared at the beginning of the script, so the function picks up those values at the time of code execution.
def SnClearEmail(hostname, message, source, metric_name, sntype, message_key, severity, resource, description, event_class, additional_info):
try:
o_source = (source)
o_node = (hostname)
o_metric_name = (metric_name)
o_type = (sntype)
o_message_key = (message_key)
o_severity = ("0")
o_resource = (resource)
o_description = ("Clear: On " + hostname + " Message: " + message) # this works
o_event_class = (event_class)
o_additional_info = (additional_info)
print("-" * 50)
print(o_source, o_node, o_metric_name, o_type, o_message_key, o_severity, o_resource, o_description, o_event_class,
o_additional_info)
print("-" * 50)
data = {"source": o_source, "node": o_node, "metric_name": o_metric_name, "type": o_type,
"message_key": o_message_key, "severity": o_severity, "resource": o_resource, "event_class": o_event_class,
"description": o_description, "additional_info": o_additional_info}
TEXT = json.dumps(data, indent=4, sort_keys=True)
gmail_sender = mailuser
gmail_passwd = mailpass
SUBJECT = 'Monitoring Email - Clear'
#TEXT = 'Here is a message from python.'
mserver = smtplib.SMTP(mailsmtp, mailport)
mserver.ehlo()
mserver.starttls()
mserver.login(gmail_sender, gmail_passwd)
BODY = '\r\n'.join(['To: %s' % TO,
'From: %s' % gmail_sender,
'Subject: %s' % SUBJECT,
'', TEXT])
try:
mserver.sendmail(gmail_sender, [TO], BODY)
logger.info('email sent')
print('email sent')
except:
logger.error('error sending mail')
print('error sending mail')
mserver.quit()
except IOError as e:
logger.error("Error: ", e)
print(e)
pass
This ends the template portion of the script.
Remainder of the script is first a block of variables with static values. This is used for testing that the functions we have written actually post the values from these variables.
This is followed by the main function in the execution block, which for now, is just taking in the functions for sending events, incidents, emails, taking the values we declared in the above block and then sending the data northbound via RESTful API (JSON) or SMTP (Email).
##Testing purpose only
def main():
try:
#send email
SnInfoEmail(hostname, message, source, metric_name, sntype, message_key, severity, resource, description,event_class, additional_info)
# SnWarningEmail(hostname, message, source, metric_name, sntype, message_key, severity, resource, description, event_class, additional_info)
# SnMinorEmail(hostname, message, source, metric_name, sntype, message_key, severity, resource, description, event_class, additional_info)
# SnMajorEmail(hostname, message, source, metric_name, sntype, message_key, severity, resource, description, event_class, additional_info)
# SnCriticalEmail(hostname, message, source, metric_name, sntype, message_key, severity, resource, description, event_class, additional_info)
# SnClearEmail(hostname, message, source, metric_name, sntype, message_key, severity, resource, description, event_class, additional_info)
logger.info("Emails Sent to ServiceNow")
print("Emails Sent to ServiceNow")
except IOError as e:
logger.error(e)
pass
try:
#Post event
SnInfoEvent(hostname, message, source, metric_name, sntype, message_key, severity, resource, description, event_class, additional_info)
# SnWarningEvent(hostname, message, source, metric_name, sntype, message_key, severity, resource, description, event_class, additional_info)
# SnMinorEvent(hostname, message, source, metric_name, sntype, message_key, severity, resource, description, event_class, additional_info)
# SnMajorEvent(hostname, message, source, metric_name, sntype, message_key, severity, resource, description, event_class, additional_info)
# SnCriticalEvent(hostname, message, source, metric_name, sntype, message_key, severity, resource, description, event_class, additional_info)
# SnClearEvent(hostname, message, source, metric_name, sntype, message_key, severity, resource, description, event_class, additional_info)
logger.info(" SN Events POSTED")
print("SN Events POSTED")
except IOError as e:
logger.error(e)
print(e)
pass
try:
#Post Incident
SnPlanningIncident(caller, categoryList, subcategoryList, businessServiceLookup, cmdb_ciLookup, contact_typeList, stateList, impact, urgency, assignmentGroupLookup, assigned_toLookup, short_description, description)
# SnModerateIncident(caller, categoryList, subcategoryList, businessServiceLookup, cmdb_ciLookup, contact_typeList, stateList, impact, urgency, assignmentGroupLookup, assigned_toLookup, short_description, description)
# SnCriticalIncident(caller, categoryList, subcategoryList, businessServiceLookup, cmdb_ciLookup, contact_typeList, stateList, impact, urgency, assignmentGroupLookup, assigned_toLookup, short_description, description)
logger.info("SN Incidents POSTED")
print("SN Incidents POSTED")
except IOError as e:
logger.error(e)
print(e)
pass
finally:
pass
# endTry
###end main- Comment all of the above and create your own function.
if __name__ == "__main__":
main()
Result being a chunk of code that you can paste in your python scripts that can make your integrations from other tools or actions to send data into ServiceNow a fairly cookie cutter process.
Next:
I hope this helps you speed up your own integrations. For my part, I will continue to learn and create new functions, update this script or create a script repo on my GitHub. To download the complete script in its raw format, here is the link where I will keep the latest updated file for your download.
https://github.com/dtembe/ServiceNowEmailEventIncident
Disclaimer:
The opinions expressed here are my own and not those of my employer. As I research and learn different technologies my thoughts and opinions will change. As technology changes, so will my opinions.
I also offer no warranty or support on this code. Thanks for taking the time to read.
Dan