When the Database Costs

Our organization focuses heavily on training and so our primary database is designed around training management.  The greatest resource for training is time.  When things are managed properly more time is available to conduct more training.  When things aren’t managed properly then less time is available for training, making the organization less effective when called upon to perform.

The training database is centrally controlled, closed sourced and made by the lowest bidder making it difficult for an end user or lower level supervisor to conduct any analysis on its structure or make recommendations for improvement.

The database is accessed through a graphical user interface via a web browser that was designed around the workflow of the programming engineers and not the end user.  The site was coded to use modern HTML standards, but our work hasn’t authorized upgrading beyond Internet Explorer 9 making the site slow and its performance choppy.  But while the HTML coding may be modern the other technology its build upon has been abandoned by today’s most popular browsers because its simply out dated.  Yes, streamlining the acquisitions process is important.  

The site grants access to leadership six levels up and grants larger administrative rights to those in higher levels of leadership than those who are most familiar with the data.  This has resulted in frequent data loss as someone unfamiliar with the information will accidentally delete large amounts of data requiring individualized manual re entry through the aforementioned clunky GUI.

The whole experience has convinced several members of the organization that databases are not cost effective and to continue to rely heavily on spreadsheets and printed documents.  There is no good way to customize a summary of information using the database, whereas a spreadsheet with a few formulas and conditional formatting can quickly communicate areas of emphasis and allow managers to correct issues at the lowest level.

This database technology is responsible for costing the organization a great deal of time in retraining and creating unrewarding jobs of redundant data entry using a frustrating graphical user interface.

In a recent interview Nate Silver commented on how it took human skillsets a while to catch up to the capabilities of technology and really be able to capitalize on their full value.  Just because someone can crunch the data doesn’t mean they can understand it.  This isn’t a problem for my organization since we’re still learning principles of data entry and preservation.  

Because of this learned incompetence we also don’t have to worry about something else Nate Silver talks about, big data and false correlations:

“I think one of the false promises that was made early on is that, well if you have a billion data points or a trillion data points, you’re going to find lots and lots of correlations through brute force. And you will, but the problem is that a high percentage of those, maybe the vast majority, are false correlations, are false positives. Where there could be significance, but you have so many lottery tickets when you can run an analysis on a trillion data points, that you’re going to have some one in million coincidences just by chance alone. If you bet all your money on them, you might wind up looking very foolish in the end.”

Thankfully I don’t work for an organization that will someday look foolish because of false conclusions.  I work for an organization that is struggling with data entry.

The role database technology should play within the organization is efficiently capturing, storing, and making accessible the data that will allow the organization to move through the Data-Information-Knowledge model.

In 1980 economist Thomas Sowell began his book Knowledge and Decisions with the statement that “Ideas are everywhere, but knowledge is rare.”  Throughout most of the book Sowell explains that decision makers can only make decisions based upon the information they have available to them at the time they make a decision.  In 1980 the ideas outweighed the information available to use them.  Today the same may still be true.  Knowledge may still be rare, but the data needed to create knowledge is certainly more prevalent now than at any point in human history.

Someday I hope to work in an organization where the data is compiled for human consumption allowing the Data-Information-Knowledge model to work effectively.

this post originally appeared at jfroecker.com