Analytic Modernization Jam Session Summary

At the 2017 USGIF Symposium, Ben Tinker and Jonathan McColgan from OGSystems and I hosted an interactive workshop to discuss the topic of "Analytic Modernization." The idea came out of a conversation with Jackie Barbieri and Victoria Nguyen of Whitespace Solutions. We thought that we should try something a little more exciting than a standard meeting for the last day of the symposium. I had experience in "Visioneering" from off-sites with the government. In these sessions, a graphic artist visually depicts a free flowing conversation and tries to hit the high points with a colorful and creative depiction. These wall-sized pictures are fun, but they also can't capture all the nuances of the conversation. Let me walk you through the highlights. Remember: it's a non-attribution meeting so we won't mention names.

One of the first things questions was "do we mean NGA or outside NGA?" We agreed that we meant "the GEOINT Community" but the discussion was admittedly slanted in the direction of big intelligence processes.

Which brings us to the first point: one of the analysts mentioned that Imagery Analysts (IAs) and Geospatial Analysts (GAs) have different workflows, but they get intertwined. IA's are spending a lot of time doing GA-like work to condition their data. We asked how much time people spend doing "not analysis" and the answer was "more than we'd like." The community wants creative solutions to reduce this entropy. We've known that for a while, but ongoing conversations need to identify specific elements of the workflow(s) that can be streamlined. Since there is already an AI and machine learning working group, we tried to avoid too much discussion about automation, but if you look at our picture (AWWW!) you can see humans and computers holding hands and working together. Look at that happy little computer!?

Next, we discussed labels. Are IA's and GA's too limiting? Shouldn't we all be all-source analysts? Does the law allow us to without getting our hands slapped? (Thankfully we avoided a three-hour throw down on multi-INT vs. all-source).

One of our firsts agreements is that we need to identify the skills needed to solve problems and develop policies and workplace constructs that encourage self-forming teams. The problems are evolving so quickly that the Government needs flexibility and agility in how they put people towards problems. We need to be more dynamic and responsive in how we tackle problems. We don't need one analyst to know everything, we need them to easily be able to find and assemble the teammates they need to solve a problem.

The discussion then shifted to whether we are problem-driven, question-driven, mission-focused, or some other hyphenated thing. One participant said "outcome driven" and we went with that. Instead of being driven by counting the number of products delivered, we should be focused on the outcomes. What is the value of what we're producing and how do we know? An international partner chimed in and said that they have a very robust system for rating delivered products in his country. Close coordination with customers provides rapid feedback (Agile anyone????) that lets the analysts tailor the products and continue to improve them. I admitted that we don't do this as well in the U.S. and sometimes we purposely ignore instrumenting systems to provide metrics and feedback. No one really likes to know the computer is watching them and grading them. :/

Another analyst dropped the bomb that we're not spending enough time learning and a lot of the audience agreed with the sentiment. With the crush to produce outputs every day, we're not setting aside enough time to learn new techniques - especially in managing data. A participant said "We need a GEOINT Doctrine for the Age of Big Data." Boom! We discussed how the nature of data and products is changing, and before we flood the analysts with new tools we need to work with them to be more comfortable with more and disparate types of data. This is a challenge in almost every industry. Fortunately for us, geospatial data is everywhere and there's a lot of non-traditional players entering the space... but we need to find time to learn from each other and share.

I asked a government rep if I would be allowed to put "number of risks I took this year" in my annual performance appraisal instead of "number of products I made." We talked a little about how to incentivize risk taking. One of the challenges I see is that analysts always tell me that they won't put their name on something unless it's 100% perfect, because in the Intelligence Community, reputation is everything. After hearing this from hundreds of analysts, I get it, but engineers are like "if my thing breaks I'll just build a better thing and it's OK if it breaks." I think this cultural disconnect is causing engineers and analysts to talk past one another. Should we be driving a culture that takes risks or a culture that strives for perfect on the first try? Can you have both? When is each appropriate? I think this notion of establishing risk tolerance - formally - is going to be a major discussion point going forward.

Wrapping up the session, a law enforcement partner asked if we were incorporating lessons learned from outside the I.C. Doh! Sometimes, we lose track of those external partners in homeland security, law enforcement, and disaster response. We talked a little about how they need new tools for organizing and visualizing information... and their budgets are not like those of the national agencies.

Jonathan summed up the session by saying "we came here today to have a session about Activity-Based Intelligence (ABI), Object-Based Production (OBP), Structured Observation Management (SOM), and analytic models... but we just talked for an hour and used none of those words." Whoa. Mind-blown! I guess we were outcome-driven instead of buzzword-driven! Awesome.

As we concluded the session we took an informal feedback poll - like our international partner taught us - see how quickly and dynamically we incorporated lessons learned? A majority of the audience agreed:

1. I learned something new today.
2. I had FUN today.
3. I would do this again!

So, this was a fun and informative way to wrap up our crazy week. We had excellent participation from the U.S. Government, foreign partners, academia, and industry. I thought it was a great example of open communication. The USGIF Analytic Modernization Working Group meets monthly in Dulles, VA and on a teleconference. You can find out more info about the group and our meetings at:

http://usgif.org/community/Committees/_ABI

If you were in the session and remember anything I forgot, please comment below!