The Golden Age of Computing (for Security Analytics)?

I have said this statement to a few customers recently, and its something that I firmly believe in. Our computers are more powerful than they have ever been and we have access to high performance systems at a price that we couldn't previously imagine! Whether it is in the cloud, on site or virtualized, its never been more available at a decent price!

This is good, isn't it? Well yes and no. It is good because we are able to do things that we could only imagine previously and we can now start to address issues and problems we know we have. But its also bad, and not necessarily for the obvious reasons. 

Why do I say it's bad? Well, I am not suggesting its all bad, far from it. In fact, the positive aspects far outweigh the negatives. But the problem is around how we do things and ultimately why we do it. A lot of organizations are looking at Analytics to solve complex and difficult problems, especially in information security. The world of security analytics has exploded in the last few years and is set to continue going forward. So surely, with cost effective high performance systems, doesn't this help solve the information security problems we have? Can we not use security analytics to solve these?

There in lies the problem though; we aren't going to solve anything today. Yes, we can deploy systems to allow to crunch numbers at a rate we could only dream of only a few years ago, but if it's not actually going to support what we are trying to solve; does it do anything? 

The problem here is just like anything else, new toys, shiny things and some features you haven't had before. What can I do? What can I see? Surely I do need to store 3 years worth of data? What do you mean, running baseline comparisons on every users activity for the last year isn't going to determine anything? The list goes on.

The challenge is around building up and solving problems. We need to bring security analytics to help us solve problems, but usually in small reasonable and fine grained chunks. While we may be able to bring massive compute capabilities and storage to bear, unless we know what we are trying to solve, we will struggle to solve it! This is often referred to as "finding the needle in a stack of needles" problem. The way to solve this is to use security analytics to identify the indicators. We aren't going to identify an attacker or a breach with analytics alone. We might but this isn't good enough. We have to build up a number of indicators so that we have a much more conclusive picture to make a decision. Yes, the fact that someone is uploading a considerable amount of data to the internet is unusual, but coupled with the previously suspicious network activity AND that they have been accessing systems outside of their role; now that's a good set of indicators that mean we can start to investigate. Importantly though, this isn't all from analytics. Its likely that we will have indicators from other sources; threat intelligence, internal assessments, an SIEM, HR data, log data and more. 

And that's the way to use security analytics. Use it to help solve the questions we couldn't answer before. Help us make accurate decisions around what has happened and what is about to happen, but in the context of other activity. We now have the computer systems and storage to support doing vast amounts of processing, but in this case we don't need to spin up a 50 node cluster and access 10 PB of storage. We can actually make security analytics work for us in a much more contained, controlled and predictable way. Focus on solving the indicators and we start to make out the wider picture - we don't need to understand everything on all of the data, but we can start to break it down to solve the problem we couldn't previously.

Just because we could store 10 years worth of activity data and run analytics across it, doesn't mean we should. Building up indicators absolutely will give you a much better view and it won't cost the earth either! Yes, we are in a golden age of compute power, but we don't need it all to solve security analytics problems. We just need some of it......

And the picture at the top? Thats a comparison of where we have come from. That's an Osborne portable computer and an older iPhone; 25 years apart and the iPhone has 100 times the clock cycles, 1/100th the weight and 1/10th of the price! We really have come a long way!