Detecting Fraud, Machine Learning & Big Data

Machine learning sounds great, but what does it mean in practice? I had the opportunity to get closer to this with a proof of concept I ran with a vendor who had a very clever implementation of machine learning in the fraud detection space.

Basic fraud detection in the asset management world in my experience involves hard coded rules that trigger reports for the internal fraud team to investigate, i.e. if a customer does this, then that and then tries to sell or transfer their assets and it's over a certain value then it might be a fraud and needs investigating. Inflexible and hard to maintain. The asset management industry currently has the luxury of T+2 settlement on many products, so the fraud team have a couple of days to investigate and prevent a payment going out the door. These processes are now not robust enough or scalable in this world of increasing threats, identity fraud and new intra-day products. A rich landscape of opportunities for Fintechs and established vendors who have clever products offering machine learning and AI capabilities.

Our focus was the detection of account takeovers and fraudulent networks. An account takeover is where the fraudster accesses your account either via the call-centre or on-line, makes changes to your data e.g. the address, telephone number, email or bank details and then attempts to sell or transfer those assets into the bank account they own. All done and dusted before the unsuspecting customer realises, with all emails and documents diverted to the fraudster.

So on to the Proof of Concept: We challenged the vendor to demonstrate the value of their product during a 1-week on-site PoC. To do this their product needed real data… something we protect above all. So we had to be both creative and rigorous to ensure our customers were protected, while keeping this to a PoC and not a full blown implementation.

• We reserved one of our secure rooms with key-card access limited to the team;
• The vendor brought with them their own hardware with their product pre-installed, these were fully virus checked on-site;
• To ensure that data could not leave our premises we agreed to keep their hardware after the PoC (purchased from them at cost price!) and to wipe the drives of our data and their software after completion, the machines were then re-purposed internally, thus protecting both parties and our customers;
• The hardware was not connected to our network and all wifi access was disabled. In fact the only way to get data in was via a secure encrypted data stick via a technician dropping in from the ceiling Mission Impossible style! (OK I made that bit up). We wiped and shredded the data stick afterwards (we have a nice machine for that!);
• All their on-site consultants went through our full vetting process, the same process we use for staff;
• We formalised the PoC into a contract so both sides were protected

That sounds like a lot of work and in practice this prep took a few weeks, but well worth it as it allowed us to move at speed once they were on-site.

Into their product we loaded 1 years’ worth of data history, covering all manner of customer data changes (address, bank, email etc), plus web logins and the timing of any calls made to our contact centre, mixed in with a sub-set of the associated core customer data.

We also provided 10 known cases of attempted fraud, 10 cases that looked like fraud but were investigated and found not to be (i.e. false positives). These helped the product “learn” and adapt its algorithms.

And here’s the sneaky bit. We also hid a further 10 known attempted fraud cases in the data and asked them to try and find them with their product during the PoC.

I was really keen to understand how the machine learning worked. The team came with an impressive CV of PHDs in maths, computing and AI, so please forgive my attempt at a simplified explanation of one of the models used to detect fraud!

Many products use advanced pattern matching coupled with the chronology of events and the time between them to build a multi-dimensional model. I can’t begin to put that into words so here’s my alphabetic analogy… So let’s say “A” represents an Address change, “B” a bank account change, “C” a call to the contact centre, “E” an email change, “P” an online password change, “S” a sell instruction and “W” a web login and so on. The product mines the data and assembles these interactions in chronological order so each customer gets a pattern. For example “CACEPWS” in my crude analogy represents a customer who called us, made an address change, called again, changed their email address, changed their online password, logged in online and sold some assets. So this pattern may be more "risky" than say CWWPWS. They also look for patterns within, which is where rule based algorithms fall down. If this all occurred in the last 5 days it would be a higher risk than if it was over the last 5 years. Hope you get the picture!

Once the software had trawled through the data, it sat there rather like an excitable puppy waiting to be trained… The consultants then fed the confirmed fraud attempts one-by-one like a doggy treat and then the false positives. Each time the product updated it’s algorithms about what a fraud and a false positive looks like. The great thing about products like these is that they can be applied to any industry: asset management, banking, online retail etc. It’s the machine learning capability that provides the context.

There were lots of options for measuring fraud and we chose a simple scale from 0 to 1000, where 0 is definitely not a fraud case and 1000 definitely is. We created a dashboard and picked a minimum figure of 800 that showed us around 50 potential cases for investigation. The big question: were our 10 hidden cases were there…?

Before I reveal the results, a quick word on fraud network detection. Using big-data concepts these products are able to draw a visual map of connections. So we know about joint accounts and customers sharing the same address, but what about customers sharing the same (or similar) bank accounts, telephone numbers, email addresses, post-codes or using the same IP address to login with? Pulling those together drew some fascinating hot-spots: An extended family or a fraudulent network? The software also looks for fuzzy patterns and is able to detect in near real time whether an account becomes connected to a suspicious network, e.g. by a seemingly innocent change of email address.

Only one member of the team from the fraud department knew what the hidden cases were. It was Friday, the last day of the PoC, the core team were quite frankly knackered and we held our breaths for the results while she reviewed the top 50, with nods, smiles, some frowns and head shaking..

For a bit of drama we did the big reveal "Strictly style" in front of a panel of key stakeholders and the vendor. We were all blown away with the results! The software had put 8 out of the 10 hidden cases in the top 50 with 1 near miss. With a bit more training it would have been 10 out of 10. It also threw up a number of new potential fraud cases and so we agreed to extend the PoC for a week so that the fraud team could review the findings and get to know the system better, create some dashboards and complete an evaluation.

For a live implementation products like this capture any data change or customer interaction in near real-time, score it and provide near real-time alerts and updates to dashboards that the fraud teams can monitor. Vastly improving our capability to react to any potential fraud attempt as it is happening.

All this while doing the day job: quite an exhausting week but with a really positive outcome. We also got a hugely valuable insight on working close up with the vendor before signing any major contracts, something not possible with a traditional RFI/RFP approach.

So here’s the big paradox: if you can achieve so much during a short PoC, why does it take 12-months to implement a new vendor product into a large FS organisation? Something I’ll come back to, as it doesn't have to be that way! And finally, if you found this article useful please click that like button! Regards Marc