Link Analysis vs Graph Database - not the same, Roger Rabbit.

Many predict that graph databases will be at the center of digital adoption activities when/as the hype of AI rolls into widespread adoption. There are many reasons for that but also many obstacles to overcome. Probably the most challenging comes from how many “tech evangelists” conflate link analysis as "graph visualization." Understanding the difference really matters for organizations who want to future-proof their digital investments. So let's explain. The questions are these: what is the difference between a link analysis and a graph database? Why does it matter? And why are graph so incredibly powerful, especially in National Security.

Link Analysis

While both deal with identifying and depicting the relationships between data points in a way akin to a system diagram, the data used in a traditional link analysis sits squarely on a "vanilla" relational database. It helps to think of relational database like spreadsheets. Each data entry is a row, and the columns act as "keys" to relate entries to one another. So how the data can be queried is limited by the selected "keys" and how otherwise well-structured (or "clean") the data is. It also means this data is not stored in an AI-friendly way.

Graph Database

Conversely, a graph database is a specialized database management system optimized for storing and querying graph data in a way that gives equal value to the relationships and the data itself. In this database, relationships are data entries too, meaning that they can be queried to expose truly unanticipated patterns and iterate into new emerging ones, something not possible in relational databases. Graph databases therefore empower the analysis of both structured and unstructured data, with location analytics to boot (in the case of Esri's ArcGIS Knowledge solution). And they already incorporate AI superbly.

The “main” (conceptual) difference

If these previous explanations on the power of graph databases are still a bit obtuse, let me try a different way. In fact, inspired by Donald Rumsfeld and his famous knowns and unknowns, I’ll jump to conclusions:

To do a link analysis you must know what answers you are looking for. What you are after are known-unknowns. To use a graph database you must only know what questions to ask. What you are after includes unknown-unknowns.

An example

For example, let's imagine that Roger Rabbit is investigating a theft in the Toontown museum. Using link analysis, Roger could examine communication records, financial transactions, and correlate suspects in place and time, all in separate and procedural (linear) steps, to find people (or talking taxi cabs) that might be involved, confirm motives, dispel alibis, etc. Traditional investigative work: "Check"

But if Roger was just a rabbit with no investigative experience, but still had a lot of creative and critical thinking skills? He could build a graph database with nodes for each suspects, the museum and the artifact, and also described the relationships between those "nodes". Doing so, he could then query the data, ie "ask it questions" either using pattern finding algorithms or AI. Doing so Roger would see patterns or relationships that he could not have anticipated (from a lack of experience, an overly complex system to study, or just a sheer lack of beta carotene). He would catch the thief, but ALSO expose the scale of the corruption that gave motive to the crime and take action on the system that enables it. Complex system change: "Check !"

Now this example was quite "cartoonish.” But in more complex and real-life situations graph databases are even more compelling. For example, imagine trying to understand the perceived resilience of certain supply chains amidst economic sanctions... with all the data of a national security apparatus. Or finding the source, mechanisms and social habits relevant to a disease spreading in a country like Canada, taken by 500% increases in forest fires, devastating floods, and either tropes of mosquitoes or black flies, or ticks ... or geese ! Darn them!

Graph Databases are therefore key to finding emerging patterns in ultra-large and diverse data sources to make sense of the unknown. It’s right up there in the realm of complexity and systems thinking. And it seamlessly integrates location.

What of Entity Resolution?

There is another thing super important to graph databases: its called "Entity Resolution." Its a bit funky to understand for anybody who hasn't filled a swear jar trying to sort through poor data. To put it simply, it a sort of "automated-on-the-fly" tool for record linkage or deduplication. I recommend anybody interested look at this example of maritime analysis, or have a chat with Lexxi Clement, Tim Murphy , Amy Clarke : https://lnkd.in/gF6kz9DG.

Conclusion

In conclusion, the widespread anticipation of graph databases taking center stage in digital adoption post-AI hype is well-founded, but often misunderstood. While commonly conflated with link analysis, graph databases represent a radical departure (or I’ll say it – a paradigm shift), offering unparalleled capabilities in uncovering unknown-unknowns. Unlike link analysis, which relies on predetermined queries for known-unknowns, graph databases empower users to explore data dynamically, revealing unanticipated patterns and insights. This distinction is pivotal, as demonstrated by scenarios ranging from criminal investigations to complex geopolitical analyses. Moreover, the importance of entity resolution in enhancing data accuracy cannot be overstated. As exemplified by pioneering projects such as maritime analysis, the transformative potential of graph databases continues to expand, promising new frontiers in data-driven decision-making.