Encryption – Is it a 100% defense for sensitive data?

When checking your email over a secure connection, or making a purchase from an online retailer, have you ever wondered how your private information or credit card data is kept secure?

Our information is kept away from prying eyes thanks to cryptographic algorithms, which scramble the message so no-one else can read it but its intended recipient. But what are these algorithms, how did they come to be widely used, and how secure really are they?

Encryption, while not a physical defense, is much like other basic defense mechanisms that serve to block access to items of value. Like other basic defenses, encryption is not foolproof. It can be evaded and undermined, and it can be prone to errors in deployment; encryption keys can be lost, stolen, or inadvertently exposed. Perhaps even more likely is a situation where we believe we’ve encrypted everything, when in fact we’ve encrypted almost everything. Most encrypted data is unencrypted at some point in its usage lifecycle. The bad guys are pretty good at finding the one window left open.

Analytics are to encryption what intelligence services are to military defenses. The increasing number, variety, speed, and severity of cyber attacks necessitate a dynamic cyber intelligence posture. In the past, cybersecurity analytics were focused on gathering data about compromises, developing threat “signatures,” and using those signatures to protect against future threats, all comprising another form of defense that served to block an attacker.

Identifying threats in real time

Advanced detection analytics, by contrast, identify emerging threats by recognizing anomalous patterns in real time. Many of these techniques have commercial and technical roots in high-volume network assurance applications (e.g., telecommunications) as well as financial fraud detection (e.g., banks and insurance). While many firms label their signature based detection methods as “analytics " the analytics are largely static and built to block known threats and therefore fall into the category of basic defenses.

What differentiates the emerging field of detection analytics from these basic defenses (including physical security, firewalls, encryption, and signature-based detection methods) is that advanced detection analytics are focused on finding anything unusual or threatening that gets by your basic defenses. And since we brought Snowden into this already, let’s include those threats that emerge from the inside.

It’s worth noting that these analytic approaches are tried and tested. Many of the underlying technologies, including the AI/machine learning analytics, have been protecting most of the world’s credit cards for years. The fraud teams at card issuers use these systems not only to detect fraud, but to set the level of risk that triggers investigation or card blocking, in order to balance loss prevention with a positive customer experience. Moreover, these fraud systems do not require issuers to hire armies of analytic techies. By crunching data to prioritize the biggest threats, they simplify the lives of fraud professionals, and the same would hold true in information security.

While encryption and other basic defense approaches will always have their place in security strategies, encryption alone does not prevent hackers from stealing data. Adding advanced analytic techniques to cybersecurity portfolios complements and can close the gaps left by encryption (and signature-based security) by detecting emerging and evolving attack patterns in real time. As a best practice, companies must advance beyond basic defenses, and enhance their security posture with the analytic equivalent of an effective intelligence service. It’s time to bolster our walls and moats with spies and intelligence. But finding that middle ground has proved challenging as the tech giants’ goals seem to conflict with what the government wants. While the tech giants argue that stronger encryption tools better protect their consumers, they have also pushed that opening up encryption standards would give cybercriminals the ability to exploit people and would also pave the way for police and the government to act as “Big Brother” and spy on those people.

Encoded messages

The first cryptographic methods actually go back thousands of years to the time of ancient Greece. Indeed, the word “cryptography” is a combination of the Greek words for “secret” and “writing”.

For example, the Spartans famously used a system where they wrapped a piece of papyrus around a staff of a certain girth, and wrote their message down the length of the staff. When the papyrus was untraveled, the message was jumbled until it reached its destination and was wrapped around another staff of the correct circumference.

Early encryption algorithms like these had to be applied manually by the sender and receiver. They typically consisted of simple letter rearrangement, such a transposition or substitution.

The most famous one is the “Caesar cipher”, which was used by the military commanders of the Roman emperor Julius Ceaser. Each letter in the message was replaced in the encrypted text the ciphertext by another letter, which was shifted several places forward in the alphabet.

But over time such simple methods have proved to be insecure, since eavesdroppers – called cryptanalysts – could exploit simple statistical features of the ciphertext to easily recover the plaintext and even the decryption key, allowing them to easily decipher any future messages using that system.

Modern computing technology has made it practical to use far more complex encryption algorithms that are harder to “break” by cryptanalysts. In parallel, cryptanalysts have adopted and developed this technology to improve their ability to break cryptosystems.

This is illustrated by the story of the Enigma cryptosystem used by the German military during the Second World War, as dramatized most recently in the movie The Imitation Game.

Enigma’s relatively complex encryption algorithm was implemented using electromechanical computing technology to make it practical for German military communications. An extension of the same technology was used by the “bombe” machines of the British cryptanalysts to make it practical to break the cipher.

Current cryptosystems

The cryptosystems in wide use today have their origins in the 1970s, as modern electronic computers started to come into use. The Data Encryption Standard (DES), was designed and standardized by the American government in the mid-1970s for industry and government use. It was intended for implementation on digital computers, and used a relatively long sequence transposition and substitution operations on binary strings.

But DES suffered a major problem: it had a relatively short secret key length (56 bits). From the 1970s to the 1990s, the speed of computers increased by orders of magnitudes making “brute force” cryptanalysis which is a simple search for all possible keys until the correct decryption key is found increasingly practical as a threat to this system.

Its successor, the Advanced Encryption Standard (AES), uses minimum 128-bit keys by contrast, and is currently the most popular cryptosystem used to protect internet communications today.

Key problem

The AES also has limitations. Like all earlier cryptosystems, it is known as a symmetric-key cryptosystem, where the secret key is known to both the sender who encrypts the message (lets call her Alice), and the receiver who decrypts the message (lets call him Bob).

The secret key, being secret, cannot simply be exchanged over a public communication channel like the internet. If that was intercepted, that would compromise all future encrypted messages. And if you want to encrypt the key, well that produces another problem of how to secure that encryption method.

So, Alice and Bob must first use a private communication channel, such as a private meeting in-person, to exchange the secret key before they can use the cryptosystem to communicate privately. This is a significant practical hurdle for internet communications, where Alice and Bob often have no such private communication means.

To overcome this hurdle – known as the key distribution problem – an ingenious different type of cryptosystem, called an asymmetric-key, or public-key, cryptosystem was devised in the 1970s.

In a public-key cryptosystem, the receiver Bob generates two keys: one is a secret key that Bob keeps to himself for decryption; while the second is a public encryption key that Bob sends to Alice over a public channel. Alice can use the public encryption key to encrypt her messages to Bob. But only Bob can decrypt it with his private key. It thus provides a solution to the key distribution problem of symmetric-key cryptosystems.

In practical applications, due to the higher computational demands of public-key systems compared to symmetric-key systems, both types of cryptosystems are used. A public-key cryptosystem is used only to distribute a key for a symmetric key system like AES, and then the symmetric key system is used to encrypt all subsequent messages.

Consequently, the resulting privacy depends on the security of both symmetric and public key cryptosystems in use. The most commonly used public-key cryptosystems in use today were devised in the 1970s by researchers from Stanford and MIT. They are known as the RSA cryptosystem (from the initials of the designers, Ron Rivest, Adi Shamir, and Len Adleman) and the Diffie-Hellman system, and make use of techniques from an area of mathematics known as number theory.

The future of cryptography

I see two potential developments which may have a major impact on cryptography.

The first one relates to the development of quantum technology. In a breakthrough theoretical result in the 1990s, the mathematician Peter Shor demonstrated the potential of a large scale quantum computer. This exploits the principles of quantum mechanics to solve the integer factorization and discrete logarithm problems efficiently, thus rendering the RSA and Diffie-Hellman systems insecure.

While large scale quantum computing technology has not yet been realized (and prospects for its realization remain unclear), the impact such a realization could have on cryptography cannot be overestimated.

Fortunately, researchers already have devised two possible approaches to deal with this problem if it arises in the future. One is the development of public-key cryptosystems that are believed to be secure even against quantum computing attacks. The other is quantum cryptography, a communication technique that relies on physical assumptions and the laws of quantum physics to provide security.

The second potential development relates to the increasing usage of cloud computing. Unfortunately, unless encryption is used to protect our stored private data, the privacy of that data from the cloud server (or any other entity having access to the cloud server data, such as a hacker) is compromised.

The use of conventional encryption algorithms by the user, on the other hand, also has the side effect of preventing the server from performing useful processing on the data for the user (e.g. to search the data). New types of cryptosystems are currently under development by the cryptographic research community to overcome this apparent paradox by allowing the server to process the encrypted data without revealing the data to the server.

As you can see, the state of the art in cryptography is currently strong enough to protect most of our email and online transactions, but its future status isn’t entirely certain. The future – particularly quantum computing – may result in even stronger cryptographic systems, but also raises the prospect of new means to break them. The cat-and-mouse continues.