Snowflake Powered by Cloud and driven by SQL -Things to know

Snowflake is the perfected product that is powered by the Cloud and Driven by SQL which lets you manage your data like an Uber ride.

What is Unique about Snowflake?

Snowflake has created a perfect marriage between Cloud and SQL. Snowflake is currently available on Aws, Azure, and GCP. Below are some of the points which makes it unique:

Architecture

1)    Snowflake has a unique 3-layer architecture where each layer can scale independently

2)    The 3 layers are the Cloud services Layer which is the brain of Snowflake, the Compute layer which is the engine of Snowflake, and the Storage layer.

3)    Snowflake has de-coupled Storage and Compute; storage costs are very cheap and for compute, it charges on a per-second basis so essentially you are only paying for the time you are using it and that’s the magic.

4)     Each business unit can have it's own compute (Virtual warehouses) but still, reference the same storage, so each department is being billed separately as previously it was a bone of contention during the operational budget discussion.

5)    The Computer layer supports Auto-scaling where a cluster can be added to a virtual warehouse within 6 minutes and you can have both Economy and Standard scaling which lets you decide how the scaling out of clusters will work.

6)    Snowflake stores data in Micro-Partitions which are blocks of data of 16 MB in size, 50MB to 500MB of uncompressed data. Each block represents a set of rows from the table and it also stores a set of statistics of each field, like the min and max values, number of distinct values.

7)    The Cloud services act as a Centralized Metadata Management, it stores all the metadata for it’s a customer in a key-value store, storing data in this manner let’s snowflake do Zero-copy cloning where you can just copy the metadata from one environment to the other and still point to the same data, this lets you avoid moving data between environments which is both time-intensive and costly.

Data Loading

1)    Snowflake support both Structured and Semi-Structured data as a source.

2)    You can load data in Snowflake from both External (AWS, Azure, and GCP) and Internal Stages (Storage owned by Snowflake offered in AWS, Azure, and GCP)

3)    Snowflake supports Continuous Data Loading using Snowpipe where you can connect to both External and Internal storage to load data continuously.

4)    Snowflake supports Change Data Capture (CDC) using Streams where you can load only changed data from Staging tables.

5)    Snowflake offers both Schema on Write and Schema on Read so you always do not need to load data into snowflake to do analysis.

Maintenance

1)    You do not need to Partition and index any table in Snowflake.

2)    Snowflake uses the roles ACCOUNTADMIN, SYSADMIN, or SECURITYADMIN to manage account, user, and data security

2)    The Zero-Copy-Cloning lets you refresh one environment from the other by not copying data but just Metadata so you do not need to run the same process in the lower environment to load data for any bug fix or testing a change in business logic.

3)    Snowflake features Time-travel and Fail-Safe which are independent of each other but helps you revert to a previous state, Time-travel helps retrieve deleted data, and Fail-Safe acts as disaster Recovery. These properties are not mutually exclusive.

4)    Snowflake lets you Share Data with anyone irrespective of the fact if you are a Snowflake Customer or not.

5)    Snowflake Upgrades are applied without any downtime or impact to customers.

6)    Snowflake is continuously expanding its portfolio of Security & Compliance Reports, currently, it supports HIPAA, SOC 1 Type II, SOC 2 Type II, PCI-DSS, ISO/IEC 27001, and FedRAMP Moderate.

 Lastly, Snowflake is a game-changer in the SaaS service market in the Cloud Datawarehouse space and it’s simple in nature that is its biggest selling point as Data management need not be complicated and time-consuming. Your Analytics requirement may or may not need all the functionalities it offers and as it has decoupled all the architectural layers you can use it as per your need and pay for the services in a pay-as-you-go model to give you an Uber ride experience.