Distributed Storage Cluster

Hey…

        In the technical world big corporation and big companies are facing most common problem is called as BIG-DATA.

Big data is nothing but it’s just huge amount of data. Let me give just Example –

1)     Facebook  revealed some big, big stats on big data to a few reporters at its HQ today, including that its system processes 2.5 billion pieces of content and 500+ terabytes of data each day. It’s pulling in 2.7 billion Like actions and 300 million photos per day, and it scans roughly 105 terabytes of data each half hour.

2)  A place where google stores and handles all its data is a data center. Google doesn’t hold the biggest of data centers but still it handles a huge amount of data. A data center normally holds petabytes to exabytes of data.Google currently processes over 20 petabytes of data per day

 So a large amount of data is not critical, the rather critical part is how organizations are using this data. So a large amount of data is not critical, the rather critical part is how organizations are using this data. There are challenges to managing such a huge volume of data such as capture, store, data analysis, data transfer, data sharing, etc. Big Data follows the 3V model as “High Volume”, “High Velocity” and “High Variety”.

To solve this BIG-DATA type problem Facebook and Google are used DISTRIBUTE STORAGE with the help of MASTER SLAVE MODEL.

A distributed storage system is infrastructure that can split data across multiple physical servers, and often across more than one data center. It typically takes the form of a cluster of storage units, with a mechanism for data synchronization and coordination between cluster nodes.

Hadoop Distributed File System (HDFS) – a distributed file-system that stores data on commodity machines, providing very high aggregate bandwidth across the cluster;