Simplify and Accelerate Big Data Deployment

When the going gets tough, the tough get going.

Big Data has the potential to enrich our lives by enabling scientific discoveries, business models and consumer experiences. According to IDC, the market for Big Data will reach $41.5 billion by 2018, growing at a 26 percent CAGR, six times faster than the overall IT market. As Big Data deployments move beyond early adopters to mainstream adoption within enterprises, the complexity of implementing Apache Hadoop* and related technologies has become an even more significant problem.

 Why?

Because today’s infrastructure for most Apache Hadoop and Apache Spark deployments was not purpose-built to handle Big Data workloads or the needs of data scientists. As a result, converting enterprises’ existing infrastructure to handle tomorrow’s needs can often be expensive and time-consuming to implement. To address that challenge, modernizing and virtualizing the infrastructure for these Big Data workloads will become a key mechanism for organizations to future-proof their Big Data investments.

Given Intel’s vested interest in Apache Hadoop and Apache Spark, and our engagement with these communities and ecosystems (see Intel’s previous Cloudera and Spark announcements) to drive the foundation of Big Data and analytics in enterprise, we are exploring solutions to help the transition from early adoption to broader adoption and enterprise deployment become a reality.

Today, Intel announced that we are collaborating with BlueData to simplify and virtualize Big Data infrastructure. The collaboration will bring together BlueData’s infrastructure software for Big Data with the leading data center architecture based on Intel® Xeon® processor technology. The goal is to accelerate the adoption of Big Data solutions by making it easier, faster, and more cost-effective to deploy the applications and infrastructure required for Big Data analytics.

Intel technology like the Intel® Xeon® processor E5 family is built to overcome

key virtualization challenges such as network latency, infrastructure security and power inefficiencies. It also provides a high-performance, secure, robust foundation for Big Data analytics.

With the BlueData EPICTM software platform, BlueData simplifies the infrastructure complexity that can slow down Big Data deployments. Here is how:

• Works with all of the major Apache Hadoop distributions as well as Apache Spark.
• Enables self-service Hadoop or Spark clusters, helps maintain security and control in a multi-tenant environment, and improves hardware utilization.

• Integrates with the leading analytical applications, so data scientists can use the tools they prefer.

• Runs with any shared storage environment, eliminating the need to move data and reducing data duplication.

• Delivers the simplicity of Hadoop-as-a-Service or Spark-as-a-Service in an on-premises deployment model, with the enterprise-grade security and governance that IT teams require.

Together, Intel and BlueData are focused on bringing Big Data into the

mainstream and unlocking the value for our enterprise customers for faster time to insights with Big Data. BlueData will develop and optimize its Big Data infrastructure software for Intel architecture. The total solution enables easy deployment of Hadoop and Spark with greater simplicity and agility in a virtualized environment while delivering the security and performance that enterprise IT organizations require for their Big Data deployments.

My Big Data Technology team at Intel will be working closely with BlueData team on development, integration and testing of the optimizations in order to improve the function and performance of BlueData EPIC software platform on Intel architecture and other Intel platform technologies. We expect BlueData optimizations to enable Hadoop and Spark virtualization with close to bare metal performance, leveraging containers  and other infrastructure technology advancements.

I am very excited for this deep engagement and collaboration with BlueData, solving challenging problems and bringing more innovative solutions to our customers and the industry at large to accelerate Big Data deployment in the enterprise. For more about the collaboration, you can also refer to this blog post from BlueData’s CEO, Kumar Sreekanti.

Intel is committed to make Big Data analytics faster, easier and more insightful. Our Intel processors are the foundation of most high-performance data centers. We are also developing critical software components and solutions with Big Data and analytics ecosystem players to help get enterprises get the most from Intel® processor-based platforms.

Big Data analytics is transformational to all sectors and will drive new compute demand growth. We need hardware plus software optimizations and innovative solutions to scale-up and scale-out data analytics workloads.

Working together with innovative software companies such as BlueData, we can impact adoption barriers by making Big Data analytics cost-effective and easy to deploy.