Big Data Performance

/Big Data Performance

Big Data Workload Simulations and Cost Estimates Now Available for EC2 Instances

By | 2018-01-04T22:57:59+00:00 December 18th, 2017|Categories: AWS, Big Data Performance|Tags: , , , , |

Figuring out the right kinds of cloud machines (commonly known as instances) to run your Big Data workloads. To get the best price to performance ratio is often a lengthy trial and error exercise. To that end, MityLytics MiCPM© can now simulate your Spark and Hadoop (more to come) application workloads. Simulation can be on various [...]

Configuring YARN Capacity Scheduler Queues in AWS EMR

By | 2017-12-01T16:01:50+00:00 November 2nd, 2017|Categories: AWS, Big Data Performance, EMR, Hadoop, Scheduler, Spark|

Introduction AWS EMR clusters by default are configured with a single capacity scheduler queue and can run a single job at any given time. This blog talks about how you can create and configure multiple capacity scheduler queues in YARN Capacity Scheduler during the creation of a new EMR cluster or when updating existing EMR clusters. [...]

MityLytics for IoT Performance and Scalability Testing

By | 2017-10-26T12:37:23+00:00 October 23rd, 2017|Categories: Big Data Performance, Cassandra, Hadoop, IoT, Kafka, Spark, Storm|

In speaking with folks running IoT stacks, which typically span several streaming and NoSQL technologies such as Kafka, Spark, Storm and Cassandra, it became apparent that they often struggle with understanding how an existing setup scales and performs with varying workloads. So to address this, we enhanced our performance testing module to now offer IoT sensor [...]

MiCPM™ is here, and MityLytics is excited to invite you to trial MiCPM for Spark clusters!

By | 2017-10-10T15:48:40+00:00 May 15th, 2017|Categories: Big Data Performance, Devops|Tags: , , , |

Thanks to our partners at Packet.net, MityLytics can offer you a test drive of MiCPM with Spark-HDFS clusters. MiCPM is a zero-touch, point and deploy SaaS platform that provides simulation and prediction capabilities for workloads and performance tuning at the touch of a button. With MiCPM, your clusters run more efficiently, reducing infrastructure TCO and increasing [...]

Reduce Costs by Improving Your Analytics Efficiency

By | 2017-10-10T15:49:33+00:00 March 30th, 2017|Categories: Big Data Performance, Devops|

According to IDC, companies are expected to be spending over $187B on Big Data and Business Analytics software by 2019. Approximately $55B of that is expected to be spent on software, with another $28B to be spent on hardware and the balance spent on IT services. With such large numbers, even incremental gains in efficiency can [...]

Stress Testing Kafka, Spark and Cassandra on Bare Metal

By | 2017-06-17T23:03:47+00:00 November 30th, 2016|Categories: Big Data Performance|

We were introduced to Packet, a cloud-provider that offers bare metal servers in the cloud, so we decided to build a data pipeline testbench in an increasingly popular configuration (Kafka message broker, Spark for streaming and Cassandra for data storage) using our software and share our observations and experiences. To learn more about the study, please [...]

Performance Prediction and bottleneck identification for Spark, Hadoop and Hive

By | 2016-12-01T13:31:03+00:00 September 14th, 2016|Categories: Big Data Performance, Devops|

Characterize and predict performance, identify bottlenecks as you scale up your, do all that without actually spinning up your cluster. A way for you to correctly size your resources (CPU, RAM, Storage and Networking) as your dataset grows. Illustrated in this talk with benchmark suites. If interested drop us a line, we are working with a [...]

Keeping up with open source software for Big Data

By | 2016-12-01T13:31:28+00:00 November 5th, 2015|Categories: Big Data Performance, Devops|

Hello from MityLytics again! Have been meeting a lot of people involved in the analytics space and one thing stands out, the field is definitely evolving at breakneck pace with various degrees of backward compatibility. Trends seem to catch on and then just as quickly disappear or become obsolete or need to be plugged into something [...]