Test Cluster Setup - aerospike-community/aerospike-hadoop GitHub Wiki

In the following sections, we describe in detail how to test the Aerospike Hadoop Map-Reduce Connector.

Apache Hadoop Cluster

We will start by installing a 3 node standalone Apache Hadoop cluster.

Edge Node

We will then install a 4th Edge Node. It will be configured to access the Apache Hadoop cluster and run map-reduce jobs on it.

Aerospike Server

To minimize the number of nodes deployed, we will also install the Aerospike server on the Edge Node, though in production, Aerospike should be its own cluster.

Hadoop Connector

On the 4th Edge Node, we will also install the Aerospike Hadoop Connector.

Users on Hadoop Cluster and Edge Node

As part of the installation and configuration of the Apache Hadoop Cluster and the Edge Node, we will add two users - hduser and hdclient. hduser will be the hadoop administrator with sudo privileges. hdclient will be a typical hadoop developer who will access the Hadoop Cluster from the Edge Node.

Hands-on Examples

We will demonstrate the examples discussed in the Examples Overview section using the Aerospike Map-Reduce Hadoop Connector.