EMR 007 Decommisioning a particular EMR core node - qyjohn/AWS

You would like to gracefully decommission a core node from your EMR cluster. This can be achieved using the following steps:

(1) Enable the cluster termination protection for the EMR cluster

(2) SSH into the master node, run the following commands to stop service-nanny and instance-controller

sudo service service-nanny stop
sudo service instance-controller stop

(3) On the master node, edit /emr/instance-controller/lib/yarn.nodes.exclude.xml, with the following information:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<hosts>
<host>
<name>ip-10-191-222-151.us-west-2.compute.internal</name>
</host>
</hosts>

(4) On the master node, edit /emr/instance-controller/lib/dfs.hosts.exclude, with the following information:

ip-10-191-222-151.us-west-2.compute.internal

(5) On the master node, run the following commands:

hdfs dfsadmin -refreshNodes
yarn rmadmin -refreshNodes
hdfs dfsadmin -report

(6) Looking into the output from HDFS report, confirm that the desired data node is decommissioned.

(7) Terminate the EC2 instance from the EC2 console

(8) On the master node, start service-nanny (instance-controller will be started automatically by service-nanny).

sudo service service-nanny start

(9) On the master node, start HDFS rebalance

hdfs balancer -threshold 1

EMR 007 Decommisioning a particular EMR core node - qyjohn/AWS_Tutorials GitHub Wiki