Configuring Hadoop and Starting cluster services using Ansible Playbook

Ansible playbooks, inventory file, configuration file….. can be found at GitHub Link :

asks1012/ARTH_Task_11.1 (github.com)

I’ve written an ansible playbook for configuring namenode and datanode seperately. Before running the playbook, there are some steps to be followed. Without these, the playbook won’t run.

Configuring NameNode :

For running namenode.yml, follow the below steps :

1. Download JDK from the url and place it in the location : /root/jdk-8u171-linux-x64.rpm
2. Download Hadoop from the url and place it in the location : /root/hadoop-1.2.1–1.x86_64.rpm
3. Download or copy content of hdfs-site.xml file from the url and place it in the location : /root/hdfs-site.xml
4. Download or copy content of core-site.xml file from the url and place it in the location : /root/core-site.xml
5. Place the Inventory file in the location : /root/inventory.txt and content of inventory.txt is as follows (Replace the IP’s, usernames and passwords):

[namenode]
192.168.43.71 ansible_user=root ansible_ssh_pass=122020
	
[datanode]
192.168.43.57 ansible_user=root ansible_ssh_pass=122020

6. My configuration file of ansible is as follows (/etc/ansible/ansible.cfg) :

[defaults]
inventory = /root/inventory.txt
host_key_checking = false
command_warnings = false
deprecation_warnings = false

7. Check if ansible is able to connect to target nodes with the command :

ansible all -m ping

Output for me :

Finally, run the ansible playbook namenode.yml (First, copy the playbook in any directory):

ansible-playbook namenode.yml

You will be prompted to enter the directory for configuring your target node as namenode!

Running the playbook :

Configuring DataNode :

For running datanode.yml, follow the below steps :

1. Download JDK from the url and place it in the location : /root/jdk-8u171-linux-x64.rpm
2. Download Hadoop from the url and place it in the location : /root/hadoop-1.2.1–1.x86_64.rpm
3. Download or copy content of hdfs-site.xml file from the url and place it in the location : /root/hdfs-site.xml
4. Download or copy content of core-site.xml file from the url and place it in the location : /root/core-site.xml
5. Place the Inventory file in the location : /root/inventory.txt and content of inventory.txt is as follows (Replace the IP’s, usernames and passwords):

[namenode]
192.168.43.71 ansible_user=root ansible_ssh_pass=122020
	
[datanode]
192.168.43.57 ansible_user=root ansible_ssh_pass=122020

6. My configuration file of ansible is as follows (/etc/ansible/ansible.cfg) :

[defaults]
inventory = /root/inventory.txt
host_key_checking = false
command_warnings = false
deprecation_warnings = false

7. Check if ansible is able to connect to target nodes with the command :

ansible all -m ping

Finally, run the ansible playbook datanode.yml (First, copy the playbook in any directory):

ansible-playbook datanode.yml

You will be prompted to enter the directory for configuring your target node as datanode!

Running the playbook :

As I have shown in the above video, in the WebUI of Hadoop, the Hadoop Cluster is configured with ansible playbook and working good !! Try yourself !!

Thank You