How to install Hadoop and run WordCount program

1. Install Java

Hadoop depends on Java, and first requires the installation of the Java Development Kit (JDK). Take Ubuntu as an example:

sudo apt update
sudo apt install openjdk-8-jdk

After installation, set environment variables:

echo 'export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64' >> ~/.bashrc
echo 'export PATH=$PATH:$JAVA_HOME/bin' >> ~/.bashrc
source ~/.bashrc

2. Install Hadoop

Download Hadoop：

wget /hadoop/common/hadoop-3.3.6/hadoop-3.3.
tar -xzf hadoop-3.3.
mv hadoop-3.3.6 /opt/hadoop

Configure environment variables：

echo 'export HADOOP_HOME=/opt/hadoop' >> ~/.bashrc
echo 'export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin' >> ~/.bashrc
source ~/.bashrc

Configure Hadoop：：

<configuration>
    <property>
        <name></name>
        <value>hdfs://localhost:9000</value>
    </property>
</configuration>

：

xml

<configuration>
    <property>
        <name></name>
        <value>1</value>
    </property>
    <property>
        <name></name>
        <value>file:/opt/hadoop/data/namenode</value>
    </property>
    <property>
        <name></name>
        <value>file:/opt/hadoop/data/datanode</value>
    </property>
</configuration>

3. Start Hadoop

Format NameNode：

hdfs namenode -format

Start HDFS and YARN：

Verification Service：
Visit http://localhost:9870 to view the HDFS interface, http://localhost:8088 to view the YARN interface.

4. Run the WordCount program

Prepare to enter data：

echo "Hello Hadoop World" > 
hdfs dfs -mkdir -p /user/input
hdfs dfs -put  /user/input

Execute WordCount：

hadoop jar $HADOOP_HOME/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.3. wordcount /user/input /user/output

View the results：

hdfs dfs -cat /user/output/part-r-00000

This is the article about how to install Hadoop and run WordCount program. For more related Hadoop installation and WordCount program content, please search for my previous articles or continue browsing the related articles below. I hope everyone will support me in the future!