Sunday, September 13, 2015

Can not visit YARN resource manager Web URI at http://:8088


$ netstat -tunalp | grep LISTEN

tcp        0      0 0.0.0.0:50070           0.0.0.0:*               LISTEN      10532/java     
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      -              
tcp        0      0 0.0.0.0:50010           0.0.0.0:*               LISTEN      10694/java     
tcp        0      0 0.0.0.0:50075           0.0.0.0:*               LISTEN      10694/java     
tcp        0      0 0.0.0.0:10020           0.0.0.0:*               LISTEN      11498/java     
tcp        0      0 0.0.0.0:50020           0.0.0.0:*               LISTEN      10694/java     
tcp        0      0 0.0.0.0:50090           0.0.0.0:*               LISTEN      10886/java     
tcp        0      0 0.0.0.0:19888           0.0.0.0:*               LISTEN      11498/java     
tcp        0      0 0.0.0.0:10033           0.0.0.0:*               LISTEN      11498/java     
tcp        0      0 127.0.0.1:8020          0.0.0.0:*               LISTEN      10532/java     
tcp6       0      0 :::22                   :::*                    LISTEN      -              
tcp6       0      0 127.0.0.1:8088          :::*                    LISTEN      11027/java     
tcp6       0      0 :::13562                :::*                    LISTEN      11166/java     
tcp6       0      0 127.0.0.1:8030          :::*                    LISTEN      11027/java     
tcp6       0      0 127.0.0.1:8031          :::*                    LISTEN      11027/java     
tcp6       0      0 127.0.0.1:8032          :::*                    LISTEN      11027/java     
tcp6       0      0 127.0.0.1:8033          :::*                    LISTEN      11027/java     
tcp6       0      0 :::36580                :::*                    LISTEN      11166/java     
tcp6       0      0 :::8040                 :::*                    LISTEN      11166/java     
tcp6       0      0 :::8042                 :::*                    LISTEN      11166/java    

  

notice that port 8088 started at tcp6 instead of tcp.

Implementing following resolved issue.

modify  yarn-site.xml only on master node as follows. Do not modify the yarn-site.xml in slave nodes:


    yarn.nodemanager.aux-services
    mapreduce_shuffle


yarn.nodemanager.aux-services.mapreduce.shuffle.class
    org.apache.hadoop.mapred.ShuffleHandler

 
Specifying hostname causes the ports started as tcp6.

This is related to a bug. Also add following to $HADOOP_HOME/bin/yarn will force ports start at tcp

YARN_OPTS="$YARN_OPTS -Djava.net.preferIPv4Stack=true"

No comments:

Post a Comment