HDFS commands are taking around 12 seconds on hadoop 2.7.1 release, while on the same hardware if I install hadoop 2.4 release, hdfs commands are taking less than 3 seconds.
e.g. <span style=”line-height: 1.5;”>hdfs –loglevel DEBUG dfs -ls /data/collector command took around 12 seconds where majority of the time taken in the following log:</span>
The detailed call stack is available at following link:
https://docs.google.com/document/d/1ao64gsk4pUDKeerzXAYvRK5nbc_6S6mC6DL4BVtiJz8/edit?usp=sharing
<p class=”p1″><span class=”s1″>15/11/06 05:49:44 DEBUG hdfs.BlockReaderLocal: dfs.domain.socket.path = </span></p>
<p class=”p1″><span class=”s1″>15/11/06 05:49:44 DEBUG retry.RetryUtils: multipleLinearRandomRetry = null</span></p>
<p class=”p1″><span class=”s1″>15/11/06 05:49:44 DEBUG ipc.Server: rpcKind=RPC_PROTOCOL_BUFFER, rpcRequestWrapperClass=class org.apache.hadoop.ipc.ProtobufRpcEngine$RpcRequestWrapper, rpcInvoker=org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker@1d296da</span></p>
<p class=”p1″><span class=”s1″>15/11/06 05:49:44 DEBUG ipc.Client: getting client out of cache: org.apache.hadoop.ipc.Client@139982de</span></p>
<p class=”p1″><span class=”s1″>15/11/06 05:49:55 DEBUG unix.DomainSocketWatcher: org.apache.hadoop.net.unix.DomainSocketWatcher$2@5246ca7d: starting with interruptCheckPeriodMs = 60000</span></p>
<p class=”p1″><span class=”s1″>15/11/06 05:49:55 DEBUG util.PerformanceAdvisory: Both short-circuit local reads and UNIX domain socket are disabled.</span></p>
<span style=”line-height: 1.5;”>15/11/06 05:49:55 DEBUG sasl.DataTransferSaslUtil: DataTransferProtocol not using SaslPropertiesResolver, no QOP found in configuration for dfs.data.transfer.protection</span><span style=”line-height: 1.5;”> </span>