I have a cluster with YARN log aggregation enabled. There is a compression flag enabled for the aggregated logs.
When I try to view the log files using hdfs dfs -cat it doesn’t work. Even after getting the files locally I cannot read or decompress (using gzip -d).
My question is: Is there any way I can view the log files using the -cat output?
I tried the following command it didn’t work either:
hdfs dfs -text /path/to/aggregation/logs/directory/file
I understand I can use the yarn logs command. I am trying to check the logs from the cli without requiring to use yarn logs command on the compression enabled yarn log files.
Any help would be appreciated.