Quantcast
Channel: Hortonworks » All Replies
Viewing all articles
Browse latest Browse all 3435

how to suppress mapper output files if the output file does not have any data?

$
0
0

Apologies if i haven’t put the question properly.

I have a combined file format, which returns file name as key and filecontent as value.

I customized Mapper class’s run method and runs the map method if the file meets specific conditions only.

lets say, it calls map method if the file content is greater than 200 kb . If 200 files are sent as input, 200 mappers will commence, and if only 100 files met the criteria and ran map method, we will still have 200 output files in output folder.

Is there a way, to make sure to ensure no output file should be there if the file does not have any data.? or other way around, to create files only if the data is there for files?


Viewing all articles
Browse latest Browse all 3435

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>