GC Overhead Limit

Hello,

I get random GC Overhead limit exceeded error when I use Hive (on Tez). It seems to happen when I get significant (but not massive) amount of data using HiverServer. I have to restart HiveServer when it happens.

Any idea where does this come from?

I would like to increase heap size for HiveServer, but I am not sure where to do that.

Thanks for your help.

—

2015-11-26 17:05:58,879 ERROR [HiveServer2-Handler-Pool: Thread-37]: thrift.ProcessFunction (ProcessFunction.java:process(41)) – Internal error processing FetchResults
java.lang.OutOfMemoryError: GC overhead limit exceeded
at java.lang.StringBuilder.toString(Unknown Source)
at java.math.BigInteger.toString(Unknown Source)
at java.math.BigInteger.toString(Unknown Source)
at java.math.BigDecimal.toPlainString(Unknown Source)
at org.apache.hadoop.hive.common.type.HiveDecimal.toString(HiveDecimal.java:102)
at org.apache.hive.service.cli.ColumnValue.stringValue(ColumnValue.java:142)
at org.apache.hive.service.cli.ColumnValue.toTColumnValue(ColumnValue.java:174)
at org.apache.hive.service.cli.RowBasedSet.addRow(RowBasedSet.java:60)
at org.apache.hive.service.cli.RowBasedSet.addRow(RowBasedSet.java:32)
at org.apache.hive.service.cli.operation.SQLOperation.prepareFromRow(SQLOperation.java:371)
at org.apache.hive.service.cli.operation.SQLOperation.decode(SQLOperation.java:363)
at org.apache.hive.service.cli.operation.SQLOperation.getNextRowSet(SQLOperation.java:347)
at org.apache.hive.service.cli.operation.OperationManager.getOperationNextRowSet(OperationManager.java:248)
at org.apache.hive.service.cli.session.HiveSessionImpl.fetchResults(HiveSessionImpl.java:654)
at org.apache.hive.service.cli.CLIService.fetchResults(CLIService.java:450)
at org.apache.hive.service.cli.thrift.ThriftCLIService.FetchResults(ThriftCLIService.java:587)
at org.apache.hive.service.cli.thrift.TCLIService$Processor$FetchResults.getResult(TCLIService.java:1553)
at org.apache.hive.service.cli.thrift.TCLIService$Processor$FetchResults.getResult(TCLIService.java:1538)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:206)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)