Performance improvement techniques for Spark SQL DataFrames
Sorry for being very abstract. I would like to ask this question, as many of you came across situation like me. We need to build a machine learning based application. As part of our application we...
View ArticleReply To: Performance improvement techniques for Spark SQL DataFrames
It’s hard to give direct solution for your problem. First you need figure what’s the performance bottleneck of you current problem. What kind of join do you use (is it one large table join with one...
View ArticleCreate ORC table in hive cause "java.lang.OutOfMemoryError: Java heap...
Hi, I’m going through the tutorial “Hello World – Getting Started with HDP“. I’ve succeeded loading the csv files (step 1.2) and loaded the data into a Hive table correctly (step 2.3), but i’m having...
View ArticleReply To: problem in access phoenix and Hbase remotely
Hi, i have the same exact problem!
View Articleuser [admin] does not have [READ] privilege on HDFS
Hi, I tried to load data into table from file in the HDFS, like it is written in the tutorial – on SandBox 2.3.2 on Azure. But I got the following execption:...
View ArticleReply To: Error creating ORC trucks tutorial table
Hi Carlos, Any advances made since last message? Have you succeeded to pass all exercises? What was the command that you run when you mentioned “compress command to 1024”? Thanks, Shaul
View ArticleReply To: Welcome to the new Hortonworks Sandbox!
The 2.3.2 version of Sandbox exists critical configuration issue. Some basic hive queries can’t run on it. Does HDP has any plan to split the Sandbox to 2 boxes? Leave enough memory to fundamental part?
View Articleimage identity_1 broken?
I am trying to install another instance of cloudbreak on a laptop. While doing that I ran into the following error: Creating cbreak_identity_1… Cannot start container...
View ArticleReply To: HCAT returns NPE when hive.security.authorization.manager not set...
Thanks Thejas but you just copied and pasted the link I gave in my post. I wanted a workaround as HCat does not work with this authorizer which is problematic…
View ArticleReply To: Hive Import Error: SemanticException Error 10028
Can you please specify the reason why it is executed as hadoop jar instead of JAVA jar
View ArticleReply To: Issue on HDPCD Practice Test with sqoop
Try this <span style=”font: 12.8px/normal arial, sans-serif; color: #222222; text-transform: none; text-indent: 0px; letter-spacing: normal; word-spacing: 0px; float: none; display: inline...
View ArticleReply To: Issue on HDPCD Practice Test with sqoop
Try this sqoop export –connect jdbc:mysql://namenode:3306/flightinfo –table weather –export-dir user/hortonworks/weather –username root –password hadoop make sure you have the files copied from your...
View ArticleHBase Batch Insertion
Hi everyone, I’m currently trying to implement batch puts into HBase (ver 1.0.0 but it can be changed if needed) from Spark Streaming. I’ve read in the HBase: The Definitive Guide about batch...
View ArticleReply To: Falcon Tutorial / Entity Scheduling Error / FalconWebException / E1004
This will be fixed in the next sandbox update. Please see the following document on the oozie site configuration to be done for Falcon in sandbox 2.3.2 Falcon oozie configuration
View ArticleReply To: Setting up multiple environments in a single Hadoop cluster
we can create diff directories on HDFS and put the restrictions based on your requirement but (Ambari,Hue) URLs are the same for dev/test/prod environments. we can’t give diff urls as all are having...
View ArticleReply To: Amabari Hive view error (broken pipe)
Try restarting hive-server2 <p class=”p1″><span class=”s1″>service hive-server2 stop</span></p> <p class=”p1″></p> <p class=”p1″><span...
View ArticleReply To: Data node installation problem Ambari server 1.7
Is this fresh installation or you uninstall and trying to install again?
View ArticleReply To: user [admin] does not have [READ] privilege on HDFS
run below command and try again. hadoop fs -chmod -R 777 /hdp/
View ArticleReply To: Ambari HDFS File View – Sandbox VM
Hi Daniel, Can you check what version of the Sandbox you are using? In the last couple of updates we have included a sandbox-version command you can run which should tell you what version you are...
View ArticleReply To: Error while moving Ambari server 1.7
Can you try changing the log level to DEBUG to see if you see any more relevant log messages that can help you/us troubleshoot? To do so, you can edit /etc/ambari-server/conf/log4j.properties and...
View Article