Reply To: Kafka and Spark Streaming – NoSuchMethodError:...
For example after downloading spark-streaming-kafka_2.10-1.3.1.2.3.2.0-2650.jar to my local directory and placing cluster-log4j.properties , content below, in the same directory:...
View ArticleReply To: HDP 2.0 – SparkR (Spark 1.4)
When is the technical preview to be posted and would this integration be slated for the next 2.X release? Trying to get a timeframe so that I know if I should just go ahead and install it into 2.3...
View ArticleReply To: Kafka and Spark Streaming – NoSuchMethodError:...
Note that I verified I hit this same error with spark-streaming-kafka_2.10-1.3.1.2.3.0.0-2557.jar which is the same build number as the Sandbox. I also get the following error which I believe to be...
View ArticleError while importing Sandbox in Oracle Virtual Box 4.3
I am getting error while importing Sandbox in Oracle Virtual Box 4.3. My host OS – Windows Vista, 32 bit with 3GB Physical RAM. Is 4 GB the minimum RAM requirement ? I am also getting warning on...
View ArticleHow to retrieve metadata information from hadoop?
Past few weeks I’m studying Hadoop. My task is to complete Data Discover and File Masking for Big Data. For File Masking, I found the file path where the tables are stored. For Discovering the...
View ArticleReply To: How to retrieve metadata information from hadoop?
The metadata information you seem to want is stored by HCatalog. It is possible to talk to HCatalog http://hortonworks.com/hadoop/hcatalog/ That page mentions It also presents a REST interface to...
View ArticleUnable to install Oracle JDBC file on Sandbox 2.3.0
Hi , I have downloaded the ORACLE JDBC File (ojdbc6.jar). The Hortonwork documentation says I should copy this file under /usr/lib/sqoop/lib. when Installing for SQOOP. I’m not able to find this...
View ArticleLocal Spark talking to remote HDFS?
I have a file in HDFS inside my HortonWorks HDP 2.3_1 VirtualBox VM. If I go into the guest spark-shell and refer to the file thus, it works fine val words=sc.textFile(“hdfs:///tmp/people.txt”)...
View ArticleReply To: Unable to install Oracle JDBC file on Sandbox 2.3.0
From HDP 2.2, the SQOOP_HOME is /usr/hdp/current/sqoop-client. Please copy the file to that directory.
View ArticleReply To: HDP for Azure – Sandbox root login
Hi Suresh, Once you log in via ssh as the user you specified in the blade, you should be able to then execute sudo su. You will be prompted with your password again, this is the same one you specified...
View ArticleUsing the phoenix RDD/dataframe inside Spark
My team and I have been trying to use phoenix inside of the spark shell. Whenever we run the “import org.apache.phoenix.spark._” we recieve the error “object spark is not a member of package...
View ArticleNot able to get Storm Starter Kit
Hi, I am not able to connect to any website on internet from Sandbox. I have to connect to “public-repo-1.hortonworks.com” as part of the Apache Storm tutorial...
View ArticleReply To: HDFS File Permissions – Bug?
Hello Patrick, Before Ranger was integrated with Sandbox, dfs.permissions in Sandbox was set to false. The reason was to allow Hue and some other use cases to create databases and tables. After Ranger...
View ArticleReply To: Services failing to start during Kerberos setup from Ambari
Hi Venkata, do you have a link or anything that you followed to setup kerberos using AD authentication for your HDP 2.2.4 cluster? Thank you
View ArticleReply To: Does Ambari now NEED KDC Admin rights?
Hi Alex, I think I am ina similar situation. For me, its ok to get a domain admin to login for kadmin but it fails during “Create keytabs” and creates another user for an already existing hadoop user....
View ArticleReply To: Does Ambari now NEED KDC Admin rights?
I believe that this problem has been removed in Ambari 2.1 Hortonworks seem to have put in some effort into fixing this. Thanks!
View ArticleUpgraded HDP 2.2 to 2.3, get this error Invalid maximum heap size: -Xmx512mm
Hello, Please note it looks like hive-env.sh file is incorrectly reading the hive heap size. The Hive Client heapsize looks to be passing an “m” when called by HADOOP_HEAPSIZE. Not sure if anyone else...
View ArticleReply To: Does Ambari now NEED KDC Admin rights?
Hi Alex, thanks. Is there documentation you are aware of around krb configuration with AD for HDP2.2 that you cna maybe point me towards? I am lost on what will need to be done to use AD...
View Article