I’m using HDP 2.3. HDFS encription disabled.
↧
Reply To: Hive query creates temp folders
↧
What are techniques for bring huge data form client to our environment
I have the doubt,that is “What are techniques for bring huge data form client to our environment, in production environment .
↧
↧
How to put huge data in Hdfs in Production
I have the doubt that , HOW TO PUT HUGE FILES INTO HDFS IN PRODUCTION.
it is not possible to use put command always “hadoop fs -put ‘/inpu_path /out_path’ “then how to put thous files in hdfs
↧
CloudBreak Networking Azure hosts file not correct
I have created a cluster using the CloudBreak and that all works and I can log into the servers just fine. The problem that I am having is the network setups on the host os and the docker containers seems to not be setup right. The host os and the containers /etc/hosts file like like this
cloudbreak# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
This causes a problem for the hadoop cluster because all the nodes then don’t know how to communicate. If I change the hosts files to contain the other nodes then things start to work. However this does not seems like something I should have to do. This will also be a problem when trying make new clusters, as I will have to go in and make changes, and the auto scaling will not work if i have to change the host file on every host and docker container.
Any help would helpful, thanks.
↧
HDP 2.3 on Centos7 clashing with MariaDB
The distribution is trying to install mysql community server from a non existing path in the repository. When attempting to help it with loading the rpm and creating the path structure, it clashes with MariaDB which apparently replaced mysql in Centos 7.. any direction on how to proceed? Is overriding Ambari process a viable option? How?
↧
↧
Issue with Ambari start
Getting the error
java.lang.RuntimeException: org.postgresql.util.PSQLException: FATAL: Ident authentication failed for user “ambari”
↧
Ambari 2.0 Error getting timeline metrics. Can not connect to collector socket
I’m running HDP cluster on RedHat Linux 6.5.
The Ambari version is 2.0.
ambari-server:/root # rpm -qa | grep ambari
ambari-server-2.0.1-45.noarch
ambari-log4j-2.0.1.45-1.noarch
datanode1:/root # rpm -qa | grep ambari
ambari-metrics-hadoop-sink-2.0.1-45.x86_64
ambari-log4j-2.0.1.45-1.noarch
ambari-agent-2.0.1-45.x86_64
ambari-metrics-monitor-2.0.1-45.x86_64
All the services are running on the HDP 2.2 cluster including ambari-metrics.
I’m seeing below error in ambari server log:
22 Sep 2015 20:38:18,349 ERROR [qtp-client-12552] MetricsPropertyProvider:201 – Error getting timeline metrics. Can not connect to collector, socket error.
22 Sep 2015 20:38:36,356 ERROR [qtp-client-12217] MetricsPropertyProvider:201 – Error getting timeline metrics. Can not connect to collector, socket error.
22 Sep 2015 20:38:46,367 ERROR [qtp-client-12217] MetricsPropertyProvider:201 – Error getting timeline metrics. Can not connect to collector, socket error.
22 Sep 2015 20:39:04,337 ERROR [qtp-client-12636] MetricsPropertyProvider:201 – Error getting timeline metrics. Can not connect to collector, socket error.
Someone please help me…
Thanks
↧
Reply To: HIVE – JSON-Serde problems
Yes, I have exactly the same problem (second one).
Did you manage to solve it?
When you do SELECT string field from table everything is ok (f3 in your example) but trying to select integer (f1 in your example) returns Error: java.lang.ClassCastException: java.lang.String cannot be cast to java.lang.Integer
Strange…
↧
Reply To: CloudBreak Networking Azure hosts file not correct
Hi Brian,
Did you try to submit a job to the cluster, or why do you think that it is “not working”? You won’t find anything in the /etc/hosts files, that’s just simply not how things are working in a Cloudbreak cluster. Cloudbreak is using Consul for service discovery and DNS, you can find more about it in the documentation: https://github.com/sequenceiq/cloudbreak#consul.
Marton
↧
↧
Unable to Locate csv-serde JAR
I am trying to import a file that has the fields quoted. I found out about the csv-serde JAR and have attempted to use this to minimal success.
I downloaded the JAR from:
https://github.com/ogrodnek/csv-serde
I added to my HDFS system in the /tmp directory.
When I try to use the View to HIVE from within Ambari I set the following:
ADD JAR /tmp/csv-serde-1.1.2-0.11.0-all.jar;
This throws an error that the file does not exist.
I then try from terminal calling hive:
hive> ADD JAR /tmp/csv-serde-1.1.2-0.11.0-all.jar;
This succeeds. I then create my EXTERNAL TABLE and then select some fields to make sure it is good from within the CLI hive environment and all works.
I then go back to the Hive View from within Ambari and when I run any type of operation on the table I get:
H110 Unable to submit statement. Error while compiling statement: FAILED: RuntimeException MetaException(message:java.lang.ClassNotFoundException Class com.bizo.hive.serde.csv.CSVSerde not found) [ERROR_STATUS]
When I try to query this from MS SQL via the ODBC drive I also get the same error.
I have looked at multiple posts on this forum about adding so and so .jar and this and that to:
/usr/lib/hive
That directory does not exit on my cluster either in the base OS level or in the HDFS file system. Where exactly, and how, should I put the .jar file such that I can query my tables outside of the hive CLI?
↧
Oozie problem with multiple servers-Same job run by all servers
I have Amabri – <span style=”font-family: ‘Helvetica Neue’, Helvetica, Arial, sans-serif; font-size: 14px; line-height: 20px;”>2.1.0 with HDP 2.3 (Oozie 4.2.0).</span>
I have installed oozie server on 3 linux machines and oozie client on 6 machines.
I am using MySQL as oozie database.
Now when I make a coordinator job and run it – then there is sometimes 3 instances of the Job (workflow job specified in workflow.xml) running simultaneously. This is happening many times.
is this something because I have 3 oozie servers running side by side? But this should have been managed internally by oozie right? OR I am missing some configuration?
below is my coordinator.xml
<coordinator-app name=”TEST-COORDINATOR” frequency=”5″ start=”2015-09-23T12:53Z” end=”2015-10-23T00:00Z” timezone=”US/Eastern” xmlns=”uri:oozie:coordinator:0.1″>
<controls>
<concurrency>1</concurrency>
</controls>
<action>
<workflow>
<app-path>/path/to/job.xml</app-path>
</workflow>
</action>
</coordinator-app>
↧
Reply To: Flume spoolDir
Modified Code I used
agent.sources = r1
agent.sinks = k1
agent.channels = c1
# Bind the source and sink to the channel
agent.sources.r1.channels = c1
agent.sinks.k1.channel = c1
# Describe/configure the source
agent.sources.r1.type = spooldir
agent.sources.r1.spoolDir = /home/splunk/data
#agent.sinks.k1.type = logger
# Describe the sink
agent.sinks.k1.type = hdfs
agent.sinks.k1.hdfs.path = /incoming/hunk/ecm
agent.sinks.k1.hdfs.filePrefix = %{basename}-
agent.sinks.k1.hdfs.round = true
agent.sinks.k1.hdfs.roundValue = 10
agent.sinks.k1.hdfs.roundUnit = minute
↧
Reply To: Flume spoolDir
Even modified code also not working. Basically I am looking for code to copy files to HDFS as soon as files arrived to log directory..
↧
↧
Could not log into ambari using user id "admin" and password "admin"
Can some one help? I am practicing using the sandbox but cannot log into ambari with url http://127.0.0.1:8080, and id “admin”/password “admin”. I did run “ambari-server start” and ambari-agent start”. The error I got is “http://127.0.0.1:8080 requires username and password. The server says XDB”
↧
Reply To: Could not log into ambari using user id "admin" and password "admin"
Hi Bin,
What sandbox version are you using? It looks like you tried to install some other software in the vm and now that may have taken over the 8080 port. If so, please shutdown the other service you installed.
Regards,
Robert
↧
Reply To: Cannot login to Ambari
Hi @Kesav and other gurus: I also installed Oracle in my PC. How can change the port number? Which number I shall use for Oracle?
↧
Reply To: Could not log into ambari using user id "admin" and password "admin"
Hi Bob: Thanks for replying me. I am using the Oracle VirtualBox “VirtualBox-5.0.4-102546-Win”. I searched for the same issue. It seems that it was caused by that the Oracle is installed and ran on my PC. So I have to change the oracle port. Do you know how to change it?
↧
↧
Reply To: Could not log into ambari using user id "admin" and password "admin"
After I stop the Oracle TNS listner, I can get into ambari. 
↧
Reply To: Problem installing HDP2.2
Are you performing a fresh install or an upgrade?
↧
Reply To: How to find super user name?
It’s not really a cut and dry answer. If we are talking about Ambari, from the web interface the super user will be admin (that’s if it wasn’t linked to LDAP). In regards to from the Linux command line, you can do anything from the root account. But many of the services will have local accounts (such as HDFS).
↧