How to give custom name to Sqoop output files
When I import data to hive using sqoop bydefault it creates file name as part-m-0000, part-m-0001 etc on HDFS. Is it possible to rename these files? If i wish to give some meaningfull name like...
View ArticleIncremental loads in Sqoop
Hi All, I have a table in Teradata which is loaded with new data on daily basis. I need to import this data to Hive. I’m trying to use Sqoop but how should I do incremental load using Sqoop? I checked...
View ArticleReply To: How to give custom name to Sqoop output files
Hi Chhava, In your sqoop export/import command use the option -D mapreduce.output.basename=<PrefixNameOfTheFile> basically the ‘part’ is the base name and part-m refers to Mapper. This would...
View ArticleReply To: scheduled jobs are failing in sqoop
Hi Ankush, The messages states that the AS400 JDBC driver is not available for sqoop to use. Now the questions is where and from where you are running this scheduled job. If you are running from...
View ArticleReply To: can't establish connection to 127.0.0.1:8000
type “ip addr show” in vm . then whatever VNET ip it shows .. type that ip in your browser
View ArticleReply To: can't establish connection to 127.0.0.1:8000
thanks for the reply Mr.Nachiket …where i should type “ip addr show ” in vm?
View ArticleReply To: can't establish connection to 127.0.0.1:8000
sir, i got ip address as 192.168.56.101 in vm..and i typed it on the browser i got same result as web page is not available..Then i pinged the host ip address it says destination host unreacheable…is...
View ArticleApache Storm algorithms
hi, Im a student and web developer. And i have a work for school about drift information. I installed hortonworks in a virtual machine. and I have the following questions: 1- I need to use Twitter to...
View ArticleReply To: How to give custom name to Sqoop output files
Hi Mahesh, Thanks for your reply Actually after running the query I’m getting error as /user/hduser/testing . My table name is “testing” it is going to default user directory instead of...
View ArticleMemory configuration for 1NN (16GB RAM ) and 4 DN (8GB RAM)
I have set up HDP cluster using Ambari 1.7. I am able to process 50GB of data (2750 tasks) on cluster. But the problem come when i run another script to proces 50 GB the total tasks comes to 4300 ....
View ArticleReply To: How to configure multi node cluster
Hi Supported OS is ubuntu 12.04 for multinode cluster using Ambari1.7 and HDP2.2 Follow the steps below echo “os Check” cat /etc/issue echo “program check” whereis rpm whereis scp whereis curl whereis...
View ArticleReply To: Memory configuration for 1NN (16GB RAM ) and 4 DN (8GB RAM)
Hi Vikash, You must work out how many containers can run on your cluster. This is done by taking the MINIMUM of the following calculations: 2 x Number of Cores 1.8 * Number of disks (configured for...
View ArticleHive Server fails after some time
Hi, I am trying to deploy a hadoop cluster using Ambari Blueprint. All the services are up and running except one i.e. Hive Server 2. I tried to look into the logs(/var/log/hive/hiveserver2.log) and...
View ArticleReply To: Apache Storm algorithms
Hi Pedro, You should be able to use Storm but you could also look at the blog post below: Have you looked here:...
View ArticleReply To: Pig(Tez) job won't run every time from Hue's Job Designer
Hi Aare, This looks like a connectivity issue in your cluster. How many nodes are you running and are they all on the same subnet or VLAN? Thanks Dave
View ArticleA black hole 12 billion times bigger than the sun exists
http://www.reddit.com/r/fullytrusted/comments/2xs325/ the auto money app software that is being given by Daniel Rosenberg to select invitees. I used his app for about a week to trade in binary options.
View ArticleReply To: Pig(Tez) job won't run every time from Hue's Job Designer
Hi Dave, I have 4 node test cluster set up in one subnet. Thanks Aare
View ArticleProblems with pig
Hi Guys, I have some problems when running Pig(HDP 2.2). Im doing the tutorial at:http://hortonworks.com/hadoop-tutorial/hello-world-an-introduction-to-hadoop-hcatalog-hive-and-pig/ I have tryed...
View ArticleReply To: Memory configuration for 1NN (16GB RAM ) and 4 DN (8GB RAM)
Hi Dave, Thanks for support. I have tried these configurations from the link you have given. I am using the below configuration: mapreduce.map.java.opts=3072 mapreduce.reduce .java.opts=6144...
View Article