Quantcast
Channel: Hortonworks » All Replies
Viewing all articles
Browse latest Browse all 3435

GenericUDF fails when creating view

$
0
0

Hello,
I’m trying to create a GenericUDF. I’m starting from an example I found on the net.

Everything compiles just fine, but when I try to create the view using the jar, the create fails, or at least doesn’t succeed. I’ve tried to find logs where there could be a message, but so far I haven’t found the correct log files.

Any ideas or suggestions on why this may be happening?

Thanks,

Harlan…

Output:

0: jdbc:hive2://> add jar /usr/lib/hive/lib/hive-contrib.jar;
15/10/29 23:56:27 INFO operation.Operation: Putting temp output to file /tmp/user/f3010515-58bd-4b89-8ed0-f9fe2453b30f5924808163592247027.pipeout
Added /usr/lib/hive/lib/hive-contrib.jar to class path
15/10/29 23:56:27 INFO SessionState: Added /usr/lib/hive/lib/hive-contrib.jar to class path
Added resource: /usr/lib/hive/lib/hive-contrib.jar
15/10/29 23:56:27 INFO SessionState: Added resource: /usr/lib/hive/lib/hive-contrib.jar
No rows affected (0.02 seconds)
0: jdbc:hive2://> add jar hdfs:///cust/dev/DataUDF.jar;
0: jdbc:hive2://>
0: jdbc:hive2://> DROP FUNCTION IF EXISTS scrub_dev;
0: jdbc:hive2://> DROP FUNCTION IF EXISTS scrubdev;
0: jdbc:hive2://> CREATE FUNCTION scrubdev as ‘com.company.group.data.cust.DataUDF’ using jar ‘hdfs:///cust/dev/DataUDF.jar’;
0: jdbc:hive2://>
0: jdbc:hive2://> DROP VIEW IF EXISTS customer_info_dev;
0: jdbc:hive2://>
0: jdbc:hive2://> create view customer_info_dev (prtcptg_ecn_no, phone_no)
. . . . . . . . > as select cust.scrubdev( prtcptg_ecn_no, phone_no )
. . . . . . . . > as prtcptg_ecn_no, phone_no
. . . . . . . . > from cust.customer_info
. . . . . . . . > ;

JAR code:

package com.company.group.data.cust;

import org.apache.hadoop.hive.ql.exec.UDFArgumentException;
import org.apache.hadoop.hive.ql.metadata.HiveException;
import org.apache.hadoop.hive.ql.udf.generic.GenericUDF;
import org.apache.hadoop.hive.serde2.objectinspector.ListObjectInspector;
import org.apache.hadoop.hive.serde2.objectinspector.ObjectInspector;
import org.apache.hadoop.hive.serde2.objectinspector.ObjectInspector.Category;
import java.util.ArrayList;

public class DataUDF extends GenericUDF {
ListObjectInspector listInputObjectInspector;

// the return variable. Java Object[] become hive struct<>. Java ArrayList<> become hive array<>.
// The return variable only holds the values that are in the struct<>. The field names
// are defined in the ObjectInspector that is returned by the initialize() method.
private ArrayList ret;

public ObjectInspector initialize(ObjectInspector[] args)
throws UDFArgumentException {
assert (args.length >= 1); // This UDF accepts one or more arguments
// The first argument is a list
assert (args[0].getCategory() == Category.LIST);

listInputObjectInspector = (ListObjectInspector) args[0];

/*
* Here comes the real usage for Object Inspectors : we know that our
* return type is equal to the type of the elements of the input array.
* We don’t need to know in details what this type is, the
* ListObjectInspector already has it
*/
return listInputObjectInspector.getListElementObjectInspector();
}

public Object evaluate(DeferredObject[] args) throws HiveException {
// Empty the return array (re-used between calls)
ret.clear();

if (args.length < 1)
return null;

// If passed a null, return a null
if (args[0].get() == null)
return null;

// HOW TO RETURN THE OUTPUT?
// A struct<> is stored as an Object[], with the elements being ,,…
// See GenericUDFNamedStruct
// The object inspector that we set up below and return at the end of initialize() takes care of the names,
// so the Object[] only holds the values.
// A java ArrayList is converted to a hive array<>, so the output is an ArrayList
ret = new ArrayList();

for (int ix = 0; ix < args.length; ix++) {
// The struct<> we’re returning is stored as an Object[] of length 1
// (it has 1 field)
Object[] e;
//e = new Object[args.length];
e = new Object[1];
e[0] = args[ix].get();

ret.add(e);
}

return ret;
}

@Override
public String getDisplayString(String[] args) {
return “Here, write a nice description”;
}
}


Viewing all articles
Browse latest Browse all 3435

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>