Greetings,
Has anyone successfully incrementally imported data from SQL Server using the lastmodified mode? I am attempting to (1) perform an initial import of data into HDFS from SQL Server using Sqoop and then (2) add new data to HDFS via an incremental import. The initial import of data works as expected. Incrementally loading data using the append mode also works as expected. The lastmodified mode fails and no messages exist to provide additional troubleshooting guidance.
The command I am running is:
sqoop import –connect ‘jdbc:sqlserver:blahblah;’ \
–table sqoop_dates \
–incremental “lastmodified” \
–check-column “date_id” \
–last-value “‘01012014′”
I have tried a few variations with the “last-value” value. Setting the verbose option does not actually print anymore information to the screen. Sqoop’s source code include numerous references to a log and I am not seeing those message anywhere. I’ve triple-checked my column names and data types. I’ve also tried varying the SQL Server data type from datetime to datetime2.
On the SQL Server side, a trace shows Sqoop connecting a performing a basic object check (e.g. select from table where 1=0) and obtaining a list of columns from SELECT COLUMN_NAME FROM INFORMATION_SCHEMA.COLUMNS. After that the connection is closed and the Sqoop process quits. There is no attempt made to get any actual data from the table nor any values from the check-column such as its max value.
Any tips or guidance would be much appreciated.
Thanks,
Jon
Software versions:
Sqoop 1.4.5.2.2.0.0-1084
Hadoop 2.6.0.2.2.0.0-1084
