I have encountered a problem with what I choose to call a “shadow coordinator”.
Using Jenkins to deploy coordinators (this enables full version control in the code repository), we filter for status “running” and grep for coordinator name, like this:
[abru@hadoop-namenode ~]$ oozie jobs -oozie http://localhost:11000/oozie -jobtype coordinator -filter status=RUNNING | grep my-strange-coordinator
0000010-150304115410850-oozie-oozi-C my-strange-coordinatorRUNNING 1 DAY 2015-03-05 00:16 GMT 2015-03-06 00:16 GMT
[abru@hadoop-namenode ~]$
However, we started getting mysterious problems with data output, sometimes the partition path was already in existence, and the data didn’t make sense. Digging around, we found a “shadow coordinator”, previously deployed (this one, on feb 13th) running with the same name.
So, this “shadow coordinator” does not show up when listing all running coordinators, but only when filtering specifically by the name. Same name, different Job ID.
Anyone else come across this?
[abru@hadoop-namenode ~]$ oozie jobs -oozie http://localhost:11000/oozie -jobtype coordinator -filter name=my-strange-coordinator status=RUNNING
Job ID App Name Status Freq Unit Started Next Materialized
------------------------------------------------------------------------------------------------------------------------------------
0000010-150304115410850-oozie-oozi-C my-strange-coordinatorRUNNING 1 DAY 2015-03-05 00:16 GMT 2015-03-06 00:16 GMT
------------------------------------------------------------------------------------------------------------------------------------
0000066-150212142339118-oozie-oozi-C my-strange-coordinatorRUNNING 1 DAY 2015-02-13 00:16 GMT 2015-03-06 00:16 GMT
------------------------------------------------------------------------------------------------------------------------------------
[abru@hadoop-namenode ~]$