Java – get existing MapReduce jobs from the cluster (jobs can be run or completed)

Previously, I was using org apache. hadoop. mapred. Jobclient#getjob (org. Apache. Hadoop. Mapred. Jobid) get runningjob This call is made from the job completion callback method, but for me, there is a time problem. If the job has been completed, the above getjob () method cannot be found and returns null I can confirm whether the job has been completed from the cluster UI

Keeping runningjobs separate, is there a way to get a given org. Org apache. hadoop. mapreduce. Org. Of the jobid mapping job apache. hadoop. mapreduce. Job object, whether the job is currently running or completed?

I try to write the following code:

Cluster cluster = jobClient. getClusterHandle(); Job job = cluster. getJob(JobID.forName(jobId)); log. Info ("try to use ID {}, find {} on cluster {}, jobid.forname (jobid), job, cluster) to obtain the actual work;

I can see the correct jobid and the cluster object But cluster The getjob () method returns null, so the job itself is null

Is there anything I missed here?

Solution

The recent upgrade requires the MR history server to be enabled on my system This solves the problem I recently upgraded from Mr V1 to v2. During the upgrade, all completed jobs are now moved to the history server