java – Apache Hadoop setXIncludeAware UnsupportedOperationException

I tried to run Apache Hadoop 1.21, but I got this exception:

Failed to set setXIncludeAware(true) for parser org.apache.xerces.jaxp.DocumentBuilderFactoryImpl
org.apache.xerces.jaxp.DocumentBuilderFactoryImpl@2662e5cf:java.lang.UnsupportedOperationException

Full stack trace:

13/10/17 17:22:52 ERROR conf.Configuration: Failed to set setXIncludeAware(true) for parser org.apache.xerces.jaxp.DocumentBuilderFactoryImpl@2662e5cf:java.lang.UnsupportedOperationException:  setXIncludeAware is not supported on this JAXP implementation or earlier: class org.apache.xerces.jaxp.DocumentBuilderFactoryImpl
java.lang.UnsupportedOperationException:  setXIncludeAware is not supported on this JAXP implementation or earlier: class org.apache.xerces.jaxp.DocumentBuilderFactoryImpl
    at javax.xml.parsers.DocumentBuilderFactory.setXIncludeAware(DocumentBuilderFactory.java:589)
    at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1131)
    at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1107)
    at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1053)
    at org.apache.hadoop.conf.Configuration.get(Configuration.java:460)
    at org.apache.hadoop.fs.FileSystem.getDefaultUri(FileSystem.java:132)
    at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:124)
    at main.JobExecutor.executeModelCreation(JobExecutor.java:223)
    at main.JobExecutor.main(JobExecutor.java:256)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:160)

I have been looking for solutions for nearly three days, and I found several websites (such as Hadoop "failed to set xincludeaware (true) for parser" error and how to resolve it). It is suggested to add Xerces and xalan to Maven dependencies There are other websites, and the recommendation is almost the opposite - remove all Xerces references from the classpath However, no suggested solution is to work:（

I also tried to:

System.setProperty("javax.xml.parsers.DocumentBuilderFactory","com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderFactoryImpl");

To my code, but it doesn't help

I am using Apache Hadoop 1.21, JDK 1.7 0-17.

Solution

In my case, it is an outdated Maven component that relies on Xerces impl 2.4 0 (since mockrunner 1.0.3 is used in the test phase) Add the following managed dependencies to the < dependencymanagemen > section help

<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>xerces</groupId>
            <artifactId>xercesImpl</artifactId>
            <version>2.11.0</version>
        </dependency>
    </dependencies>
</dependencyManagement>

Hadoop version is 2.3 0. Here is another article describes a similar situation