Java – OutputStream outofmemoryerror when sending http

I tried to publish a large video / image file from the local file system to the HTTP path, but after a period of time, I encountered an out of memory error

Here is the code

public boolean publishFile(URI publishTo,String localPath) throws Exception {
    InputStream istream = null;
    OutputStream ostream = null;
    boolean isPublishSuccess = false;

    URL url = makeURL(publishTo.getHost(),this.port,publishTo.getPath());
    HttpURLConnection conn = (HttpURLConnection) url.openConnection();


    if (conn != null) {

        try {

            conn.setDoOutput(true);
            conn.setDoInput(true);
            conn.setRequestMethod("PUT");
            istream = new FileInputStream(localPath);
            ostream = conn.getOutputStream();

            int n;
            byte[] buf = new byte[4096];
            while ((n = istream.read(buf,buf.length)) > 0) {
                ostream.write(buf,n); //<--- ERROR happens on this line.......???
            }

            int rc = conn.getResponseCode();

            if (rc == 201) {
                isPublishSuccess = true;
            }

        } catch (Exception ex) {
            log.error(ex);
        } finally {
            if (ostream != null) {
                ostream.close();
            }

            if (istream != null) {
                istream.close();
            }
        }
    }

    return isPublishSuccess;

}

Here is the error I'm receiving

Exception in thread "Thread-8773" java.lang.OutOfMemoryError: Java heap space
    at java.util.Arrays.copyOf(Arrays.java:2786)
    at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:94)
    at sun.net.www.http.PosterOutputStream.write(PosterOutputStream.java:61)
    at com.test.HTTPClient.publishFile(HTTPClient.java:110)
    at com.test.HttpFileTransport.put(HttpFileTransport.java:97)

Solution

Httpurlconnection is buffering data so that it can set the content length header (per HTTP spec)

As an alternative, use "chunked" transport if your target server supports it This will buffer a small amount of data at a time However, not all services support it (for example, Amazon S3) does not

Another (and better) option is to use Jakarta httpclient You can set "entity" in the request of the file, and the connection code will set the request header appropriately

Editor: NOS commented that OP can call httpurlconnection setFixedLengthStreamingMode(long length). I don't know this method; It was added in 1.5 and I haven't used this class since then

However, I still recommend using Jakarta httpclient because it reduces the amount of code that OP must maintain The code is a template, but there are still potential errors:

>OP correctly handles the loop copied between input and output Usually when I see an example, the poster does not check the returned buffer size correctly, or constantly reallocate the buffer Congratulations, but you must now make sure that your successor is very careful. > Exception handling is not very good Yes, Op remember to close the connection in the last block. Congratulations again Except that the close () call may throw an IOException, keep another one from executing Moreover, the entire method throws an exception, so the compiler will not help catch similar errors. > I calculate 31 lines of code to set and execute the response (excluding response code checking and URL calculation, but including try / catch / finally) Using httpclient, this will be in the range of dozens of locs

Even if the op writes this code perfectly and reconstructs it into a method similar to Jakarta commons IO, he / she should not do so This code has been written and tested by others I know it's a waste of my rewriting time and doubt it's a waste of OP's time