The fastest way to write integer arrays to files in Java?

As the title says, I'm looking for the fastest way to write integer arrays to files Arrays vary in size and can actually contain anywhere from 2500 to 25 million interiors

Here is the code I currently use:

DataOutputStream writer = new DataOutputStream(new bufferedoutputstream(new FileOutputStream(filename)));

for (int d : data)
  writer.writeInt(d);

Since dataoutputstream has a method to write byte array, I have tried to convert int array into byte array, as shown below:

private static byte[] integersToBytes(int[] values) throws IOException {
    ByteArrayOutputStream baos = new ByteArrayOutputStream();
    DataOutputStream dos = new DataOutputStream(baos);
    for (int i = 0; i < values.length; ++i) {
        dos.writeInt(values[i]);
    }

    return baos.toByteArray();
}

Like this:

private static byte[] integersToBytes2(int[] src) {
    int srcLength = src.length;
    byte[] dst = new byte[srcLength << 2];

    for (int i = 0; i < srcLength; i++) {
        int x = src[i];
        int j = i << 2;
        dst[j++] = (byte) ((x >>> 0) & 0xff);
        dst[j++] = (byte) ((x >>> 8) & 0xff);
        dst[j++] = (byte) ((x >>> 16) & 0xff);
        dst[j++] = (byte) ((x >>> 24) & 0xff);
    }
    return dst;
}

Both seem to have increased slightly, accounting for about 5% I didn't test them strictly to confirm

Is there any technology that can speed up this file write operation, or is there a guide to Java IO write performance best practices?

Solution

I looked at three options:

>Using dataoutputstream; > Use objectoutputstream (int [] Yes for serializable objects); And > use filechannel

The result is

DataOutputStream wrote 1,000,000 ints in 3,159.716 ms
ObjectOutputStream wrote 1,000 ints in 295.602 ms
FileChannel wrote 1,000 ints in 110.094 ms

So NiO version is the fastest It also has the advantage of allowing editing, which means that you can easily change an int, and objectoutputstream will need to read the entire array, modify it and write it to a file

The code is as follows:

private static final int NUM_INTS = 1000000;

interface IntWriter {
  void write(int[] ints);
}

public static void main(String[] args) {
  int[] ints = new int[NUM_INTS];
  Random r = new Random();
  for (int i=0; i<NUM_INTS; i++) {
    ints[i] = r.nextInt();
  }
  time("DataOutputStream",new IntWriter() {
    public void write(int[] ints) {
      storeDO(ints);
    }
  },ints);
  time("ObjectOutputStream",new IntWriter() {
    public void write(int[] ints) {
      storeOO(ints);
    }
  },ints);
  time("FileChannel",new IntWriter() {
    public void write(int[] ints) {
      storeFC(ints);
    }
  },ints);
}

private static void time(String name,IntWriter writer,int[] ints) {
  long start = System.nanoTime();
  writer.write(ints);
  long end = System.nanoTime();
  double ms = (end - start) / 1000000d;
  System.out.printf("%s wrote %,d ints in %,.3f ms%n",name,ints.length,ms);
}

private static void storeOO(int[] ints) {
  ObjectOutputStream out = null;
  try {
    out = new ObjectOutputStream(new FileOutputStream("object.out"));
    out.writeObject(ints);
  } catch (IOException e) {
    throw new RuntimeException(e);
  } finally {
    safeClose(out);
  }
}

private static void storeDO(int[] ints) {
  DataOutputStream out = null;
  try {
    out = new DataOutputStream(new FileOutputStream("data.out"));
    for (int anInt : ints) {
      out.write(anInt);
    }
  } catch (IOException e) {
    throw new RuntimeException(e);
  } finally {
    safeClose(out);
  }
}

private static void storeFC(int[] ints) {
  FileOutputStream out = null;
  try {
    out = new FileOutputStream("fc.out");
    FileChannel file = out.getChannel();
    ByteBuffer buf = file.map(FileChannel.MapMode.READ_WRITE,4 * ints.length);
    for (int i : ints) {
      buf.putInt(i);
    }
    file.close();
  } catch (IOException e) {
    throw new RuntimeException(e);
  } finally {
    safeClose(out);
  }
}

private static void safeClose(OutputStream out) {
  try {
    if (out != null) {
      out.close();
    }
  } catch (IOException e) {
    // do nothing
  }
}