Java 8 HashMap high memory usage

I use HashMap to store qtable to implement reinforcement learning algorithm My HashMap should store 15 million entries When I run the algorithm, I see that the memory used by the process exceeds 1000000k When I calculate memory, I don't expect it to use more than 530000k I tried to write an example, and I got the same high memory utilization:

public static void main(String[] args) {
    HashMap map = new HashMap<>(16_000_000,1);
    for(int i = 0; i < 15_000_000; i++){
        map.put(i,i);
    }
}

My memory:

Each entry set contains 32 HashMap instances with a capacity of 15000000 bytes. Usage: 32 * Size 4 * capability memory = (15000000 * 32 15000000 * 4) / 1024 = 527343.75k

Where is my memory wrong?

Solution

Well, in the best case, let's assume a word length of 32 bits / 4 bytes (using compressed oops and compressed classes pointers) Then, the mapping entry consists of two words, JVM overhead (Klass pointer and tag word), key, value, hash code and next pointer, a total of 6 words, in other words, 24 bytes Therefore, having 15 million entry instances will consume 360 MB

In addition, there is an array of entries HashMap uses a power of 2, so for 15000 entries, the array size is at least 16777216 and consumes 64 MIB

Then you have 30000 integer instances The problem is map Put (I, I) performs the boxing operation twice, and encourages the JVM to reuse objects when boxing, but it is not necessary to do so, and will not be reused in simple programs that you may complete before the optimizer interferes

Specifically, the first 128 integer instances are reused because sharing is necessary for values in the range of - 128... 127, but the implementation is realized by initializing the entire cache when used for the first time. Therefore, for the first 128 iterations, it will not create two instances, but the cache consists of 256 instances, which is twice the number. Therefore, we finally use 30000 integer instances again The integer instance contains at least two JVM specific words and actual int values, which will generate 12 bytes. However, due to the default alignment, the actual memory consumed will be 16 bytes, which can be divided into 8 bytes

Therefore, 30000 created integer instances consume 480 MB

This makes a total of 360 MB 64 MIB 480 MB, more than 900 MB, making the 1 GB heap size completely reasonable

But this is the purpose of the analysis tool After running your program, I got

Note that this tool only reports the used size of the object, that is, the 12 bytes of the integer object, regardless of the padding that will be noticed when viewing the total memory allocated by the JVM