Java – BufferedReader in multi-core environment

I have eight documents Each of them is about 1.7 GB I'm reading these files into a byte array, and the operation is fast enough

Then read each file as follows:

BufferedReader br=new BufferedReader(new InputStreamReader(new ByteArrayInputStream(data)));

When processing sequentially with a single core, it takes 60 seconds to complete However, when allocating calculations on eight separate cores, each file takes far more than 60 seconds

Because the data is in memory and no IO operation is performed, I think the time required for each core to process a file should not exceed 60 seconds Therefore, a total of 8 files should be completed in 60 seconds, but this is not the case

Did I miss something about the BufferedReader behavior? Or any reader used in the above code

It is worth mentioning that I am using this code to upload files first:

byte[] content=org.apache.commons.io.FileUtils.readFileToByteArray(new File(filePath));

All codes are as follows:

For each file
 read the file into a byte[]
 add the byte[] to a list
end For
For each item in the list
 create a thread and pass a byte[] to it
end For

Solution

How do you actually "distribute calculations"? Is synchronization involved? You just create 8 threads to read 8 files?

What platform are you running (Linux, windows, etc.)? I saw the seemingly strange behavior from the windows scheduler, and then it moved a single process from the core to the core to try to balance the load between the cores This eventually leads to performance degradation, not just allowing a single core to use more than the rest of the core

The content of this article comes from the network collection of netizens. It is used as a learning reference. The copyright belongs to the original author.
THE END
分享
二维码
< <上一篇
下一篇>>