Improve I / O performance by extending RandomAccessFile class with buffer — reprint

< H3 id = "n1002e" > main body:

At present, the most popular J2SDK version is the 1.3 series. Developers using this version need random access to files, so they have to use the RandomAccessFile class. Its I / O performance is far from that of other common development languages, which seriously affects the operation efficiency of the program.

Developers urgently need to improve efficiency. Next, analyze the source code of RandomAccessFile and other file classes, find out the crux, improve and optimize them, and create a random file access class bufferedrandomaccessfile with good "sex / price ratio".

We can see that the gap between the two is about 32 times, and RandomAccessFile is too slow. First look at the source code of the two key parts, compare and analyze them, and find out the reasons. 1.1. [RandomAccessFile] it can be seen that every time RandomAccessFile reads / writes a byte, it needs to perform an I / O operation on the disk. 1．2．[BufferedInputStream] = count) { fill(); if (pos >= count) return -1; } return buf[pos++] & 0xff; // Read} private void fill() throws IOException {if (markpos < 0) POS = 0; / * no mark: throw away the buffer * / else if (POS > = buf. Length) / * no room left in buffer * / if (markpos > 0) {/ * can throw away early part of the buffer * / int SZ = pos - markpos; system.arraycopy (buf, markpos, buf, SZ); POS = SZ; markpos = 0;} else if (buf.length >= marklimit) { markpos = -1; /* buffer got too big,invalidate mark */ pos = 0; /* drop buffer contents */ } else { /* grow buffer */ int nsz = pos * 2; if (nsz > marklimit) nsz = marklimit; byte nbuf[] = new byte[nsz]; System.arraycopy(buf,nbuf,pos); buf = nbuf; } count = pos; int n = in. read(buf,pos,buf.length - pos); if (n > 0) count = n + pos; } } 1.3. [bufferedoutputstream] < div > it can be seen that for each byte read / write of buffered I / O putstream, if the data to be operated is in the buf, read / write the memory buf [] directly; Otherwise, fill buf [] from the corresponding position of the disk, and then directly read / write the buf [] of the memory. Most of the read / write operations are operations on the memory buf []. 1.3. Summary the unit of memory access time is nanosecond (10e-9), and the unit of disk access time is millisecond (10e-3). Similarly, the overhead of one operation is one million times faster than that of the disk. Theoretically, it can be predicted that even if the memory is operated for tens of thousands of times, the time spent is far less than that of one I / O to the disk. Obviously, the latter reduces the I / O overhead of the disk and improves the access efficiency by increasing the buf access in the memory. Of course, this also increases the overhead of the buf control part. From the practical application, the access efficiency is improved by 32 times. The random access class is different from the sequential class. The former is created by implementing the datainput / dataoutput interface, while the latter is created by extending filterinputstream / filteroutputstream and cannot be copied directly. 2.1. Open the buffer buf [default: 1024 bytes] as the common buffer for read / write. 2.2. Implement read buffer first. Basic principle of read buffer logic: a wants to read a byte of the POS position of the file. B check whether the buf exists? If yes, read directly from buf and return the character byte. C if not, the buf relocates to the location of the POS, fills the buffer with the file content of bufsize bytes near the location, and returns B. The key codes and their descriptions are given below: this bufendpos ) { this.flushbuf(); this.seek(pos); if ((pos < this.bufstartpos) || (pos > this.bufendpos)) throw new IOException(); } this. curpos = pos; return this. buf[(int)(pos - this.bufstartpos)]; } // Void flushbuf(): if bufdirty is true, write the data in buf [] that has not been written to the disk to the disk. private void flushbuf() throws IOException { if (this.bufdirty == true) { if (super.getFilePointer() != this.bufstartpos) { super.seek(this.bufstartpos); } super. write(this.buf,this.bufusedsize); this. bufdirty = false; } } // Void seek (long POS): move the file pointer to the POS position, and fill the buf[] map into the file block where the POS is located. public void seek(long pos) throws IOException { if ((pos < this.bufstartpos) || (pos > this.bufendpos)) { // seek pos not in buf this.flushbuf(); if ((pos >= 0) && (pos <= this.fileendpos) && (this.fileendpos != 0)) { // seek pos in file (file length > 0) this.bufstartpos = pos * bufbitlen / bufbitlen; this.bufusedsize = this.fillbuf(); } else if (((pos == 0) && (this.fileendpos == 0)) || (pos == this.fileendpos + 1)) { // seek pos is append pos this.bufstartpos = pos; this.bufusedsize = 0; } this. bufendpos = this. bufstartpos + this. bufsize - 1; } this. curpos = pos; } // Int fillbuf(): fill buf [], according to bufstartpos. private int fillbuf() throws IOException { super.seek(this.bufstartpos); this.bufdirty = false; return super.read(this.buf); } } So far, buffered read is basically realized, Copy a 12 megabyte file byte by byte (this involves reading and writing. Use bufferedrandomaccessfile to try the reading speed): reading and writing takes time (seconds) the visible speed is significantly improved, which is comparable to bufferedinputstream + datainputstream. 2.3. Realize write buffering. The basic principle of write buffering logic: a wants to write a byte of the POS location of the file. B checks whether there is such a mapping in the buf. If so, write directly to the buf and return to true. C if not, the buf relocates to the POS location and returns the location The file content of nearby bufsize bytes fills buffer and returns B. The key codes and their descriptions are given below: = this bufstartpos) && (pos <= this.bufendpos)) { // write pos in buf this.buf[(int)(pos - this.bufstartpos)] = bw; this.bufdirty = true; if (pos == this.fileendpos + 1) { // write pos is append pos this.fileendpos++; this.bufusedsize++; } } else { // write pos not in buf this.seek(pos); if ((pos >= 0) && (pos <= this.fileendpos) && (this.fileendpos != 0)) { // write pos is modify file this.buf[(int)(pos - this.bufstartpos)] = bw; } else if (((pos == 0) && (this.fileendpos == 0)) || (pos == this.fileendpos + 1)) { // write pos is append pos this.buf[0] = bw; this.fileendpos++; this.bufusedsize = 1; } else { throw new IndexOutOfBoundsException(); } this. bufdirty = true; } this. curpos = pos; return true; } So far, the buffer write is basically realized. Copy a 12 megabyte file byte by byte, (this involves reading and writing. Combined with buffered reading, use bufferedrandomaccessfile to try the speed of reading / writing): reading and writing takes time (seconds) it can be seen that the comprehensive read / write speed has exceeded bufferedinput / OutputStream + datainput / OutputStream. Optimization principle: frequently called statements need to be optimized most, and the optimization effect is the most obvious. When multiple nested logical judgments are made, the most likely judgments should be placed on the outermost layer. Unnecessary new should be reduced. Here is a typical example: the seek function is used in various functions , calls are very frequent. The above stressed line of statement determines the mapping location of buf [] corresponding to the current file according to POS and bufsize, and it is obviously not a good method to use "*" and "/". Optimization 1: this bufstartpos = (pos << bufbitlen) >> bufbitlen; Optimization 2: this bufstartpos = pos & bufmask; // this. bufmask = ~((long)this. bufsize - 1); Both are more efficient than the original, but the latter is obviously better, because the former requires two shift operations The latter requires only one logic and operation (bufferask can be obtained in advance). So far, the optimization has been basically realized. Copy a 12 megabyte file by byte (this involves reading and writing. Combined with buffered reading, try the reading / writing speed with the optimized bufferedrandomaccessfile): reading and writing takes time (seconds) bufferedoutputstream + dataoutputstream. Although the optimization is not obvious, it is still faster than that before optimization. Perhaps this effect will be more obvious on older machines. The above comparison is that sequential access, even random access, has more than one byte in most cases, so the buffer mechanism is still effective. General sequential access classes need to realize random access It's not easy. Provide file append function: provide file current location modification function: return the file length (different from the original RandomAccessFile class due to buf reading and writing): return the current pointer of the file (it is different from the original RandomAccessFile class because it is read and written through buf): it provides the function of buffering and writing multiple bytes of the current position: this. Fileendpos) this fileendpos = writeendpos; this. seek(writeendpos+1); } public void write(byte b[]) throws IOException { this.write(b,b.length); } Provides buffered reading of multiple bytes in the current position: buf [] if (readendpos > this. Fileendpos) {/ / read B [] part in file len = (int) (this. Length() - this. Curves + 1);} super. seek(this.curpos); len = super. read(b,len); readendpos = this. curpos + len - 1; } this. seek(readendpos + 1); return len; } public int read(byte b[]) throws IOException { return this.read(b,b.length); } public void setLength(long newLength) throws IOException { if (newLength > 0) { this.fileendpos = newLength - 1; } else { this.fileendpos = 0; } super. setLength(newLength); } public void close() throws IOException { this.flushbuf(); super.close(); } So far, the improvement work is basically completed. Try the new multi byte read / write function to copy a 12 megabyte file by reading / writing 1024 bytes at the same time, (this involves reading and writing. Try the speed of reading / writing with the improved bufferedrandomaccessfile): reading and writing takes time (seconds) jdk1.4 provides NiO classes, in which mappedbytebuffer class is used to map buffer and random file access. It can be seen that Java designers also see the problem of RandomAccessFile and improve it. How to copy files through mappedbytebuffer + RandomAccessFile? The following is the main part of the test program: stem. Currenttimemillis(); for (int i = 0; i < size; i++) { byte b = mbbi.get(i); mbbo.put(i,b); } fcin. close(); fcout. close(); rafi. close(); rafo. close(); System. out. println("Spend: "+(double)(System.currentTimeMillis()-start) / 1000 + "s"); Try jdk1 4 mapping buffer read / write function, copy a 12 megabyte file byte by byte, (this involves reading and writing): reading and writing takes time (seconds) bufferedoutputstream + dataoutputstream is really good. It seems that jdk1.4 has made great progress over 1.3. If random access to files is required when developing software with version 1.4 in the future, it is recommended to use mappedbytebuffer + RandomAccessFile. However, in view of the fact that most programs developed with jdk1.3 and earlier versions account for the vast majority, if you The developed Java program uses the RandomAccessFile class to randomly access files, and is worried about being criticized by users because of its poor performance. Please try the bufferedrandomaccessfile class provided in this article. You don't need to override it. Just import this class and change all randomaccessfiles to bufferedrandomaccessfile. The performance of your program will be greatly improved, That's all you have to do. Reprint: http://www.ibm.com/developerworks/cn/java/l-javaio/ To sum up, the above is collected and sorted out by the programming home for you. By extending the RandomAccessFile class to have buffer to improve I / O performance -- reprint all the contents. I hope this article can help you solve the program development problems encountered by extending the RandomAccessFile class to have buffer to improve I / O performance -- reprint. If you think the content of the programming home website is good, you are welcome to recommend the programming home website to programmers and friends. This graphic content is collected and provided by netizens on the Internet as a learning reference. The copyright belongs to the original author. Like to share programming technology and work experience with others. Welcome to join the programming team