Optimizing for Best Throughput

The following parameters can have an impact on the average data throughput for file I/O.

Number of buffers
Ideally, all FAT data and all directory data required should fit into the buffer cache. For example, a FAT-16 partition with 40000 clusters, 256 root directory entries, and 5 medium-size directories (32 entries each) would require about 157 + 16 + 10 = 183 buffers to hold this data. To have a few spare buffers for application files, 200 buffers would be a good value. For applications frequently accessing several logical drives, enough buffers for all drives should be allocated. Allocating too many buffers can, however, reduce performance as RTFiles-32 must search the complete buffer cache every time a sector is required. More than 1000 buffers will rarely improve performance. The optimal number of buffers depends on many factors such as speed of the disk, speed of the CPU, number or disks in concurrent operation, file I/O characteristics of the application, number of simultaneously open files, etc. The best buffer configuration can only be found by trying different values, typically in the range 8 to 1000. Function RTFBufferInfo can be used to inspect the efficiency of the buffer cache.
Reading/Writing on sector boundaries
Applications that keep file pointers aligned on sector boundaries and read and write full sectors bypass the RTFiles-32 internal buffer cache. The data is transferred directly between the device and the application's data area.
Reading/Writing large data blocks
Most devices will exhibit superior performance when many sectors are read in a single driver call. For example, reading 18 contiguous sectors from a single track on a 1.44M diskette may take approximately 18 times as much time if RTFRead is called 18 times to read 512 bytes instead of a single call to read 9k.
Avoid many seeks
Except for contiguous files on exFAT volumes, seek operations are slow. Seeks can be avoided by reading/writing sequentially (i.e., avoiding calls to RTFSeek), using unfragmented files (see function RTFExtend), and not accessing several different files interleaved or simultaneously on the same device.
RTFOpen flag RTF_CACHE_DATA
This flag will have a positive effect on performance if file read accesses use small block sizes and frequent seek operations are performed. For sector-aligned accesses, it has no effect. If the file is read sequentially, performance may suffer if this flag is set as FAT and directory data will be pushed out of the buffer cache by application data which will never be used again.
RTFOpen flag RTF_LAZY_DATA
This flag will have a positive effect on performance if file write accesses use small block sizes and frequent seek operations are performed. For sector-aligned accesses, it has no effect. If the file is written sequentially, performance may suffer if this flag is set.
Device flag RTF_DEVICE_LAZY_WRITE
This flag will improve performance if many small files are frequently created, renamed, deleted, extended, etc. It does, however, compromise data security. If the program terminates abnormally, not all data may be flushed to the disk, leaving the volume in an inconsistent state.
Device flag RTF_DEVICE_SINGLE_FAT
This flag will improve performance if many files are frequently created, renamed, deleted, extended, etc. However, the volume will be left with an invalid second copy of the FAT, causing disk checking utilities such as SCANDISK or CHKDSK to report errors. Nevertheless, the volume will be in a consistent state. The only disadvantage is that disk repair utilities will not be able to recover a volume when the primary copy of the FAT has been corrupted (something that only rarely succeeds in practice anyway).

Left Advanced Topics

Down Optimizing for Best Data Security