34 replies to this topic

[Olof Lagerkvist]

I was using virtual memory in the same sense you did. As memory, which is not exclusively RAM.

Now that i know, that there are RAMDisks and VirtualMemoryDisks, i'm looking for a way to tell the two fast and easy apart.
In case of a PE (which operates without a page file), it probably doesn't matter, but on a full Windows, i use those kinds of drives for one reason only. To speed certain things up. A drive, which ends up in the page file is slower, not faster, than running straight of the HDD.

A driver that allocates non-swappable physical RAM pages must use virtual memory to access the physical RAM. Therefore, even though this memory is never swapped out to disk, it still needs to be paged in to virtual address space of the system process to be accessed. The performance loss for that is probably not huge, but compared to drivers that allocate virtual memory with no other requirements, it may be slower. At least as long as the virtual memory storage is kept in physical RAM and especially when the access is largely random so that the physical page allocating drivers cannot benefit form having the correct page already mapped when next I/O request comes.

In most cases this is quite clearly visible in test results where people have tested various access methods (ranom, linear, small/large buffers etc) on various kinds of memory disks.

[okee]

I want a 5GB ramdisk for maximum speed and to avoid disk thrashing. Which way of using imdisk and perhaps awealloc would be best for that?

 

Up until reading this thread I had assumed that ImDisk always created ramdisks (using physical RAM). But if I understand the posts above correctly then when not using the awealloc driver a virtual memory disk made in imdisk may be using the (disk based) page file rather than physical RAM and the user cannot control which is the case.

 

My first reaction to that was: I'd better start using awealloc then! But Olof's #26 above seem to indicate that there might not be any speed gain from using awealloc to force a ramdisk (using physical RAM) anyway.

 

So now I'm a bit confused. I hope someone here can de-confuse me! :-)

[Sha0]

I want a 5GB ramdisk for maximum speed

A disk-based filesystem on a RAM disk is slower than a RAM-based filesystem in RAM.

 

Please let me ask: Other than the secure erasure of data provided by using non-swappable RAM disks, what are the advantages to using RAM disks?

• The contents have to be populated from somewhere, originally.  Programs using the contents now means the contents have been read into memory twice, instead of once from the original source.
• How many programs do you know that frequently read and write to the disk?  If they mostly read files, then see the last point about the files being read twice.  If you might guess a program as being one that mostly writes, why wouldn't it do most of its work in memory anyway, before writing a final result to a file?

If anyone could enlighten, please do.

 

and to avoid disk thrashing.

Is disk thrashing ever caused by a fragmented filesystem?  If so, defragmenting seems like it would help.

 

Is disk thrashing ever caused by frequent swapping of virtual memory when there is high demand but either too little virtual memory available or too small a total amount of page-file space?  If so, tracking which programs have a high demand and increasing the total amount of page-file space seem like they would help.

 

My first reaction to that was: I'd better start using awealloc then! But Olof's #26 above seem to indicate that there might not be any speed gain from using awealloc to force a ramdisk (using physical RAM) anyway.

 

So now I'm a bit confused. I hope someone here can de-confuse me! :-)

If you use AWE, then the accesses of virtual memory and physical memory do not involve any physical disks, so no disk machinery, if that's a concern.  You could always test and compare the difference; knowing what patterns of usage are typical for yourself.

[okee]

shao0, I use image and audio processing software that frequently read and write.I do not have more answers to your questions. But I'd like to hear what Olof's answer to my initial question is.

[Olof Lagerkvist]

A disk-based filesystem on a RAM disk is slower than a RAM-based filesystem in RAM.

 

Please let me ask: Other than the secure erasure of data provided by using non-swappable RAM disks, what are the advantages to using RAM disks?

• The contents have to be populated from somewhere, originally.  Programs using the contents now means the contents have been read into memory twice, instead of once from the original source.
• How many programs do you know that frequently read and write to the disk?  If they mostly read files, then see the last point about the files being read twice.  If you might guess a program as being one that mostly writes, why wouldn't it do most of its work in memory anyway, before writing a final result to a file?

If anyone could enlighten, please do.

 

Personally, I only use memory disks for files that are created as temporary storage of various kinds. For example web browser cache, object directories used by compilers and other sorts of temporary files. For example, building a large project in Visual Studio could be a lot faster when compilers can store object data on a memory disk instead of physical disk.

 

Also, when migrating 32 bit servers to virtual machines, I usually create a memory disk on the virtual machine host and let the virtual machine mount a virtual disk file on that memory disk. That means that the 32 bit virtual Windows system sees a physical disk that is actually stored in memory of the host. This is a perfect place for the 32 bit virtual system to store page files, temp directories etc. Since 32 bit Windows servers usually use the pagefile a lot, especially if it runs many applications and services, this sometimes gives a lot better performance.

 

shao0, I use image and audio processing software that frequently read and write.I do not have more answers to your questions. But I'd like to hear what Olof's answer to my initial question is.

 

I would say that you need to test both with ImDisk own virtual memory disks and awealloc to see which gives best performance for what you specifically use it for. Awealloc guarantees that physical RAM is used in all cases, but for 64 bit Windows with lots of RAM it is not very likely that a virtual memory backed virtual disk is ever swapped out to disk either. The driver code to access the virtual disk data is a lot simpler for virtual memory disks, but on the other hand awealloc makes sure that it does not consume a large piece of virtual address space for the system process in one block. This is especially important on 32 bit Windows versions.

 

So, I would think that you need to try and compare to figure out which one suits your usage best.

[okee]

Thank your for the reply. I run Win 7 x64 with 16GB RAM. Would the following way of using ImDisk without awealloc minimize the chance of a ImDisk virtual memory disk using the page file rather than physical RAM: 1. A script is set to create a ImDisk virtual memory disk after each reboot. 2. Thereafter the script creates a random file that fills the entire virtual memory disk. 3. Then the script deletes the same file. The idea is that there will be a lot of free RAM right after reboot for ImDisk to use and then "lock" that RAM for the virtual memory disk until it is unmounted or the system rebooted. Is that right?

[Olof Lagerkvist]

Thank your for the reply. I run Win 7 x64 with 16GB RAM. Would the following way of using ImDisk without awealloc minimize the chance of a ImDisk virtual memory disk using the page file rather than physical RAM: 1. A script is set to create a ImDisk virtual memory disk after each reboot. 2. Thereafter the script creates a random file that fills the entire virtual memory disk. 3. Then the script deletes the same file. The idea is that there will be a lot of free RAM right after reboot for ImDisk to use and then "lock" that RAM for the virtual memory disk until it is unmounted or the system rebooted. Is that right?

 

Step 2 and 3 looks a little bit strange to me. Is that just to make sure all pages are committed before "real" usage of the memory disk? I am reasonably sure that it will not affect the risk for the memory disk to be swapped out to disk or not, but it could mean a difference in speed when used for the first time, since Windows has already committed all pages when it has already been used.

[okee]

Is that just to make sure all pages are committed before "real" usage of the memory disk?

Yes, that was the idea.

I am reasonably sure that it will not affect the risk for the memory disk to be swapped out to disk or not

Ok so during a single session, for a single ImDisk mount, the ImDisk can be based on first RAM, then the page file, then RAM again, and so on?

What if the system page file is disabled or set to some minimum size?

[Olof Lagerkvist]

Yes, that was the idea.
Ok so during a single session, for a single ImDisk mount, the ImDisk can be based on first RAM, then the page file, then RAM again, and so on?

 

Yes, that is the whole idea of disk swapping. When an application is in use, the memory it uses is paged in to physical RAM. Then, when another application needs to access more of its memory, the first application's memory may be swapped out and the second application's memory swapped in. When you use the first application again, Windows may need to swap in memory to physical RAM again and so on.

 

What if the system page file is disabled or set to some minimum size?

 

There is usually no reason to do so. In my experience the pagefile is not often used on 64 bit Windows with much physical RAM. It could be good to have a pagefile, at least for the rare cases when it actually will be needed. Because Windows will not use it more than needed there is no reason to disable it just to make sure that it is not unnecessarily used.

 

Try for example my "swapadd" tool and see how much your pagefiles are used!

[okee]

Ok thank you for the explanation. Swapadd is a useful tool!