18 replies to this topic

[eeb]

First of all, I will give deep appreciation to the author of imdisk, because the imdisk helps me to save $119 (the price of VSuit Ramdisk Sever edition).

Currently, I use imdisk to create ramdisk and put the pagefile on it. However, I find that the read/write speed of the created ramdisk is dramatically decreased if the awealloc is active. Below lists the setting of my computer and some performance tests.

My computer: Intel i5-2500k with 16GB ram, running win7 64 bit.

imdisk setting:

1. Scenario 1:
net start imdisk
imdisk -a -t vm -s 8G -m R: -p '/fs:ntfs /q /y'
swapadd r:\pagefile.sys 16M 2048M

2. Scenario 2:
net start imdisk
imdisk -a -f c:windows\system32\drivers\awealloc.sys -s 8G -m R: -p '/fs:ntfs /q /y'
swapadd r:\pagefile.sys 16M 2048M

In scenario 2, I type "net start imdisk" instead of "net start awealloc" because if I type "net start awealloc" in command line, the windows tell me that "system occurs 193 error" and "not a correct Win32 application". It seems that the awealloc is active even if I type "net start imdisk" because the memory usage of system process in windows task manager is invariant (80KB).

Performance test:

I use CrystalDiskMark 3.0.1 to test the read/write speed of the created ramdisk. Here is the result.

Scenario 1:
Read (MB/s), Write (MB/s)
Seq: 6240, 10410
512K: 6197, 9920
4K: 933.8, 727
4K QD32: 1706, 1167

Scenario 2:
Read (MB/s), Write (MB/s)
Seq: 455.9, 112
512K: 390.6, 111
4K: 26.45, 64.95
4K QD32: 27.31, 69.73

From the test results, the performane of scenario 1 is significantly higher than that of scenario 2. To verify these results, I also made a file copying test for both scenarios. I created a 3.5 GB image file on the ramdisk, and then counted the required time for copying this file form ramdisk to ramdisk. For scenario 1, it took less than 2 sec to complete the copying process while scenario 2 needed more then 5 sec.

I am wondering why such things happened. Could someone give me the advice if I made huge or stupid mistakes in the imdisk setting?

Thanks for your help.

Sincerely,

[Olof Lagerkvist]

imdisk -a -f c:windows\system32\drivers\awealloc.sys -s 8G -m R: -p '/fs:ntfs /q /y'

This is entirely wrong. What you do here is that you create an 8 GB image file on disk that overwrites c:windows\system32\drivers\awealloc.sys. This is why you get an error message when you try to start awealloc driver later, because instead of the real awealloc.sys driver you have a huge image file there which Windows will try to load as a driver. This also means that you will not get a RAM disk in this case, your virtual disk I/O will go to an image file on your harddisk which is why you get such bad performance.

Now, to use awealloc, first remove all virtual disks, remove c:windows\system32\drivers\awealloc.sys and reinstall ImDisk. You need to reinstall, because you have overwritten awealloc.sys and need to get the original one installed again. Then, read the FAQ again and use correct syntax for using awealloc (it is supposed to be something like -f \\.\awealloc on the command line instead).

[eeb]

Thank you for your quickly and kindly reply.
Now I realize that what a stupid mistake I made. After uninstall and reinstall the imdisk, I can enable the awealloc driver now.
However, when I attemp to create a ramdisk equal or larger than 4GB with awealloc enabled, an error message appeared as:
-----
Error creating virtual disk:
Not enough storage is available to process this command.
-----
The imdisk setting is listed below:
-----
net start awealloc
imdisk -a -f .awealloc -s 8G -m R: -p '/fs:ntfs /q /y'
-----
Although I can not create a ramdisk above 4GB, I can create many < 4GB ramdisks simultaneously until the memory is run out.
I am really confused because I can create > 4GB ramdisk if the awealloc driver is not used (scenario 1 setting in original post).

Besides, a minor question also arise: the read/write speed of ramdisk in the case with awealloc driver is still significantly lower than that in the case without awealloc driver. Below is the CrystalDiskMark test result in the case with awealloc driver:
-----
Read (MB/s), Write (MB/s)
Seq: 3248, 3898
512K: 3031, 3630
4K: 361.5, 323.3
4K QD32: 546.5, 442.4
-----

Thank you for your help.

Sincerely,

[Olof Lagerkvist]

Thank you for your quickly and kindly reply.
Now I realize that what a stupid mistake I made. After uninstall and reinstall the imdisk, I can enable the awealloc driver now.
However, when I attemp to create a ramdisk equal or larger than 4GB with awealloc enabled, an error message appeared as:
-----
Error creating virtual disk:
Not enough storage is available to process this command.
-----
The imdisk setting is listed below:
-----
net start awealloc
imdisk -a -f \\.\awealloc -s 8G -m R: -p '/fs:ntfs /q /y'
-----
Although I can not create a ramdisk above 4GB, I can create many < 4GB ramdisks simultaneously until the memory is run out.
I am really confused because I can create > 4GB ramdisk if the awealloc driver is not used (scenario 1 setting in original post).

This surprises me too. Could do just do one check for me, it helps me to recognize where this problem may occur. I would like to know what happens if you create a < 4 GB disk and then extends it to be > 4 GB. Does that work?

imdisk -a -f \\.\awealloc -s 3G -m R: -p '/fs:ntfs /q /y'
imdisk -e -s 3G -m R:

Besides, a minor question also arise: the read/write speed of ramdisk in the case with awealloc driver is still significantly lower than that in the case without awealloc driver. Below is the CrystalDiskMark test result in the case with awealloc driver:
-----
Read (MB/s), Write (MB/s)
Seq: 3248, 3898
512K: 3031, 3630
4K: 361.5, 323.3
4K QD32: 546.5, 442.4
-----

I am not surprised that performance is lower, but your figures show a much larger difference than I expected. There is some overhead with page address operations so it could never meet the imdisk.sys built-in RAM disk allocation, but this looks a little too much to me. If you like, you could try to compare them more in "real-life" and see whether or not your experience is in line with the reported difference or not. Would be interesting to know.

[eeb]

Sorry for reply late. I did some tests to check if everything is fine.
As your suggestion, Now, I can create a single > 4GB ramdisk through the extension command. Below is the detail about the memory usage shown in task manager.
-----
Initial: total memory 16291MB, available 13939MB, free 12757MB
Create 3GB ramdisk (R:): available memory 10874MB, free 9685MB
Add additional 3GB to R: available memory 7807MB, free 6618MB
Add additional 3GB to R: available memory 4725MB, free 3523MB
Add additional 2GB to R: available memory 2682MB, free 1474MB
Finally, I have a single 11GB ramdisk on windows.
-----
However, two issues arise during and after the entension of a existed ramdisk.

The first issue is the same as I posted previously. The additional size that will be added to the existing ramdisk can not be equal or larger than 4GB. That is,
"create 3GB ramdisk -> extend to 6GB -> extend to 8GB" is ok, and I get a 8GB ramdisk.
"create 3GB ramdisk -> extend to 8GB" does not work, I got the error message: "Extending disk size... R: Not enough storage is available to process this command"

The second issue is more serious and I suppose it might be the outcome of the first issue.
I successfully create a single 8GB ramdisk through the extension command with the awealloc driver enabled. But I found that the files which stored in the sectors beyond 4GB might be damaged. Below lists some results of my tests.

1. I prepared 4 movie clips, all of them can be played with out any problems if they are stored on the hard disk. No problem means I can ramdomly drag the time seek bar in the player and the movie is started instantly from the time I clicked.
(1) empty the 8Gb ramdisk.
(2) copy 01.avi (1.5GB) to the ramdisk. The movie can be played without problem.
(3) copy 02.avi (1.5GB) to the ramdisk. Both 01.avi and 02.avi can be played without problem.
(4) copy 03.avi (2GB) to the ramdisk, thus the total used space is beyond 4GB. The movies 02 and 03 can be played without problem. But when I opened 01.avi, the movie player is frozen if I drag the time seek bar to some time positions. I think it is because parts of 01.avi are damaged.
(5) the strange point is, if I empty the ramdisk and only copy 04.avi (4.5GB) to the ramdisk, it can be played without problem.

2. I prepared 20 movie clips and copied them to the empty 8GB ramdisk simultaneously. The size of each movie clip is approximate 350MB. Only 12 movie clips (4.2GB) can be played, the rest 8 clips can not be opened by the movie player. However, for the 12 playable clips, few (1 to 2) of them will freeze the movie player if I drag the time seek bar during playing.

3. I prepared two archive files and then,
(1) copy file 01 (3.7GB) to the ramdisk. The file can be uncompressed without error.
(2) copy file 02 (3.5GB) to the ramdisk. Now, the total used space is 7.2GB. The file 02 can be uncompressed while file 01 can not be recognized by the compression software. The software told me that file 01 is not a valid format or it is damaged.

After encountered the above problems, I rebooted the computer and changed the imdisk setting to scenario 1 as the original post (use imdisk without awealloc driver). I repeated the above test several times and no problems occurred. That is, all the files copied to the ramdisk can be accessed without error, no matter how much space of ramdisk is used.

Sincerely,

[eeb]

Hi, sorry for bother you again.
I found another computer (i7-920, Intel X58 chipset, 2*6GB ram, and win7 64bit) and performed the same tests on it.
The win7 64bit system is just installed on this computer. Except superspeed ramdisk plus, no other applications are installed on the operating system.

Firstly, I used superspeed ramdisk plus to create a single 8GB ramdisk and performed the same tests.
All the files (movie clips and archive files) copied to the ramdisk can be accessed without any errors.
Furthermore, unlike the imdisk, this ramdisk can be seen in the windows device manager (I think this is not a big deal since the logic is different between these two programs).

Then, I uninstalled the superspeed ramdisk plus, rebooted, and installed the imdisk.
The same problems (can not create > 4GB ramdisk, files might be damaged) occurred if I created the ramdisk with awealloc driver.
Of course, no problems occurred if only imdisk is used without awealloc.

By the way, during performing the tests, I set the pagefile to be stored on the physical hard disk only. That is, no pagefile on the ramdisk.

So, besides the above questions, is there any risk if I create a ramdisk without awealloc driver and put the pagefile on it?
I have seen your post in another topic which stated that "it is better (saver) to use awealloc driver for allocating memory for a ram disk that holds a swap file"

Thanks
Sincerely,

[Olof Lagerkvist]

Thanks for testing! There seems to be a few problems with sizes > 4 GB in awealloc then. I will try to see what I can do about them in near future, but I have no big 64-bit machine to test on myself so it will be more "theorietical", but I think I know where to look now with your test results! Thanks again!

So, besides the above questions, is there any risk if I create a ramdisk without awealloc driver and put the pagefile on it?
I have seen your post in another topic which stated that "it is better (saver) to use awealloc driver for allocating memory for a ram disk that holds a swap file"

Yes, the memory allocated for RAM disks by imdisk itself is not guaranteed to be non-paged. Therefore there is a risk that a pagefile will be stored on itself at some point and this will either lock up your system or cause BSOD. There is not a big chance for it, but it could happen.

[eeb]

Thank yoy for your kindly help.

Please do not hesitate to contact me if you want some body to test your modified awealloc driver.

Thanks again

Sincerely,

[Thracx]

...the memory allocated for RAM disks by imdisk itself is not guaranteed to be non-paged...

Really? Isn't there a way to allocate memory which will never get paged? Or is this only a problem when awealloc is in use?

Update: I think I've found most of my answer here.

Edited by Thracx, 08 November 2011 - 03:17 PM.

[Olof Lagerkvist]

Really? Isn't there a way to allocate memory which will never get paged? Or is this only a problem when awealloc is in use?

Update: I think I've found most of my answer here.

Yes, just think about it, how would a disk driver that handles a system pagefile work if that driver itself got paged out? In kernel mode, allocating non-paged memory for important tasks is a quite basic thing. But this particular function that imdisk.sys uses to allocate memory for RAM disks, ZwAllocateVirtualMemory, allocates memory that may be paged out. The function that awealloc.sys uses to allocate memory for RAM disks, MmAllocatePagesForMdl, allocates memory that is never swapped to disk (and never relocated in physical memory either). It is just two different ways to allocate memory, that's all.

[Thracx]

Yes...It is just two different ways to allocate memory, that's all.

Thanks for the concise summary. I know about the different allocation methods, but I was just a little shocked when I learned that the method ImDisk was using was the pagable version (i.e. ZwAllocateVirtualMemory). This means that I will need to look into utilizing awealloc.sys via ImDisk so that my ramdisk is never paged (i.e. via MmAllocatePagesForMdl).

My native Win32 is a little over 6 years rusty now - is there a technical reason why you can't provide an option in normal ImDisk to use the non-paged call instead? (I certainly understand why providing such a feature has not been done yet, I'm just curious as to if it can be done at some point.) Since it sounds like it's an option, perhaps we can add this to the feature request list?

Update: "...But this also means that memory remapping is needed when different parts of the RAM disk are accessed." (Olof)
Sounds like the answer is no :'(

Edited by Thracx, 08 November 2011 - 04:09 PM.

[Thracx]

...Please do not hesitate to contact me if you want some body to test your modified awealloc driver...

I will also volunteer to help with this, although note I am currently limited to a 6 GB RamDisk.

[Olof Lagerkvist]

Thank yoy for your kindly help.

Please do not hesitate to contact me if you want some body to test your modified awealloc driver.

I think the problem could have been a few really simple and stupid typical "used-to-code-32-bit" typos. If that was indeed the problem, it should be solved in this beta version:
http://www.ltr-data....86/awealloc.sys (i386)
http://www.ltr-data....64/awealloc.sys (ia64)
http://www.ltr-data....64/awealloc.sys (amd64)
(Chose correct version for your architecture.)

Would be great if you could test and verify if this version works with > 4 GB disks, both allocating > 4 GB in one step, also allocating < 4 GB first and then extending to > 4 GB and then in all cases verify that all of the virtual disk can be used without destroying any data.

[eeb]

Dear

I have downloaded the latest awealloc driver and performed some tests. Below is the test procedure.

Environment: i5-2500K, z68 chipset, 4*4GB RAM, win7 64bit, use awealloc driver to create 8GB ramdisk

Test 1: copy 3 archive files with total 7.6GB size to ramdisk and test if they can be uncompress without error.

Test 2: copy 23 movie clips with total 7.8GB size to ramdisk and test if they can be played without problem.

After repeated the above tests several times, no error was found yet, but I encounter a minor problem regarding the automatically start of ramdisk.

I created a script file which contained the following commands:

-----
net start awealloc
imdisk -a -f .awealloc -s 8G -m R: -p '/fs:ntfs /q /y'
swapadd R:pagefile.sys 16M 2048M
md R:TEMP
-----

I use gpedit to make the script file to be automatically executed when the system starts up.

The temp (or tmp) path in the environmental variable are all changed to R:TEMP.

I found that if I immediately typed the login password and pressed enter after the login screen appeared, the ramdisk appeared late, approximate 20 - 30 sec after logged into system.

Thus, some softwares, such as live messanger and acrobat, do not work properly because no temp (or tmp) folder was found.

If I delayed the time to log into the system, for example, 5-10 sec after the login screen appeared, no problem will occur. I can see ramdisk immediately after log into the system.

After encountered this problem, I used the Task Scheduler instead of gpedit to automatically execute the script file when the system start up. But the problem appeared again!

By the way, the above problem is not appeared if I created the ramdisk by imdisk only.

I think this is not a big problem since it can be prevented (at least ignored) by setting the temp (or tmp) path to other disk or do not rushed log into system after the login screen appeared.

Anyway, there are no > 4GB or file damage issues for the latest awealloc driver.

Thank you for your help.

Sincerely,

[Olof Lagerkvist]

Anyway, there are no > 4GB or file damage issues for the latest awealloc driver.

Great, sounds like a release candidate then. Thanks a lot for your report and for taking your time and work to test!

[Thracx]

...Anyway, there are no > 4GB or file damage issues for the latest awealloc driver.

Great, sounds like a release candidate then. Thanks a lot for your report and for taking your time and work to test!

Awesome, I guess I'll just do my testing after the next main release. Good work y'all!

[Thracx]

(Dup)

Edited by Thracx, 15 November 2011 - 02:52 PM.

[Olof Lagerkvist]

Awesome, I guess I'll just do my testing after the next main release. Good work y'all!

1.5.3 out now.
http://www.ltr-data..../imdiskinst.exe

[eeb]

Have downloaded and installed 1.5.3 imdisk, and no problem occurs when create a single > 4GB ramdisk.

I randomly copied several files to the ramdisk and checked the MD5, SHA1, and CRC32.

No checksum error occurs.

So, it seems that the > 4GB bug no longer exist after version 1.5.3.

Thanks to you and your great imdisk program.

Sincerely,