Jump to content











Photo
- - - - -

Dynamic Memory Management


  • Please log in to reply
62 replies to this topic

#1 friske

friske

    Frequent Member

  • Advanced user
  • 244 posts
  •  
    Italy

Posted 04 November 2014 - 05:49 AM

ImDisk is a great RamDisk.

But the only reason I prefer PrimoDisk is Dynamic Memory Management



#2 v77

v77

    Silver Member

  • Team Reboot
  • 501 posts
  •  
    France

Posted 04 November 2014 - 12:18 PM

The dynamic memory management is implemented in ImDisk Toolkit as a "proxy" (an extension) of the driver.
Don't know if it's as fast as Primo Ramdisk but it's free. :)



#3 friske

friske

    Frequent Member

  • Advanced user
  • 244 posts
  •  
    Italy

Posted 04 November 2014 - 03:16 PM

That's what I wanted to hear.  But how to do?



#4 v77

v77

    Silver Member

  • Team Reboot
  • 501 posts
  •  
    France

Posted 04 November 2014 - 03:40 PM

You just have to check "Allocate Memory Dynamically" in the RamDisk Configuration Tool.
This will start automatically a process called "RamDyn.exe" which will allocate the required memory depending of the requests sent by the driver.

There is also some advanced parameters about the release of memory that is no longer used in the Advanced tab of the RamDisk Configuration Tool, but in most cases you don't need to change them.



#5 friske

friske

    Frequent Member

  • Advanced user
  • 244 posts
  •  
    Italy

Posted 04 November 2014 - 03:52 PM

ImDisk release ram when delete files also in XP?

is a long time that implements "Allocate Memory Dynamically"?

Do it support Invisible Memory?



#6 v77

v77

    Silver Member

  • Team Reboot
  • 501 posts
  •  
    France

Posted 04 November 2014 - 04:28 PM

In fact, it's not exactly "when a file is deleted".
In order to not depend of the file system, it checks the difference between the memory actually allocated and the used space on the volume (or more exactly, it checks the variation of this difference). And this check is done regularly, once every 10 seconds by default (this is configurable).

Basically, when this difference is large enough, it wipes (almost) all the free space. All written block filled with 0 is freed, and so, the memory of deleted files is released.
This way, we don't need to rely on each file system particularity, and we don't even need to know where are located the deleted files.

It works on all systems supported by Imdisk Toolkit, that is, from XP to 8.1 in 32 or 64-bit (not tested on Windows Server editions but there should be no issue).
It's implemented since the beginning of year (19 January 2014).
And no, there is no support for "invisible memory".


  • friske likes this

#7 friske

friske

    Frequent Member

  • Advanced user
  • 244 posts
  •  
    Italy

Posted 04 November 2014 - 07:46 PM

Files compacted with compact.exe take up less space on ramdisk or ImDisk however, compresses the data by itself??



#8 v77

v77

    Silver Member

  • Team Reboot
  • 501 posts
  •  
    France

Posted 04 November 2014 - 08:49 PM

What do you mean?
There is no compression with the dynamic memory management of Imdisk Toolkit, unless of course you manually format a volume with the NTFS compression enabled.

RamDyn.exe allocates a block of memory only when there is data that must be written, and if these data are different to 0.
If a block has never been written, that is, if a block is not allocated, its content is considered as 0.
And if a block about to be written is filled with 0, the related memory block is freed (this is why wiping the free space with 0 will automatically free the memory blocks that were allocated by deleted files).

In fact, this is a bit more complicated because for performances, memory is allocated by blocks of 1 MB. But this is the idea.



#9 friske

friske

    Frequent Member

  • Advanced user
  • 244 posts
  •  
    Italy

Posted 04 November 2014 - 10:03 PM

In fact, it's not exactly "when a file is deleted".
In order to not depend of the file system, it checks the difference between the memory actually allocated and the used space on the volume (or more exactly, it checks the variation of this difference). And this check is done regularly, once every 10 seconds by default (this is configurable).

Do it is possible do disable regularly and force on the fly from comman line?



#10 v77

v77

    Silver Member

  • Team Reboot
  • 501 posts
  •  
    France

Posted 04 November 2014 - 11:04 PM

Do it is possible do disable regularly and force on the fly from comman line?

 

Not currently. Maybe in a future version...
However, you can tune the cleanup function in the Advanced tab of the Ramdisk Configuration Tool ("Dynamic RamDisk Cleanup..."). There is 4 parameters. Leave your mouse over an entry to display a tooltip for more informations.

 

That said, with the default parameters, even after several hours, the task manager still displays 0 second of CPU consumption for RamDyn.exe for my current ramdisk. So this test really takes almost nothing...


  • friske likes this

#11 friske

friske

    Frequent Member

  • Advanced user
  • 244 posts
  •  
    Italy

Posted 05 November 2014 - 09:38 PM

Do it is possible to use "Dynamic Memory Management" via command line?

Why "Use Awealloc" is disabled when "Dynamic Memory Management" is checked?

Now the only reason I prefer Gavotte in 32-bit OS with more that 3GB ram is PAE support. ;)



#12 v77

v77

    Silver Member

  • Team Reboot
  • 501 posts
  •  
    France

Posted 05 November 2014 - 11:52 PM

Do it is possible to use "Dynamic Memory Management" via command line?

 

Well, it's not really designed for that, but yes, it's possible. However, please note that there is no warranty that it will work in all future versions.
As it's a "proxy", RamDyn.exe must be started and ready before using imdisk.exe. The GUI does this check by testing a synchronization object (which cannot be done in command line), but you can simply assume that RamDyn.exe is ready by adding a small delay after it.
So, in short, in the install folder of ImDisk Toolkit, you can use a command like that (requires admin rights):
RamDyn "Z:" XXX 1048576 10 10 10 10 & timeout 1 & imdisk -a -t proxy -o shm -f RamDynXXX -m "Z:"

"XXX" is a string of your choice of at most 10 characters used as an identifier. Of course, it must be unique. Note that there is no space in "RamDynXXX".
1048576 is the size of the ramdisk in KB.
10 10 10 10 are the 4 parameters of the cleanup function. Check the Advanced tab of the GUI for their meaning.
timeout is a function available on Vista and later that waits a given number of seconds.

 

 

Why "Use Awealloc" is disabled when "Dynamic Memory Management" is checked?

 

Awealloc is a driver that uses the AWE API of Windows to allocate memory. Here, memory is directly allocated by RamDyn.exe (which is a user mode process). Awealloc is not designed to be used by something else than the ImDisk driver itself.


Edited by v77, 19 April 2015 - 05:38 PM.

  • friske likes this

#13 friske

friske

    Frequent Member

  • Advanced user
  • 244 posts
  •  
    Italy

Posted 05 November 2014 - 11:59 PM

Also without Awealloc RamDyn is able to handle RAM > 4GB?

#14 friske

friske

    Frequent Member

  • Advanced user
  • 244 posts
  •  
    Italy

Posted 05 November 2014 - 11:59 PM

Also without Awealloc RamDyn is able to handle RAM > 4GB?



#15 v77

v77

    Silver Member

  • Team Reboot
  • 501 posts
  •  
    France

Posted 06 November 2014 - 12:12 AM

Not on a 32-bit system. As far as I know, a driver is required for doing such a thing, and RamDyn.exe is only a user mode process.
So, unless Olof changes the driver for using the unmanaged memory, you should not expect such a feature from ImDisk.



#16 friske

friske

    Frequent Member

  • Advanced user
  • 244 posts
  •  
    Italy

Posted 06 November 2014 - 12:23 AM

on a 64-bit system ImDisk self-handle > 4GB?



#17 v77

v77

    Silver Member

  • Team Reboot
  • 501 posts
  •  
    France

Posted 06 November 2014 - 12:35 AM

Of course! :)



#18 Olof Lagerkvist

Olof Lagerkvist

    Gold Member

  • Developer
  • 1315 posts
  • Location:Borås, Sweden
  •  
    Sweden

Posted 06 November 2014 - 10:27 AM

Awealloc is a driver that uses the AWE API of Windows to allocate memory. Here, memory is directly allocated by RamDyn.exe (which is a user mode process). Awealloc is not designed to be used by something else than the ImDisk driver itself.

 
 
The corresponding API for user mode is VirtualAlloc with MEM_PHYSICAL. You would get the same limitations as with awealloc driver though. That is, no allocation beyond 4 GB on 32 bit client versions (for example 32-bit XP, while 32-bit Server 2003 works with > 4 GB allocations). This could probably be of interest for applications like RamDyn too.
http://msdn.microsof...p/aa366531.aspx
 
It would also be possible to use awealloc from applications like RamDyn, but the problem is that awealloc emulated in-memory "files" only support growing and not shrinking allocated memory. This means that RamDyn would practically need to create one awealloc file for each block it allocates, which I guess would mean a lot of kernel overhead. (One handle, FileObject etc for each memory block.) It could sometimes be useful to use awealloc in other applications though, I have for example used it in some database applications that have originally been designed to work with a temporary file. Since awealloc emulates a file, such applications are pretty straight-forward to change to using awealloc instead. But in the case of RamDyn, I would say that using VirtualAlloc/MEM_PHYSICAL directly instead would make a lot more sense in several ways.
 
If we are talking rather about "unmanaged" memory, that is memory that Windows kernel has not claimed administration of at all, a driver is needed in all cases. The only way to use such memory is, as far as I know, to call MmMapIoSpace API in kernel mode. The awealloc driver does not support such allocations. There are risks involved, like for example there is no way to detect conflicting use of this kind of memory since there is no operating system that manages allocations from it.



#19 v77

v77

    Silver Member

  • Team Reboot
  • 501 posts
  •  
    France

Posted 06 November 2014 - 01:31 PM

The corresponding API for user mode is VirtualAlloc with MEM_PHYSICAL. You would get the same limitations as with awealloc driver though. That is, no allocation beyond 4 GB on 32 bit client versions (for example 32-bit XP, while 32-bit Server 2003 works with > 4 GB allocations). This could probably be of interest for applications like RamDyn too.
http://msdn.microsof...p/aa366531.aspx

 

Thanks for the explanation. I was not very optimistic about the result because of the required privileges, but the example seems to work (with admin rights of course), and even with the evil "void main()"... :D



#20 friske

friske

    Frequent Member

  • Advanced user
  • 244 posts
  •  
    Italy

Posted 06 November 2014 - 04:17 PM

do it not possible to use -p imdisk parameter?

#21 friske

friske

    Frequent Member

  • Advanced user
  • 244 posts
  •  
    Italy

Posted 06 November 2014 - 04:18 PM

do it not possible to use -p imdisk parameter?



#22 friske

friske

    Frequent Member

  • Advanced user
  • 244 posts
  •  
    Italy

Posted 06 November 2014 - 04:26 PM

solved -p parameter in RamDyn.exe instead of imdisk.exe



#23 friske

friske

    Frequent Member

  • Advanced user
  • 244 posts
  •  
    Italy

Posted 06 November 2014 - 09:54 PM

From the command line I do not release the memory freed.

First I was wrong to write.

I use:

RamDyn.exe "M:" XXX 1048576 10 10 10 10

ImDisk.exe -a -o fix -o hd -o rw -m M: -p "/fs:NTFS /C /Q /Y /V:Alx" -t proxy -o shm -f RamDynXXX

 

Again: Often the committ cpu many often more than 90%



#24 v77

v77

    Silver Member

  • Team Reboot
  • 501 posts
  •  
    France

Posted 10 November 2014 - 09:13 PM

The corresponding API for user mode is VirtualAlloc with MEM_PHYSICAL. You would get the same limitations as with awealloc driver though. That is, no allocation beyond 4 GB on 32 bit client versions (for example 32-bit XP, while 32-bit Server 2003 works with > 4 GB allocations). This could probably be of interest for applications like RamDyn too.
http://msdn.microsof...p/aa366531.aspx

 

In fact, there is a notable difference: the physical address range cannot be specified in user mode.

So, I wonder if a 32-bit server edition of Windows would allocate the memory with AllocateUserPhysicalPages at addresses higher than 4 GB if I request only a small amount of memory...

But, is it really useful? If a 32-bit system is able to manage more than 4 GB, there should be no difference on where we allocate the physical memory, and so, I wonder why in AWEAlloc, you attempt to allocate at the highest possible address (up to 8 GB)...



#25 Olof Lagerkvist

Olof Lagerkvist

    Gold Member

  • Developer
  • 1315 posts
  • Location:Borås, Sweden
  •  
    Sweden

Posted 10 November 2014 - 11:01 PM

In fact, there is a notable difference: the physical address range cannot be specified in user mode.


Oh, that seems correct and I had missed that part!

 

So, I wonder if a 32-bit server edition of Windows would allocate the memory with AllocateUserPhysicalPages at addresses higher than 4 GB if I request only a small amount of memory...

But, is it really useful? If a 32-bit system is able to manage more than 4 GB, there should be no difference on where we allocate the physical memory, and so, I wonder why in AWEAlloc, you attempt to allocate at the highest possible address (up to 8 GB)...

 

 

Actually, the allocation attempts are: First above 8 GB, then above 6 GB, 5 GB, 4 GB and finally anywhere. And indeed, this could seem a little bit strange. The reason it was done in this way was some conclusions that followed a few experiments after some discussions in various forums. In my own case, I had a Server 2003 Standard Edition running, that because of license limitations could only use 4 GB. I had 8 GB installed though, so I thought it would make sense to try the /PAE switch and write a driver that attempted to allocate beyond the limit where Windows itself would allocate physical pages. This is what later became awealloc driver. This turned out to work so that I could allocate the "lost memory" above 4 GB through awealloc and use it as a virtual disk where I could create a pagefile. I ran my own Server 2003 like this for several years and also installed this on a few other as well.

 

The conclusion was that Windows would never allocate physical pages above 4 GB on Standard Edition because of license based limitations. But it still allowed drivers to explicitly allocate any range of AWE addressable memory for there own purposes.

 


  • friske likes this




0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users