14 replies to this topic

[misty]

Whilst helping Erwan with the development of DiskMgr (by testing - LOTS of testing) we noticed a bug with the Microsoft Disk API - more specifically, when using DeviceIoControl+IOCTL_DISK_SET_DISK_ATTRIBUTES.

Please note that the current release of DiskMgr is not affected by this problem as a workaround was introduced (temporarilly setting any ONLINE disk as OFFLINE before applying the changes).

The following behaviour was observed when changing the Read attribute of a DISK (switching between Read-Only / Read-Write) -

OFFLINE DISK
If the target DISK was marked as OFFLINE, then changing the READ attribute worked without any problems.

ONLINE DISK
If the target DISK was marked as ONLINE, then changing the READ attribute appeared to work. As an example, an ONLINE disk was changed from Read-Write to Read-Only. Following the change, the disk attributes were correctly displayed as Read-Only in the DiskMgr UI and also in DiskPart.

Now here's the interesting part - despite the disk being reported as Read-Only it was possible to save a file to the disk (or more accurately to a mounted volume on the DISK). The disk appeared fully accessible (in regards to creating and editing files) from a file manager (Q-Dir) - it was also possible to save files to the disk from NOTEPAD

And another interesting observation. A file created on the disk marked as Read-Only disappeared following an OFFLINE/ONLINE action! I'm wondering if some kind of file based write filter was at play here.

More testing later. I will use one of the development builds of DiskMgr to check whether it's possible to delete a file/folder from a disk reported to be write protected (after an ONLINE disk is changed from Read-Write to Read-Only) and whether it remains deleted following an OFFLINE/ONLINE action.

With Erwan's permission I am happy to upload a DiskMgr development build (one of the version 0.7 builds if memory serves) if anyone else wants to test this.



Misty

[Wonko the Sane]

Side note (affecting clonedisk but possibly also DiskMgr), ONLY since you are into making tests and before I forget about this.

 

Clonedisk must *somehow* use something connected to Explorer (or a connected API/whatever).

There is a (Policy or easier Registry) setting (from way back in time) to remove in Explorer some drive letters.

I tipically use it to make on a second disk a volume that is not normally visible to the user where I store backups, tools, etc.(and that I access via command prompt or through 7-zip File Manager), I have alway used the nice Freeware HideCalc to make a .reg:
http://www.hidecalc.co.uk/

that resolves around two keys (example):

 

[HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Policies\Explorer]
"nodrives"=dword:A0
"NoViewOnDrive"=dword:A0

 

A few days ago I was fiddling with Clonedisk (on a Windows 7SP1) and noticed that it cannot save to these "Explorerdriveletterless" volumes and that it triggers a message *like* "A restriction is in effect ...".

 

Wonko

[erwan.l]

Hi Guys,

 

CloneDisk (v2.3.6) and DiskMgr (v0.9) use the same method : DeviceIoControl+IOCTL_DISK_SET_DISK_ATTRIBUTES .

 

One difference is that DiskMgr implements a workaround : it goes offline/online IF the disk is online when setting Ro or Rw.

CloneDisk will implement that same workaround shortly.

 

Sidenote, when changing the online/offline state, Explorer will be updated (by the system, not by the software).

 

Whereas, when changing the ro/rw state (when the disk is online), Explorer will ignore it.

Going from RO to RW is ok : Explorer will keep denying you write access.

Going from RW to RO is more problematic : Explorer will keep accepting write requests but these will go to the internal cache and will get lost.

 

Very similar behavior to some of the imdisk proxies I made  - see here.

 

Regards,

Erwan

[erwan.l]

Quoting Olof from the above mentionned thread .

 

 

If you create a virtual disk volume with properties that indicate write support and if write requests then fail on this disk volume, you would get something like that effect. Practically, you would fill up filesystem internal cache and the filesystem driver will keep trying to write out this data once in a while. There is always a risk for blue-screen on system freezing when you fill up filesystem cache completely, though.
 

[misty]

I forgot to mention the following in my previous post.

Step 1
DISK 0 was ONLINE and Read-Write - WinHex used to generate a checksum of Volume 1 (Partition 1 on DISK 0).

Step 2
DISK 0 attributes changed to Read-Only whilst the disk was ONLINE - WinHex used to generate a checksum of Volume 1 (Partition 1 on DISK 0).

Step 3
Text file created on Volume 1 - WinHex used to generate a checksum of Volume 1 (Partition 1 on DISK 0).

Step 4
DISK 0 set as OFFLINE then ONLINE. The file created in Step 3 dissapeared. WinHex used to generate a checksum of Volume 1 (Partition 1 on DISK 0).

The checksum did not change at all between Step 1 to Step 4. This is one of the reasons for my earlier speculation about a file based write filter being at work. Erwan's post now clarifies that this is more likely to be the FileSystem Internel Cache - something I have never heard of before today.


Regards,

Misty

[Zoso]

hi all,

cool stuff going on here!

A few days ago I was fiddling with Clonedisk (on a Windows 7SP1) and noticed that it cannot save to these "Explorerdriveletterless" volumes and that it triggers a message *like* "A restriction is in effect ...".


Wonko

Im almost certain that i have seen such a message while experimenting with FBWF on a HDD that had multiple partitions. i wasnt using clonedisc at that time though, since misty mentioned write filters, i thought i would toss that out there.

Ive been using EWF and FBWF together where EWF is active on the system OS partition and FBWF active on all others so that only certain exceptions are writable. sorry i dont remember more exact details, i just remember seeing that message when trying to figure out how FBWF worked and getting it set up the way i wanted.

[genetix]

As far I understand MS. You need read and write access to read the media or was is that you can't write without having read access.

 

Might be wrong, but as far I've tested thru C# you can't read disk without write access with NTFS, so, using:

var ptr = CreateFile(
                    String.Format("\\\\.\\{0}:", volume),
                    FileAccess.ReadWrite,
                    FileShare.ReadWrite,
                    IntPtr.Zero,
                    FileMode.Open,
                    0,
                    IntPtr.Zero
);
handle = new SafeFileHandle(ptr, true);
if (handle.IsInvalid) Marshal.ThrowExceptionForHR(Marshal.GetHRForLastWin32Error());

and passing that to DllImport kernel32.dll which well is only way I found it working from one StackOverflow post.

Side through of checksum, you could checksum on-fly while writing media or when any read is occurred (one of the features I just built-in to my project). The writing verify's written and when ever for example WinPE(base application working offline/online) at media reads data at media it has stored checksum created while writing the changes from media creator.

 

(Although, I am currently stuck with one value conversion as can't seem to find a way to convert hex decimal value string to uint or byte directly, so, above is untested until I find solution, will my idea of securing the media work as I think it would work.)

 

Also another though over checksum would be to write it in zero space after the boot sector and volume ID. There's always zeros over 1473 KBytes(as of microsoft standard) which is way more than needed to include all files individually from all volumes.

[erwan.l]

@Genetix : You can create a handle (createfile) with readonly flag only.

In some situations, you actually should (or dont have any other choice).

 

To me, what we are facing here is a bug, or lets call it an "omission"  from Microsoft.

Setting RO on an online disk reports success (thru the API or thru any existing MS or non MS tool).

Actually, one can see this updated status thru diskpart.

 

Still, the (volume) filesystem seems to ignore this new status (RO) and accepts changes to the filesystem (as if the disk was still RW) with one dramatic effect : these changes will be permanently lost!

 

The windows API should either trigger an error when performing such an action on an online disk OR should update the volume(s) (like DiskMgr does).

 

One interesting experience could be to disable the filesystem cache (possible? fsutil?) to see what happens.

We might then actually see some wierd behavior from windows  : warning, error, freeze, crash ...

 

Sidenote : if you perform this same action thru diskpart (RO on an online disk), you will see that all volumes for that disk will disappear/reappear (explorer will close it it is opened at that time on a volume depending on that disk) : so I would not be surprised if the good guys at MS did that same trick/workaround (offline/online)...

Would have been simpler/healthier to simply deny RO on an online disk (or at least document it).

[erwan.l]

[HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Policies\Explorer]

"nodrives"=dword:A0
"NoViewOnDrive"=dword:A0

 

A few days ago I was fiddling with Clonedisk (on a Windows 7SP1) and noticed that it cannot save to these "Explorerdriveletterless" volumes and that it triggers a message *like* "A restriction is in effect ...".

 

Wonko

 

That message is indeed the result of windows policies.

In your case, specific policies are being applied at the volume level.

 

Thus you may be able to circumvent by mouting a volume as folder on a drive where the policy is not applied or by simply mouting it as another drive letter.

[alacran]

AFAIR in old times when partitioning and formating from dos a HD, then you had to reboot, to start installing lets say Win98, so what I mean is changes are made but do not seen untill disk goes offline and later online (as when reboot).

 

alacran

[genetix]

@Genetix : You can create a handle (createfile) with readonly flag only.

In some situations, you actually should (or dont have any other choice).

 

To me, what we are facing here is a bug, or lets call it an "omission"  from Microsoft.

Setting RO on an online disk reports success (thru the API or thru any existing MS or non MS tool).

Actually, one can see this updated status thru diskpart.

 

Still, the (volume) filesystem seems to ignore this new status (RO) and accepts changes to the filesystem (as if the disk was still RW) with one dramatic effect : these changes will be permanently lost!

 

Do you mean that yeah changes are stored until flushed to media? or that the changes are not stored when disk connection is closed, if flushed to media before closing handle?

 

Because I don't see having access to write with write unless function closed and then another function is switched to RW status at "object". Also yep, reading seems to be possible, but for some reason it's easy to get data corruption with RO(FileAccess.ReadOnly), if try'd to save the data.

 

The windows API should either trigger an error when performing such an action on an online disk OR should update the volume(s) (like DiskMgr does).

 

One interesting experience could be to disable the filesystem cache (possible? fsutil?) to see what happens.

We might then actually see some wierd behavior from windows  : warning, error, freeze, crash ...

 

Sidenote : if you perform this same action thru diskpart (RO on an online disk), you will see that all volumes for that disk will disappear/reappear (explorer will close it it is opened at that time on a volume depending on that disk) : so I would not be surprised if the good guys at MS did that same trick/workaround (offline/online)...

Would have been simpler/healthier to simply deny RO on an online disk (or at least document it).

 

Isn't hardware cache like hardware based? I mean doesn't the NCQ happen with kind of "MySQL fast search words"-style and cached data itself is device ## MB cache which gets located at device buffer, if RO is specified by software "cache" cannot be flushed back to logical partition as driver is still at ReadOnly opened state? or am I imaging things?

 

for>sidenote: diskpart will take RW status no matter does it say it's RO at online disk. That's documented by MS was reading it few days/week ago somewhere while reading about gaining 'low level raw access' to device with C#.

 

It might be possible to flush if stored and fileaccess is closed with RO, if data is smaller than cache is I think with diskpart style "open next tool to do my dirty work" as in this case diskpart uses close & reopen explorer.exe.

 

AFAIR in old times when partitioning and formating from dos a HD, then you had to reboot, to start installing lets say Win98, so what I mean is changes are made but do not seen untill disk goes offline and later online (as when reboot).

 

alacran

 

 

heh, damn what would I pay what would I pay to get back to 80's or 90's when operating systems and programs didn't have a clue where they were at or what was attached unless you actually teach'd them a bit.

 

heck I would be just fine with F1 Grand Prix 2 24x floppy edition and MS-DOS with QEMM and deskview all day long. I mean instead of what we have today. Hmm, come to think of it after that comment I might have to replay SSI AD&D 1989 pool of radiance/darkness series, hehehe.

[genetix]

Might of gone a bit off topic over my convo there. Sorry about that.

 

Then again noticed this topic is under WinFE as in collecting information with Windows based tool. Which in case would never happen as data would be a clean RAW dump from linux with minimal set of disk drivers or layers to save the raw data imahe and after data would be extracted to separate information with cache dumped as also raw data image and access to media is in this case meaningless as it's only a matched copy of original.

 

Don't understand why should the access to media be RO while the whole image could be readonly as it should never be writtten unless it's written to new hardware based temporal media and conversation should be about how to close the ID's and disk signatures to replicate/clone the original media as exact media to restore functionality of original for forensic.

[Wonko the Sane]

That message is indeed the result of windows policies.

In your case, specific policies are being applied at the volume level.

 

Thus you may be able to circumvent by mouting a volume as folder on a drive where the policy is not applied or by simply mouting it as another drive letter.

I know where it comes from , and I can re-enable the drive letter just fine, if needed, or - as you say - circumvent/workaround the issue by mounting the volume *somewhere* else, what I was trying to say is that the "error" message comes out as soon as I try to save from Clonedisk, which means two things:

1) as said somehow you must be re-using *something* in common with Explorer (7-zip does NOT use these and can access those volume(s) just fine and - as an example - Windows 7 backup "sees" those drive letters just fine, no matter if those Registry entries are set or not, and this is exactly the reason why they are so handy)

2) a "common" user in this same situation may be scared by the error (besides not being able to save to a given volume), this same error doesn't happen in Explorer, that simply fails to list the involved drive letters.

 

I was suggesting (if possible at all ) that the Clonedisk should use a somehow more "direct" method and ignore these policies, that are actually (or should be) ONLY related to Explorer:
HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Policies\Explorer

or however at least not cause the error popup.

 

Wonko

[erwan.l]

I know where it comes from , and I can re-enable the drive letter just fine, if needed, or - as you say - circumvent/workaround the issue by mounting the volume *somewhere* else, what I was trying to say is that the "error" message comes out as soon as I try to save from Clonedisk, which means two things:

1) as said somehow you must be re-using *something* in common with Explorer (7-zip does NOT use these and can access those volume(s) just fine and - as an example - Windows 7 backup "sees" those drive letters just fine, no matter if those Registry entries are set or not, and this is exactly the reason why they are so handy)

2) a "common" user in this same situation may be scared by the error (besides not being able to save to a given volume), this same error doesn't happen in Explorer, that simply fails to list the involved drive letters.

 

I was suggesting (if possible at all ) that the Clonedisk should use a somehow more "direct" method and ignore these policies, that are actually (or should be) ONLY related to Explorer:
HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Policies\Explorer

or however at least not cause the error popup.

 

Wonko

 

I have added these keys to my registry but I now need to reboot (and feel lazy right now).

This is on my todo list of things on clonedisk (which is getting long thee days with PartMgr, DiskMgr and VolumeMgr feedback).

[Wonko the Sane]

I have added these keys to my registry but I now need to reboot (and feel lazy right now).
This is on my todo list of things on clonedisk (which is getting long thee days with PartMgr, DiskMgr and VolumeMgr feedback).

Don't worry it is a low-low priority minor thing.


Wonko