13 replies to this topic

[jgoggan]

I must be missing something about how to use rawcopy properly.  Basically, I want to create an image of  the contents of a disk -- and by that I mean a single drive letter, not the entire physical disk.

 

The other day, I created a 185GB image file using rawcopy -- and I can't get it to open in ImDisk properly -- it always mounts it and then Windows thinks it is unformatted.  So, I decided to do some tests using various methods with a smaller amount of data -- here's the details and my issues:

 

For example, let's say I have a 500GB physical disk that is partitioned as two partitions that are both NTFS formatted -- and they currently get mounted in Windows as C: and D:.   C: is the ~488GB system drive and D: is ~12GB of data.  What I'd like to do is make an image of D:, mount that image with ImDisk, and then I can work with it as I please without fear of screwing up the original.

 

I've actually done this just fine using Erwan's CloneDisk -- and also using the context-sensitive "save disk contents as image file" option of ImDisk.  Both of those created an image file that I can mount just fine using ImDisk's context-sensitive "Mount as ImDisk Virtual Disk."

 

However, everything I have tried using rawcopy has failed to produce an image file that I can properly mount with ImDisk.  All of my tests do exactly the same thing as my original 185GB image: Windows thinks it is unformatted.  I assume this is all related to offsets or something?  But I would expect ImDisk to auto-detect those offsets properly from rawcopy-created image files.

 

In my testing, I have tried numerous options with rawcopy (based on reading through the threads here, I tried different things).  Here are the file sizes that I am getting using different options:

 

CloneDisk created image: 13,736,217,088 bytes (and mounts fine with ImDisk)

ImDisk GUI created image: 13,736,217,600 bytes (and mounts fine with ImDisk)

 

Rawcopy attempts (done using "rawcopy \\.\D: blah.img" plus the following options):

13,736,214,528 bytes (using "-m")

13,735,821,312 bytes (using "-md")

13,736,214,528 bytes (using no switches)

 

Again, I can't get any of the rawcopy images to mount with ImDisk properly.  All of them are smaller than what I get from CloneDisk or the ImDisk GUI.

 

What am I missing here?  How can I use rawcopy to make an image of disk contents and have it work like the images from the ImDisk GUI work?

 

Thanks!

 

 - John

 

 

[jgoggan]

Oh, and just to add a bit more info...  WinImage will open ALL of the rawcopy-created image files no problem -- they just pop right open in WinImage.

 

 - John

[Wonko the Sane]

I would say that Olof (and thus rawcopy and consequently you) are falling in this issue here :

http://reboot.pro/to...ated-with-dsfo/

 

http://reboot.pro/to...-dsfo/?p=166421

 

And summed up here:

http://reboot.pro/to...-dsfo/?p=166592

 

The "right size" is the Clonedisk one:

13,736,217,088 

This is also correct (IMDISK):

13,736,217,600

 (as it includes the backup bootsector which is on hard disk within partition BUT outside volume/filesystem)

 

Very basically, the size "exposed" as \\.\LogicalDrive (or as a drive letter) is made of clusters, whilst the size (on media) is made of "real size" and this size is also explicited in the PBR.

 

The difference needs to be less than the size of a cluster, in your case:

13,736,217,088 - 13,736,214,528 = 2,560 bytes = 5 sectors

 

Basically you can add to any of the two images sized 13,736,214,528 bytes 5 sectors (blank) and everything should work again.

 

This has been "fixed" by erwan.l in clonedisk, that reads the size in the bootsector/PBR, but it is not "rawcopy" specific, it is "common" to *any* tool that uses a \\.\LogicalDrive as source for copying/cloning.

 

A "generic" (and very ugly) fix could be to "blindly" adding 15 00'ed sectors to any image, otherwise parsing the PBR  total sectors and calculate the difference of needed sectors will be required.

 

Wonko

[jgoggan]

Hmmm...  Ok, thanks for the info.  Some of that goes above me, but I get some of the gist of it, I guess.

 

It just seems "wrong" to me that it doesn't look like rawcopy can create an image of disk contents from a drive letter that ImDisk can open -- even though the ImDisk GUI can do it just fine.

 

So, as far as working around it...  I used the rawcopy -md switches when I made that original 185GB image that I'm trying to get to mount using ImDisk without having to re-image it.  With my sample files, the -md switch option had the biggest difference between that output file and the CloneDisk version (395,776 bytes -- the difference between the 13,736,217,088 bytes of the CloneDisk version and the 13,735,821,312 bytes of the "rawcopy -md" version).  Based on that, could I add 395,776 bytes (773 blank sectors) to the start of my 185GB image and have it open with ImDisk then?

 

 - John...

[Wonko the Sane]

Hmmm...  Ok, thanks for the info.  Some of that goes above me, but I get some of the gist of it, I guess.

 

It just seems "wrong" to me that it doesn't look like rawcopy can create an image of disk contents from a drive letter that ImDisk can open -- even though the ImDisk GUI can do it just fine.

 

So, as far as working around it...  I used the rawcopy -md switches when I made that original 185GB image that I'm trying to get to mount using ImDisk without having to re-image it.  With my sample files, the -md switch option had the biggest difference between that output file and the CloneDisk version (395,776 bytes -- the difference between the 13,736,217,088 bytes of the CloneDisk version and the 13,735,821,312 bytes of the "rawcopy -md" version).  Based on that, could I add 395,776 bytes (773 blank sectors) to the start of my 185GB image and have it open with ImDisk then?

 

 - John...

 

No.

 

The missing sectors in a "non -md" made image are missing at the END of the image (and can be at the most cluster_size-1, at the most 15 or possibly on a really huge or queer filesystem 127 - with 64 Kb sectors).

 

The -m option seemingly is related to a buffer size whilst the -d option is about DASD I/O (whatever it is), theoretically there shouldn't be any difference in size in the images.

 

WHERE did you learn to use those switches (and WHY exactly did you use them)?

 

Can you post the full command line you issued when making the image?

 

Maybe the -md option suffers from *another* issue (or there was however a "glitch in the matrix" when you took the image).

 

However, you won't do a "damage" to the current image if you APPEND at the END of it some 400 Kb of 00's, at the most it will be unmountable as it is now (while if you add anything in front of it it will become "unmountable for good" )

 

The first sector of such an image is, and must be, a valid bootsector/PBR, the issue talked about is about missing sectors at the end of the image.

 

What is the issue with taking a new image (with clonedisk) ?, I mean, it should be a matter of minutes.

 

Wonko

[jgoggan]

 

The -m option seemingly is related to a buffer size whilst the -d option is about DASD I/O (whatever it is), theoretically there shouldn't be any difference in size in the images.

 

WHERE did you learn to use those switches (and WHY exactly did you use them)? 

 

Um, it's in the FAQ for ImDisk. Someone asks a question about getting corrupted image files that you can't mount (which was the problem I thought I was having) and the response in the FAQ is this:

 

 

 

To make sure that filesystem drivers or filesystem filter drivers do not interfere with rawcopy, use the -d flag:

rawcopy -mld .R: C:\ramdisk.img

 

So, I used something similar.

 

 

 

Can you post the full command line you issued when making the image?

 

I used: rawcopy -md \\.\D: dump.img

 

 

 

The first sector of such an image is, and must be, a valid bootsector/PBR, the issue talked about is about missing sectors at the end of the image.

 

So why can WinImage open this just fine?  I assume there must be some missing data at the end and WinImage just hasn't hit it yet?

 

 

 

What is the issue with taking a new image (with clonedisk)   ?, I mean, it should be a matter of minutes.

 

I do not have easy access to the source disk at this time.  I was keeping the copy so that I could get to the data later when I had time -- and therefore do not have the laptop that it came from.  I can get it again, but not any time soon, unfortunately.

 

Thanks for your help.  I'll try adding some data to the end and see what I get just for grins.

 - John...

[Wonko the Sane]

So why can WinImage open this just fine?  I assume there must be some missing data at the end and WinImage just hasn't hit it yet?

Because Winimage OPENS the file (whilst IMDISK MOUNTS it).

(and rest assured that Winimage, in order to open the file, needs that is first sector is a valid PBR/bootsectorm, so the "beginning" of the image is "right" )

 

You can also try to open the file in 7-zip (it should as well open "normally"). 

 

Since WInimage opens the file, is another reason to try add the 400 Kb 00's at the end, unless the source disk was full "up to the brim" there won't be any issue about "missing or corrupted files".

 

The point I was trying to make that there may be TWO  "bugs" , one that is "outside IMDISK" (but that needs anyway a "fix" when using a drive letter or \\.\LogicalDrive as source) and that accounts for the missing few (5 in your case) sectors and another one that must be connected with either the -m, the -d or the combined -md switch (or of the absence of the -l one ).

 

Wonko

[jgoggan]

You can also try to open the file in 7-zip (it should as well open "normally"). 

 

Since WInimage opens the file, is another reason to try add the 400 Kb 00's at the end, unless the source disk was full "up to the brim" there won't be any issue about "missing or corrupted files".

 

7-zip was unable to open the -md created archive.  It did the CloneDisk and ImDisk ones just fine.

 

I will try adding extra to the end of the big archive and see what I get.  Worst case, I can just extract it all using WinImage.  It would just be nice to be able to open it in ImDisk.

 

Thanks!

 

 - John

[jgoggan]

Just as a followup, adding the extra ~400k of filler to the end of the 185GB image file seemed to work -- ImDisk mounted it just fine after that.  There was data right to the end in the image file -- so I'm assuming that some data was lost along the way?  But it is enough to get me by, at least.

 

When I get the drive again, I'll use CloneDisk or the ImDisk GUI to make it instead.  Still think it is weird that RawCopy can't make a usable image of this though.  

 

Thanks for your help!

 

 - John...

[Wonko the Sane]

Winimage, if you check it's option, has one about "truncated images", which if I recall correctly, is set as "default" at first install, I guess that Gilles at the time considered that to be a feature to save space.

 

7-zip most probably does "the right thing" and checks the sectors listed in the PBR.

 

We still need a word of two by Olof about the "strange" behaviour of the rawcopy tool.

 

Could it be that - if the source was actually "fulll up to the brim" - that those missing sectors were "in use" by *something* and then it was a "one time" issue connected with the missing "-l" switch?

 

Wonko 

[jgoggan]

Winimage, if you check it's option, has one about "truncated images", which if I recall correctly, is set as "default" at first install, I guess that Gilles at the time considered that to be a feature to save space.

 

There is a "truncate unused image part" option -- but I do not have it selected.

 

We still need a word of two by Olof about the "strange" behaviour of the rawcopy tool.

 

Agreed.  

 

Could it be that - if the source was actually "fulll up to the brim" - that those missing sectors were "in use" by *something* and then it was a "one time" issue connected with the missing "-l" switch?

 

I think you have it backwards.  The DEFAULT behavior is to lock the entire partition so that nothing else can use it -- I verified that the drive letter was indeed unavailable during the rawcopy operation.  The "-l" option actually does the opposite: it says that it tries to do the imaging with the drive UNLOCKED and then warns that this may damage the data.

 

Right?  Or am I looking at that wrong?  I was under the impression that you should NOT use the "-l" option unless you absolutely could not avoid it (i.e. you HAD to image a drive WHILE something else was accessing it).

 

 - John...

[Wonko the Sane]

There is a "truncate unused image part" option -- but I do not have it selected.

Yep, but that is a "save option".

If Winimage can (optionally) save an image as "truncated", it logically can open BOTH "full" and "truncated" images. 

 

I think you have it backwards.  The DEFAULT behavior is to lock the entire partition so that nothing else can use it -- I verified that the drive letter was indeed unavailable during the rawcopy operation.  The "-l" option actually does the opposite: it says that it tries to do the imaging with the drive UNLOCKED and then warns that this may damage the data.

 

Right?  Or am I looking at that wrong?  I was under the impression that you should NOT use the "-l" option unless you absolutely could not avoid it (i.e. you HAD to image a drive WHILE something else was accessing it).

I don' t know

 

Wonko

[Olof Lagerkvist]

Winimage, if you check it's option, has one about "truncated images", which if I recall correctly, is set as "default" at first install, I guess that Gilles at the time considered that to be a feature to save space.

 

7-zip most probably does "the right thing" and checks the sectors listed in the PBR.

 

We still need a word of two by Olof about the "strange" behaviour of the rawcopy tool.

 

It looks like there could be something wrong with rawcopy in this case. The whole idea with adding -d switch to rawcopy was that it should do exactly the same thing as the right-click "save as image file" option with ImDisk. No more no less. Apparently it does not. I will take a look and see what I can find out.

 

I think you have it backwards.  The DEFAULT behavior is to lock the entire partition so that nothing else can use it -- I verified that the drive letter was indeed unavailable during the rawcopy operation.  The "-l" option actually does the opposite: it says that it tries to do the imaging with the drive UNLOCKED and then warns that this may damage the data.

 

Right?  Or am I looking at that wrong?  I was under the impression that you should NOT use the "-l" option unless you absolutely could not avoid it (i.e. you HAD to image a drive WHILE something else was accessing it).

 

Correct and correct.

[Olof Lagerkvist]

After having looked at the source code, I came to this conclusion:

 

Right-click, Save disk contents as image file with ImDisk, should be the same as -d -r -m:512K flags with rawcopy. The only other difference is that the ImDisk save image function checks if saved image file became larger than reported disk volume size. In that case it truncates the image file to the disk volume size, while rawcopy never truncates anything. But this should not be the problem in your case, since your image file saved with rawcopy becomes smaller than the one saved with ImDisk save image file feature. More probably, either the specific buffer size passed with -m switch or the non-buffered read with -r switch are related to the difference in results in this case.