61 replies to this topic

[sebus]

So doing 0x490 via API seems to work, but hex edit (even external) does not

 

Only Disk ID can be done with diskpart ( unique disk id= YOURIDHERE)

[erwan.l]

So doing 0x490 via API seems to work, but hex edit (even external) does not

 

So both can be done with diskpart (set id and unique disk id)

 

Correct.

IMHO, it is never a good idea to hex edit a disk/partition.

[erwan.l]

So back to the original question i.e "How does Window get GUID of disk in MountedDevices", looks like we are getting near a final answer :

 

-For MBR disks, windows will attempt to match the GUID in mountedevices to the disk signature :

If yes, the volumeguid will be inherited from the registry.

If not, a new volumeguid will be created.

 

-For GPT disks, windows will attempt to match the GUID in mountedevices to the disk signature AND to the partitionid : 

If yes, the volumeguid will be inherited from the registry.

If not, a new volumeguid will be created.

The partitionid happens to be an exact copy of the volumeguid but we shall not confuse a windows volume and a disk partition.

 

Left to decide : the importance of the partition offset.

 

Next, it could be interesting to discuss practical cases as of why would one want to reuse a volumeguid.

[Wonko the Sane]

Correct.

IMHO, it is never a good idea to hex edit a disk/partition.

Which is good , as I see it instead as the best possible way to deal with issues (it is good to have different point of views), instead of relying on API's that may (or may not) do exactly what is needed.

 

At least on MBR disks (for the moment) I can boot in grub4dos and fix a disk in the time your fastest PE will still be at the very beginning of the "spinning circle" .

 

The question still stands.

GPT disks have replicated data, the main one at the beginning and the copy at the very end.

 

I believe that Windows checks both and probably "synchronizes" them in case of differences (but I may be wrong).

 

So it would be nice if you could do a test with Clonedisk (via API or whatever) and compare the disk "before and after", posting the results, so that we - poor users of hex editors and other low-level tools can replicate with our inferior means.

 

About MBR, you still have it IMHO wrong (or possibly wrote it down in an unclear way), as I see it:

The Registry entry (the Volume GUID) AND the dosdevice letter assigned in case a drive letter is assigned to it will BOTH have the disk data as "disk signature+offset".

The "search" is done the other way round, i.e. when a disk is booted/connected etc. Windows will:

1) Read Disk Signature
2) Read offset to the volume(s)
3) look in the Registry for the data in all the keys in HKEY_LOCAL_MACHINE\SYSTEM\MountedDevices matching the Disk Signature and offset
4) if a GUID REG_Binary value exists in the Registry with corresponding data use it, otherwise create a brand new (V1) GUID/UUID
5) if a \\DosDevices\<driveletter>: REG_Binary value exists in the Registry with corresponding data, assign it to the volume

 

Wonko

[sebus]

Next, it could be interesting to discuss practical cases as of why would one want to reuse a volumeguid.

 

I need to do shared vhdx to vhds (set) conversion & I have the idea of MS needing VM to be powered off for this to happen.

And MORE important, I want to keep existing Shadow Copies of the source disk (hence I need the destination to take its identity in FULL

[erwan.l]

I need to do shared vhdx to vhds (set) conversion & I have the idea of MS needing VM to be powered off for this to happen.

And MORE important, I want to keep existing Shadow Copies of the source disk (hence I need the destination to take its identity in FULL

 

Crystal clear.

You want to clone a disk or volume (as here you could use volume shadow copies) and you want to keep you volume shadow copies on the cloned disk/volume.

 

As i believe (but that needs to be tested/asserted) that volume shadow copies are based on the volume guid, you need to keep the same volume guid before/after.

[erwan.l]

Which is good , as I see it instead as the best possible way to deal with issues (it is good to have different point of views), instead of relying on API's that may (or may not) do exactly what is needed.

 

At least on MBR disks (for the moment) I can boot in grub4dos and fix a disk in the time your fastest PE will still be at the very beginning of the "spinning circle" .

 

The question still stands.

GPT disks have replicated data, the main one at the beginning and the copy at the very end.

 

I believe that Windows checks both and probably "synchronizes" them in case of differences (but I may be wrong).

 

So it would be nice if you could do a test with Clonedisk (via API or whatever) and compare the disk "before and after", posting the results, so that we - poor users of hex editors and other low-level tools can replicate with our inferior means.

 

About MBR, you still have it IMHO wrong (or possibly wrote it down in an unclear way), as I see it:

The Registry entry (the Volume GUID) AND the dosdevice letter assigned in case a drive letter is assigned to it will BOTH have the disk data as "disk signature+offset".

The "search" is done the other way round, i.e. when a disk is booted/connected etc. Windows will:

1) Read Disk Signature
2) Read offset to the volume(s)
3) look in the Registry for the data in all the keys in HKEY_LOCAL_MACHINE\SYSTEM\MountedDevices matching the Disk Signature and offset
4) if a GUID REG_Binary value exists in the Registry with corresponding data use it, otherwise create a brand new (V1) GUID/UUID
5) if a \\DosDevices\<driveletter>: REG_Binary value exists in the Registry with corresponding data, assign it to the volume

 

Wonko

 

Absolutely, there are cases where hex editing is defo the best way.

For that particular discussion, I had indeed assumed that all was taking place under windows.

There, IHMO, i would recommend to use windows API as windows easily gets "upset" when one fiddles with bytes.

 

About the backup sector, i am pretty convinced that windows API's do take care of that (but will eventually test it).

 

About MBT, you are definitely correct, as always ! You are wording this in a much better way than I do

[sebus]

Crystal clear.

You want to clone a disk or volume (as here you could use volume shadow copies) and you want to keep you volume shadow copies on the cloned disk/volume.

 

As i believe (but that needs to be tested/asserted) that volume shadow copies are based on the volume guid, you need to keep the same volume guid before/after.

No, the shadow copies are on its OWN dedicated separate volume!

 

Shadow copies NEED DiskID, Volume offset (as seen on MountedDevices) the same, that is why dis-to-disk works fine!

 

I think I have explained it here a couple of times (maybe not too well)

 

But with CD doing the edit I got destruction

 

 

after which the disk is dead!

 

[erwan.l]

No, the shadow copies are on its OWN dedicated separate volume!

 

But with CD doing the edit I got destruction

 

 

after which the disk is dead!

 

 

should be a guid in the form of : {9EA1EAB7-1E78-11E9-8341-94DE80C2A41E}.

with brackets.

 

hope you are not doing this on production critical data : i would be sorry to hear that

or as a minimum you did backup datas.

 

looks like that even with a wrong input guid, clonedisk will anyway proceed.

 

fixing that.

 

in all cases, before considering using clonedisk on production datas, make triple sure that you have tested that before in a dev environement.

[sebus]

Well, it is my "production copy" that I keep destroying daily. As to backups? Usually keep 3 or 4 just to be sure...

 

"Fixed" it with diskpart, changed Partition GUID, so the disk is the "same" as source

 

Whole process is now quite possible inside CD with help of vscsc.exe (snapshot, copy snapshot volume to a volume, adjust DiskID & Volume GUID)

 

 

But Shadow Copies disappeared!

 

 

And if I reconnect old source disk (of course on its own, removing the destination "clone") Shadow Copies are back!

 

 

Yet BOTH disks now look the same! (inc short Volume serial number)

 

So WHAT are Shadow Copies recognize about the disk to show themselves or not?

[erwan.l]

Well, it is my "production copy" that I keep destroying daily. As too backups? Usually keep 3 or 4 just to be sure...

 

Pfiew... you got me worried there for a mn

I too work in IT field and would litteraly shit in my pants if I was to lose a server while testing stuff !

Rebooting or shuting down a server remotely is one (quickly recoverable) thing, losing a disk is another game ...

 

Latest clonedisk version now clearly says what to input and will NOT proceed if GUID is wrong.

 

[erwan.l]

Well, it is my "production copy" that I keep destroying daily. As to backups? Usually keep 3 or 4 just to be sure...

 

"Fixed" it with diskpart, changed Partition GUID, so the disk is the "same" as source

 

Yet Shadow Copies disappeared!

 

 

Ok then it could lead a second thread whose title could be "how are volume shadow copies related to a volume?".

 

Now we know that keeping the same disk signature, same partition id and therefore same volume guid is not enough.

Surprising thus as cloning your volume + duplicating disk signature/partition id does not leave much information behind (if any).

 

@Wonko : any idea?

 

When you did your disk to disk clone (but is then a byte to byte copy of the entire disk) : did you keep your shadow copy volumes?

[sebus]

Yes, disk clone does make the Shadow Copies still alive!

[erwan.l]

Yes, disk clone does make the Shadow Copies still alive!

 

 

Interesting.

 

Basics (over simplifying) : a disk is made of a "header/table" + one or more partitions (or volumes from a windows p.o.v).

 

Here, you have cloned your volume and changed your header (disk signature + partition id) to mimic the original disk.

I also assume that partiton offset remains the same between before/after.

 

So we are either missing staff somewhere in the header or this backup sector in the volume we discussed earlier is to be looked more in depth.

[sebus]

I will make a separate thread for that...

 

This one is pretty much answered.

 

GUID in MountedDevices is just time based UUID that is calculated from (for GPT) DiskID & Volume offset (ID)

 

Once two disks have each valuse identical to one another, one can mount each disk separately &  Windows will always (at least for the purpose of drive letters) "recognize them as the SAME disk & assign the same letter

 

Thanks you for this educational two day session and great software that does it in one go

 

sebus

[erwan.l]

Was definitely a nice and instructive session.

 

I am sure it will be of use in the future.

 

This partitionid GPT attribute is also a nice discovery (to me) and it looks like this is one attribute you cannot change in diskpart (as opposed to disk signature which is called disk id in dispart (?) ).

 

Cheers,

Erwan

[erwan.l]

@wonko : I changed the partitionid with CloneDisk and I can confirm that both the main sector (sector 2 in my case) and the backup sector (at the end of the partition are modified. 

 

Also, I was wrong in saying it is offset 0x490 : it is 0x410, right after the partition type.

 

parition information description here.

 

[sebus]

Well, I think offset is not fixed. On SSD on Win 10 1803 is at 0x510

 

[erwan.l]

I realise that that there may be still confusion as of what we can do with diskpart (or cannot do).

 

Changing the disk signature is obviously well known (uniqueid disk ID=[NEW SIGNATURE]).

Note the nice confusion again between diskid and disk signature

 

I thought for a while that "set id" diskpart command was to modify the partitionid but reading this MS doc, it appears not : here.

Confusion is easy and both are GUID's.

Not to mention that "set id" refers actually to "partiton type field"...

The Diskpart Set ID command changes the partition type field for the partition with focus.

So for now, it may be that the only way to change the partitionid is thru clonedisk.

 

Also, partitionid is not to be confused with "partition type id".

 

All these different wordings... what is we just stop using the generic word ID ?

There are more words in english than many other langages, surely MS can do better.

[karyonix]

In GPT, size of each partition entry is 0x80 bytes. Partition entries start at LBA 2.
In 512-byte-per-sector GPT drive, first partition entry starts at absolute-byte-offset 0x400, next at 0x480, then 0x500 ...
Partition GUIDs are at 0x410, 0x490, 0x510 ...

If a GUID is like this in hex editor displaying byte value 00 11 22 33 44 55 66 77 88 99 AA BB CC DD EE FF, it will look like this in bracket form (deleted {33221100-5544-7766-9988-ffeeddccbbaa}) {33221100-5544-7766-8899-aabbccddeeff}.

As I see on Windows 10 1809, simple volume in GPT disk simply has the same GUID as the partition it oocupies.

When a GPT partition has drive letter D, registry has this value in HKLM\SYSTEM\MountedDevices key.
\DosDevices\D: = ( 8-byte "DMIO:ID:", 16-byte partition GUID )
Byte order of partition GUID in registry data is the same as in partition table.

when drive letter is removed.
#{some_guid_number_1} = ( 8-byte "DMIO:ID:", 16-byte partition GUID )

after drive letter D is assigned, then removed again.
#{some_guid_number_2} = ( 8-byte "DMIO:ID:", 16-byte partition GUID )

#{some_guid_number_2} is different from #{some_guid_number_1}.
It seems it does not matter what number it is in value name. It just has to be different from other registry value names in the key.
The "data" remains the same for the same volume.

I think Disk GUID and offset does not affect drive letter assignment for simple volume in GPT disk.
I changed (in CloneDisk) partition GUID of partition "Test1" in an attached VHD to be the same as partition GUID of partition "Test2" in a detached VHD, its drive letter changed immediately to the letter previously assigned to the partition "Test2". I did not edit Disk GUID in this test.

It would be nice to have partition GUID visible in partition table editor in CloneDisk when partition table is GPT.

[erwan.l]

In GPT, size of each partition entry is 0x80 bytes. Partition entries start at LBA 2.
In 512-byte-per-sector GPT drive, first partition entry starts at absolute-byte-offset 0x400, next at 0x480, then 0x500 ...
Partition GUIDs are at 0x410, 0x490, 0x510 ...

If a GUID is like this in hex editor displaying byte value 00 11 22 33 44 55 66 77 88 99 AA BB CC DD EE FF, it will look like this in bracket form {33221100-5544-7766-9988-ffeeddccbbaa}.

As I see on Windows 10 1809, simple volume in GPT disk simply has the same GUID as the partition it oocupies.

When a GPT partition has drive letter D, registry has this value in HKLM\SYSTEM\MountedDevices key.
\DosDevices\D: = ( 8-byte "DMIO:ID:", 16-byte partition GUID )
Byte order of partition GUID in registry data is the same as in partition table.

when drive letter is removed.
#{some_guid_number_1} = ( 8-byte "DMIO:ID:", 16-byte partition GUID )

after drive letter D is assigned, then removed again.
#{some_guid_number_2} = ( 8-byte "DMIO:ID:", 16-byte partition GUID )

#{some_guid_number_2} is different from #{some_guid_number_1}.
It seems it does not matter what number it is in value name. It just has to be different from other registry value names in the key.
The "data" remains the same for the same volume.

I think Disk GUID and offset does not affect drive letter assignment for simple volume in GPT disk.
I changed (in CloneDisk) partition GUID of partition "Test1" in an attached VHD to be the same as partition GUID of partition "Test2" in a detached VHD, its drive letter changed immediately to the letter previously assigned to the partition "Test2". I did not edit Disk GUID in this test.

It would be nice to have partition GUID visible in partition table editor in CloneDisk when partition table is GPT.

 

In CloneDisk main window, for a GPT disk :

-if you double click on a disk, you then get a window where you get the disk id (guid form)

-if you double click on a partition, you then get a window where you get the partition id (guid form) and the volume id (guid form) - both  id's should be identical

 

Still in CloneDisk, if you open the volumes window :

-for all disks (mbr or gpt), you should get the volume id (guid form) which, for gpt disks, should match the (gpt) partition id

 

I'll see if i can rework the partition editor window.

[Wonko the Sane]

I don' t get it.

 

Besides the usual mishmash between disk drive partition and volume, reading the last two posts is sounds like the volume GUID (i.e. what you see in mountvol) is NOT anymore a UUID V1 dynamically generated at "connection" on GPT (it is on MBR)

 

Is this the case or we are talking of different things?

 

The partition ID (actually partition type) is NOT the Volume GUID.

 

Wonko

[erwan.l]

I don' t get it.

 

Besides the usual mishmash between fisk drive partition and volume, reading the last two posts is sounds like the volume GUID (i.e. what you see in mountvol) is NOT anymore a UUID V1 dynamically generated at "connection" on GPT (it is on MBR)

 

Is this the case or we are talking of different things?

 

The partition ID (actually partition type) is NOT the Volume GUID.

 

Wonko

 

What MS calls "id" is actually/indeed the "partition type" (guid form when about GPT disks).

Diskpart command would then be "set id".

This can easily lead one to confuse this "id" with a "partition id"

 

What we are talking here (wrongly referred as "partition id") is actually the "unique partition guid".

This later, based on my experience, matches the "volume guid". See screenshot in post #42 to illustrate this.

Diskpart does NOT handle this field (to my knowledge).

 

Issue is the use and abuse of the "id" word...

 

For reference : https://en.wikipedia...Partition_Table - look at the 128bytes partition entry description.

[Wonko the Sane]

What MS calls "id" is actually/indeed the "partition type" (guid form when about GPT disks).
Diskpart command would then be "set id".
This can easily lead one to confuse this "id" with a "partition id"

What we are talking here (wrongly referred as "partition id") is actually the "unique partition guid".
This later, based on my experience, matches the "volume guid". See screenshot in post #42 to illustrate this.
Diskpart does NOT handle this field (to my knowledge).

Issue is the use and abuse of the "id" word...

For reference : https://en.wikipedia...Partition_Table - look at the 128bytes partition entry description.

We are still talking two different languages/saying different things.

With reference to the image in post #42

What you call "PartitionType" is the partition type.
What you call "PartitionID" is the actual UUID of the volume that in GPT is written to disk at the time of formatting, UNLIKE what happens on MBR where that value is NOT written to disk) and in the example has the value 9EA1EAB7-1E78-11E9-8341-94DE80C2A41F
What you call "VolumeGUID" is the actual UUID of the volume (that evidently mountvol and similar in the case of GPT read from disk) and that in the example has as well the value 9EA1EAB7-1E78-11E9-8341-94DE80C2A41F

Now, the point I was trying to make is that the PartitionID and the Volume ID are stiil UUID v1:

C:\appoggio\UUID>uuid -d 9EA1EAB7-1E78-11E9-8341-94DE80C2A41F
encode: STR:     9ea1eab7-1e78-11e9-8341-94de80c2a41f
        SIV:     210858743720271515749021598605463692319
decode: variant: DCE 1.1, ISO/IEC 11578:1996
        version: 1 (time and node based)
        content: time:  2019-01-22 19:05:01.783109.5 UTC
                 clock: 833 (usually random)
                 node:  94:de:80:c2:a4:1f (global unicast)

and that that example volume/partition was formatted on the 22th of January 2019 around 19:05 (only you can confirm or deny that the 94:de:80:c2:a4:1f is actually the MAC address of that machine or not).

So the news (if any) should IMHO be that:
1) on MBR the GUID of a volume (I will repeat, the one that is shown in Mountvol) is:
a. dependent on the time the volume was connected to a given machine(regardless of the time it was created/formatted)
and
b. stored (in what is a semi-volatile manner) in the Registry ONLY
whilst:
2) on GPT the GUID of a volume (I will repeat, the one that is shown in Mountvol) is:
a. dependent on the time the volume was created/formatted
and
b. stored in a "persistent" manner in the actual partition entry in the GPT table on the disk (besides in the Registry) and is what you call "Partition Id" (and that is actually read by Mountvol and thus becomes "Volume Id").
OR
b. stored in a "persistent" manner in the actual partition entry in the GPT table on the disk (besides in the Registry) and is what you call "Partition Id" (and that is not actually read by Mountvol or it is read only partially and thus becomes "Volume Id" while still reflecting the mounting time or whatever else and not anymore the formatting date/time according to karyonix's post[1]).

All of the above would need some experiment/tests to be verified.

As a side note, and for future memory, I will remind that on GPT ONLY "primary" partitions exist and that - due to this - the terms partition and volume are converging.
There is likely still the small issue of the backup of the first sector of the $Boot file - the 16 sectors bootsector - which remains outside the volume (appended to the very end of the volume) but still inside the partition extents.


Wonko

[1] I confess it is hard to me to follow the meta-descriptions he posted, if he actually posted the actual UUID's he found (in the Registry and on disk) I would be more convinced to have understood the post.

[erwan.l]

I believe we now all talk about the same items only wording differs.

 

Lets definitely dump the "partition type" as this one field is of no interest (in that discussion)  and only keeps introducting confusion.

Lets just store somewhere in the back of our mind that there is often a confusion between "partition type" and "unique partition GUID" since they are both GUID's and since MS simply refers to the "partition type" as "ID" ...

 

Lets focus on "unique partition GUID" and "volume GUID".

I believe that the GUID reported by mountvol and the GUID found in a GPT partition entry are 2 different "physical" things in a way that they are stored in different places : it is stored in the registry for a volume and it is stored on the hard drive itself (in the partition entry) for a GPT partition.

Furthermore, although it does not assert the above statement:

-i use IOCTL_DISK_GET_PARTITION_INFO_EX to retrieve the GPT "unique partition GUID"

-I use getvolumenameforvolumemountpointa to retrieve the "volume GUID" 

 

In a GPT scenario, however (i.e as opposed to MBR), windows will take one extra step : it will systematically "sync" these 2 GUID's.

 

Disgressing 1 mn (shoot me for that...): we could also consider that the "VolSerialNum" (which is stored in the boot sector) could be seen as the counterpart of the GPT "unique partition GUID" ?

Although Windows does not seem to care much about this field (or not?), which furthermore may be FS dependent.

On MBR, "Disk Signature" AND the "partition offset" seems to be the ones which influence the "volume GUID".

Thus I dont want to disgress too much and deviate from the main topic : GPT and  "volume GUID" vs "unique partition GUID". 

 

As of the ever struggle between volume and partition ... I may be wrong but I define :

-a partition as a completely OS independent part of a disk (with a starting offset and a length), defined/hardcoded on the first sectors of a hard drive (which I would call a "partition table").

-a volume as a partition mounted at the OS level ... somegthing "logical" which only exists during the OS life defined in fstab under linux, in the registry ("mounteddevices") under windows, etc.

 

Last but not least after this long post, I dont get what you mean by "Now, the point I was trying to make is that the PartitionID and the Volume ID are stiil UUID v1"