No replies to this topic

[caspertone2003]

Hi All,

 

After spending 20 hours searching for a tool I have given up of finding by myself the solution.

 

So, I come to you all subject matter experts because I see that the learning curve is so step and I am unable to devote another 100 hours ... And probably you have a solution for me in half a second...

 

I know I am asking for free consultancy ... In exchange I can provide a joke. This man´s car engine stopped and he called the mec. After half a second the mec hits with a hammer in a point and the issue is solved. Car owner ask how much and the mec asks for 2000 US$. So, car owner complains about how expensive for half a second job. Mec answers... half second plus hundred hours study and hundred hours experience... ;-)

 

Situation is:

 

Windows recovery produced a booting windows recovery USB, 32GBs of contents in a 64GB USB stick. Windows is 10pro x64 22H2; system boots from UEFI secured boot.

 

I would like to back up the recovery USB, so that I can recreate it any moment I need it but without having to phisicallyt keep the USB (I am always losing things). Of course I wish to have the smallest backup, not a bit-by-bit image of the USB. And in a windows environment (or live boot with GUI), no linux knowledge on my side...

 

My initial idea was to obtain an ISO file that I can later use to recreate the USB, such as using Rufus. I believed both steps, backup and recreation to be easy and simple. As per today I am unsure how to back up such USB and not sure anymore that a tool is able to recreate the USB.

 

Trying to give some further info, I happily started locating ImgBurn ability to get such ISO for MBR and, for some Windows builds, even UEFI. But... my USB has not the needed file (efisys.bin)... So I could not event test the second step (Rufus writing of the ISO making a bootable unit....). Several times similar questions when unanswered in ImgBurn forum.... So without further assistance I cannot progress with that tool...

 

I read some guy with a solution using mkisofs in linux...

 

So....

 

What I am aiming, can be done?

 

If yes, which backup method/tool could I use?

 

Then, which method/tools could I use to recreate the USB from the backup copy?

 

caspertone2003

TIA!

 

(if needed I can relate the places I have reading but simply believe that it is too boring and info that

you well need...)

 

 

[Wonko the Sane]

I don't think you will like the suggestions, anyway ....

 

The only reliable and tool independent method is to make a RAW (bit by bit) image (what you can make with a so-called dd-like tool, CLI or GUI).

 

You can - after creation - compress it and store it in compressed form, so the size issue is only temporary (when you create and when you restore it).

Do the 32 GB fit in an actual 32 GB stick or a 64 GB one is needed? (I know that nowadays 64 GB sticks are becoming more common, but if you can fit the image on a smaller stick it will be easier to manage).

 

ISO is a format suitable for CD and DVD, while very likely it is possible to convert the image you have as either a "plain" or "hybrid" iso, it is (IMHO) not worth the hassle (and besides it won't "shrink" the image size in any meaningful way).

 

You could use one of the MS own formats (the ones used for installation discs/isos) i.e. the WIM one (which "could" - not necessarily "will") save some space, but - still IMHO - it is over-complicated and it has to be seen whether it offers any other advantage, or (and this actually offers some advantage) the VHD (static or better dynamic) VHD one.

 

Then, there are Commercial (and also some free or freeware) tools, but AFAIK they all use - one way or the other - some proprietary formats that may become unsupported.

 

A few like (example):

https://www.alexpage...usb-image-tool/

do use (optionally) "standard" formats such as zip or gzip for the compression.

 

There is also the possibility (it really depends on the contents of your disk/disk image) to use a truncated image which is a midway between a "full" raw image and a compressed raw one and making use of the "sparse" capabilities of NTFS (this latter is a very good approach though it has some limitations when you want to copy/move the files) and it is a bit complex, JFYI:

http://reboot.pro/to...n-many-ramdisk/

 

If I were you I would experiment with the mentioned alexpage.de tool and see if you can get a satisfactory result, as it is the simplest approach, then, if you need/want something else, we will talk of the alternative approaches.

 

Wonko

 

P.S:: In an Italian version of that joke the mechanic asks for 2022 Euro, 2 for the hammer, 20 for the call and 2000 for knowing where exactly to hit.

[steve6375]

If you just need UEFI-bootable it should be easy.

I assume that the Recovery USB drive just has one partition? If not give details of what each partition is and what it contains.

If so simply zip up all files on that FAT32 partition and keep the zip file safe somewhere.

To re-create a new USB drive, first format it as FAT32 single partition drive (under Windows, max size is 32GB or you can use a 3rd-party format tool to create a larger FAT32 partition). Then unzip the files onto it.

[caspertone2003]

First all, thanks both to Wonko and steve6375.

From my diving in the forum, both of you are real subject matter experts in this realm of booting, thank you indeed for guiding me.

 

I will provide further info in a different post answering each, but I have to correct some previous information I sent.

 

I thought that the information in the USB was about 32GB because I read that there where 25.9GB free available and I knew (and know) that the USB device is 64GB size (well, truly it is 59.08GB). But I did not realize was that Windows had reformated the USB, crating a FAT32 32GB partition letting the other 32 unassigned. So, the real situation is that the information stored is "only" 6.04GB. So, having a raw bit-by-bit image (such as a la "dd") becomes not a drama...

 

My idea of the ISO image was to forget about having to deal with the booting aspects, but I can live with it if the rebuild is not cumbersome.

 

I had had a view to AnyBurn and ImgBurn in relation to ISO capture, and to Passmark ImageUSB and Alexander`s USB Image Tool on another kind of backups - not having an issue with a propietary format if I can have the tool to properly recreate the bootable USB.

 

ImgBurn was (is) promising as it looks could capture the proper ISO, but nobody in ImgBurn forum seems to have found a way for doing it properly with UEFI with the set of files I have in the USB. And some others asked in the past same question as me and did not receive an answer.

 

Passmark and Alexander´s: I left them in the fridge as I was looking to a no-brains solution to the boot thingy. So, while they offer bit-by-bit extraction (allowing skipping to deal with dd and unix under windows topics....), I parked them as I really do not understand what this implies in relation to boot (later on this).

 

I also identified the boot creators/managers tools in this forum. Thruth is that I was scared ... I mean, I am able to understand how MBR works, but never got to study UEFI, less grub, multiboot, boot chaining, etc - so, the core realm that is fully yours...

 

So, I am ready to try both ways (alexander image tool and easy2boot) but would need easy detailed step by step about what to do not to lose being able to recreate the UEFI bootable USB.

 

USB contents (abridged) here: https://zerobin.org/...VnG3RTmAHg9JX95

 

analizing the usb with macrorit paratition, the USB is "RSD MBR" (I am unable to find out what RSD might mean). Has a FAT32 active partition type 0C and the rest is simple unallocated.

 

Thanks again

 

PS: I wonder how Acronis TrueImage/Cyberwhatver and/or Clonezilla would handle imaging a USB, perhaps they cannnot...

In any case I must confess that I am surprised that there seems not to be available a simple to use tool (I mean, for dumbs) that captures in an ISO the needed information in a size efficient way that can be used to recreate the USB using as said Rufus...

Edited by caspertone2003, 03 January 2024 - 07:37 PM.

[caspertone2003]

@Wonko

I started playing with Alexander´s tool.

USB is way tooooo slow, over 90 minutes for image creation.

I took a devide mode image obtaining a .img file with around 59GB size. I now zipping it to see if I get any/what size reduction. Started compressing a lot (100 to 3), now it stabilized in (100 to 85), let see what comes out after the ETA of 22hours for compressing 59GB in my PC...

Probably if I would create a second partition and blank/zero it, either Alexander tool would be able to compress it (includes zip routines) or 7Zip would do it.... much smaller and faster.

 

From what I read in the tool FAQ, just restoring what I saved will create a EUFI bootable USB. As the copy and restoration are 1-by-1 fo the full device, there should be no doubt. Anyhow I will try it and see what happens

Question: Have you real experience using this tool for this purpose? Can you confirm above understanding is ok?

 

If I do a volume image, would I lose the bootability information. (FAQ of the tool confirms). 

 

Question: is there any way to "backup" the bootable configuration (in MBR it would mean so save the booting files and the table; in UEFI/GPT I h ave no idea), and then to bring it back? If so, I could make a normal copy of the files in the partition on one side, and the bootable part on another. So, this would mean 6GB for the partition (less, if compression helps) plus some little MB for the boot ...

 

Question: the tool has an option for the restore that says "Fix GPT after restore". Do you know if I have to use that for restoration?

 

PS: I imagine the joke is rather universal ... with local variations of course... ;-)

PS2: I report back when trials end...

[caspertone2003]

@steve6375

 

Just so easy?

Is the FAT32 formating including the info to requeired so that it boots in UEFI mode?

[Wonko the Sane]

Though it costs me a lot to say this , actually UEFI booting is easier to re-create (on another device) than traditional MBR booting because three variables that could be an obstacle (disk geometry, master boot record code and partition boot record code[1]) are taken out of the equation as the UEFI firmware reads (should read) the volume filesystem and find the system file through its internal filesystem driver.

 

Not that something cannot still go wrong, but it is less frequent than before.

 

If the size of data is "just"  6.5 GB (I would have never thought to use "just" for 6.5 GB ) there is frankly no real need for any particularly complex/sophisticated approach, a dd-like image (and an optional "plain" compression as .zip, .gz or .7z) is easily doable on *any* common PC.

 

The only thing you need (IMHO) to do is to "shrink" the partition to a smaller size (capable to hold the data but small enough to be restored on a 8 GB stick, so that - in case of emergency - you don't really need a 32 or 64 GB stick, even if nowadays they are becoming common enough).

 

What I would do:

 

1) make a (temporary) dd-like image of the stick "as is" (the whole 64 GB, better be safe than sorry)
2) shrink the partition to less than 8 GB

3) test if the stick still works (it will)

4) make a dd-like image of the first 2048 sectors (sectors before the partition)

5) make a dd-like image of the volume/partition

6) combine the two images into one

7) clear the stick

8) restore to the stick the combined image
9) test if the stick still works as initially (it will)

 

(steps 4, 5 and 6 can be reduced to a single step by making a single dd-like copy starting from LBA 0 to the last sector of the partition/volume)

 

The only remaining issue (which is not really an issue AFAICT) is the second copy of the GPT partition table that will be "lost" with this simplified procedure, but that anyway would be "lost" if restoring the (partial) image to a differently sized stick.

 

It can be corrected/recreated with gdisk, it is not that difficult, but I believe it is not really-really needed, alternatively it can be recreated by dd-ing it from the main one in the image, but this will need some scripting, if it works (it should) without it, I wouldn't bother.

 

Wonko

 

 

[1] though - to be fair - these were already mitigated in latest BIOS/MBR times by (finally) corrected/working BIOSes and by the sheer increase size of used devices that got rid - indirectly - of many CHS/LBA related issues

[caspertone2003]

Thanks again, Wonko.

What do you mean with the 6)? How do I do it?

By the way, I thougt  the same about "only" 6GB... my first programing was with a digital equipment corporation PDP with some kB of memory...

 

6) combine the two images into one

 

[Wonko the Sane]

I meant combined as in concatenated, under Windows you can use COPY /B, like:

COPY /B first_part.img + second_part.img whole.img

 

Personally I would use the tools in the (good ol') dsfo toolkit, JFYI:

http://reboot.pro/in...=22317&p=215243

 

 

If you prefer GUI tools our friend erwan.l's Clonedisk:

https://labalec.fr/erwan/?page_id=42

 

or the (free edition of) DMDE:

https://dmde.com/

 

are good tools, though for all of these there will be an initial learning curve to be expected.

 

Once you have the "small sized" image assembled and tested you can use *any* tool to re-deploy it to (another) USB stick.

 

The alexpage.de tool seems to have a feature for restoring/recreating the backup GPT data, but it has to be seen how it works exactly, otherwise gdisk:

https://www.rodsbooks.com/gdisk/

will surely do, but also testdisk should be able to do that

https://www.cgsecuri...g/wiki/TestDisk

 

Wonko

 

P.S.: in the thread where I attached the DSFOK toolkit there is - indirectly - some information on how the overall GPT scheme and UEFI booting works , all in all you really need to save only 34 sectors (protective MBR, GPT sector and GPT tables entries[1]):

http://reboot.pro/in...showtopic=22317

 

[1] these are normally 32 sectors, as each sector holds 4 entries, 4x32=128 partitions normally available, strictly speaking you can get away with only 3 sectors as you have less than 4 partitions, but that is only a JFYI:

http://reboot.pro/in...showtopic=22553

it's not like 3 or 34 or 2048 sectors change anything when the "meat" is 6.4 GB

[caspertone2003]

@Wonko

 

Thanks again, a lot of practical suggestions as well as food for mind. Perhaps I finally learn about UEFI....

 

By the way, I just killed the USB image zipping. After 19hours, having "compressed" (ultra) 55GB out of 60GB, the size reduction was 100->99

[caspertone2003]

My objective is achieved and I am happy to share back my findings.

 

Short story: follow steve6375 recipe, it works, has no learning time, is the fastest and requirest the less storage.

 

TLDR;

 

Wonko and steve6375 proposals both seem to solve my need of backing up the USB and afterwards reconstruct it 100% (that is, fully UEFY bootable). Thanks again to both!

 

For the steve6375 way, fully tested:

- to store: zip the files in the USB without compression (no real reduction achieved with compression, files are already very compressed).

- when/if needed: unzip them back into a windows FAT32 formated USB(windows formatting currently creates a normal MBR); USB only needs to be larger than the zip file size.

 

Besides, I tested Alexander Image Tool as well as Erwan Clonedisk, both without and after reduction of USB partition to minimun size and with zeroing the non allocated area (for those operations I used the free macrorit partition portable software), and in all cases extracting image of the device or just of the partition. While I did not tested the reconstruction, I believe it works in all cases. Note that:

- partition mode must suffice, as its reconstruction will create a MBR based USB ... While creating the image takes the same time as copy reading the files in the zip approach, one has before to reduce the partition, that takes its time... In my case, image extraction time is constrained by the read speed of the USB. Image compression does not bring real reduction advantage (image is compressed enough probably due to files being compact). Image size is the partition size (which is slighly bigger than the contained files, special arrangements have to be done to put the second file table near the existing files; this is the reason that zip approach produces the minimun size).

- device mode takes (UBS size/partition size) times that the partition mode. Image size is the USB size. But if zipped with compression, it goes to almost partition size if unallocated area was previously zeroed (note that the zipping takes its time... as well as the zeroing...).

To be noted is that Erwan tool in device mode produces and slightly bigger image, bigger by 1051 kb - close to 1048, coincidence or by any reason? Perhaps Alexander tool is not keeping all needed information? (reminder: recontruction was not tested...).

 

Resume:

zip approach: fastest, use of common (zip) tools, no learning curve, minimun size of file to store, fully tested.

 

CT

[Wonko the Sane]

Maybe the issue (it is a long-standing one) is connected to the so-called (by me ) twilight zone (JFYI):

http://reboot.pro/in...showtopic=18034

 

but it should not apply outside of NTFS and 1048/1051 kb seem to be too much for that case , it is more likely that the difference is due to the presence (or absence) of the backup GPT data, possibly combined to the above or some other quirk of Windows or of the partitioning/formatting software, one should test the alexander tool option to rebuild it or check manually, but as said it shouldn't be functionally relevant.

 

Very likely if - from the start - you had the Windows create the recovery on a smaller 8 GB USB stick you would have had a  pre-made "right sized" device, or did windows ask for 32 or 64 GB initially?

 

If it works, it works, though - from what I understand from your report - what you rebuilt was - at the end of the day - a MBR bootable stick, which is just fine as it is intended as recovery for a specific machine (on which it works) but that may (or may not) work on other UEFI based machines, so I would call it a good but not universal solution.

 

It shouldn't be particularly difficult (again just for the record) to create the stick as UEFI, the only thing that one would have to pay attention to would probably be the UEFI ID of the partition, that should be C12A7328-F81F-11D2-BA4B-00A0C93EC93B (which is what you should get with diskpart "create partition efi")

 

Anyway, all is well that ends well.

 

Wonko

[caspertone2003]

Wise words, Wonko. I went to fast....

 

For the sake of anyone later reading this topic:

 

If you need a backup copy of your USB windows 10 recovery, to be used only in the PC that created it:

 

- try the steve6375 way:

• create the backup: zip the files in the USB without compression
• test if it works: unzip them back into a windows FAT32 formated USB; USB only needs to be larger than the zip file size
• if works, done. 

- not working? Then, I would try the CloneDisk way:

• if you can not recreate the USB, do first a full image backup of the USB (dd or CloneDisk device mode); for dd = https://www.trishtec...age-on-windows/ ; for Clonedisk = https://labalec.fr/erwan/?page_id=42 ; try to recreate the USB from the backup and try if it works. if not, STOP and custody well the USB!
• If the size of the backup is ok for you, then you are DONE
• If the size of the backup is much bigger than the size of the files in the USB partition, use, for example, macrorit to a) resize the USB partition to be the minimun possible and zero the unallocated space (all this takes time but reduces backup size). Then, make a full image backup of the USB (dd or CloneDisk device mode). Recreate the USB from the bakcup and try if it works. If yes, zip it to reduze size and you are DONE. Otherwise, try the other ways proposed by Wonko ...

End of recipy...

[caspertone2003]

@Wonko

 

I gave a fast read to the "twilight zone" thread and got headache and gave up ...

 

By the way I do not undertand your reference to the storage of GPT as it is an MBR formatted USB ...

 

My main conclusions of the full thing are

• I was lucky with the copy and zip solution in that machine, but the USB will not probably work in any other machine and
• In other machine this recipy might not work ...

So, I wrote the previous post.

 

Windows asked for a USB of 16GB or bigger ... I had 8GB and 64GB ... but I did not tried if with 8 it would work ...

 

Thanks again for all help|

Edited by caspertone2003, 07 January 2024 - 01:28 AM.