Jump to content











Photo
* * * * * 1 votes

Booting VirtualBox with USB workaround

usb virtualbox grub4dos plop

Best Answer steve6375 , 26 February 2013 - 12:18 PM

Is the Win7 64-bit or special in any way? Did you try an plain MS Win7 32-bit ISO extracted to a USB drive?

How about creating a virtual HDD, mounting the HDD, copying the USB stick to the virtual HDD, dismounting it, then boot from it via VBox?

Go to the full post


  • Please log in to reply
307 replies to this topic

#51 DavidB

DavidB

    Silver Member

  • Developer
  • 611 posts

Posted 12 June 2013 - 04:10 PM

I made some changes to the way a VirtualBox VM is started.
There are now 3 independent ways to start it: exe parameters, VM ID and VM Path. Default is VM ID. In the previous version there were only the first 2 and they were not 100% independent.
 

Clipboard01.png Clipboard02.png  Clipboard03.png


On Qemu, since it uses only exe parameters, the other 2 are hidden.

Clipboard04.png

 

In the next few days I won't be able to work much on this program because I won't have a computer. A close friend of mine has a damaged video card in his computer and I will borrow mine to him until I could find a second hand one. He can't afford a new one and he needs a working computer (for good reasons).
Hopefully I will find one fast :D



#52 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 16066 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 12 June 2013 - 09:42 PM

A close friend of mine has a damaged video card in his computer and I will borrow mine to him until I could find a second hand one. 

It seems like - strangely enough - Norwegian is more similar to Latin languages than to English. :w00t:

 

 

The same verb is used, active or passive? :unsure:

 

https://translate.go... borrow to lend

https://translate.go...n/å låne å låne

It is this way in Italian, and latin languages.

 

 

In English there are two different verbs, to lend (as in "I will lend mine to him") and to borrow (as in "He will borrow mine" or He will borrow it from me").

 

Good luck in your search for the fast affordable PC :).

 

:cheers:

Wonko



#53 DavidB

DavidB

    Silver Member

  • Developer
  • 611 posts

Posted 13 June 2013 - 01:44 AM

Actually I'm not 100% Norwegian, I was just born in Norway and sometimes I live there.
But I don't know what's so important about this...



#54 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 16066 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 13 June 2013 - 08:56 AM

But I don't know what's so important about this...

Nothing important, I was just curious about the language.

 

:cheers:

Wonko



#55 DavidB

DavidB

    Silver Member

  • Developer
  • 611 posts

Posted 13 June 2013 - 12:38 PM

Lucky me, looks like the damaged video card still works but only in 800x600 resolution and 4 bit color depth. With anything higher the monitor goes to standby.
Even so I have some weird colored lines on my screen.
Well, for reading/writing in forums and for some programming 16 colors will suffice :D



#56 DavidB

DavidB

    Silver Member

  • Developer
  • 611 posts

Posted 14 June 2013 - 05:44 AM

Few improvements:
- only one instance of the application can be started now (to prevent sharing problems);
- the main window will be shown now in the middle of the screen;
- "Run" has 2 extra values: "Hidden" for Qemu (to hide Qemu log window) and Fullscreen for VirtualBox;
- when adding a new Qemu VM it sets "-L . -name "USB Boot Test" -boot c -m 512" as the default exe parameters;
- when starting a Qemu VM, the first USB drive (and the second if it's set) are automatically added to the Qemu exe command line. I'm thinking on a good way to do this with usb.vmdk also. But it's not so easy  :frusty: 

I'm interested in your opinion: should I modify the code to identify the USB drive by its serial number (to avoid the problems when other drives are connected/disconnected before it's connected) or I should just use the current code (the number)?
Each approach has its pros and cons...



#57 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 16066 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 14 June 2013 - 08:30 AM

I'm interested in your opinion: should I modify the code to identify the USB drive by its serial number (to avoid the problems when other drives are connected/disconnected before it's connected) or I should just use the current code (the number)?
Each approach has its pros and cons...

Do you mean the device serial number or the Disk Signature  or the Volume Serial?

 

(I often happen to remove device serial number from USB sticks)

 

:cheers:

Wonko



#58 DavidB

DavidB

    Silver Member

  • Developer
  • 611 posts

Posted 14 June 2013 - 09:32 AM

Do you mean the device serial number or the Disk Signature  or the Volume Serial?

 

(I often happen to remove device serial number from USB sticks)

 

:cheers:

Wonko

 

I mean the device serial number.
If you remove it from its physical surface that's no problem, my program will not ask you to type it. It will automatically take it from its chip when you will select the device from the popup menu.



#59 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 16066 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 14 June 2013 - 09:45 AM

I mean the device serial number.
If you remove it from its physical surface that's no problem, my program will not ask you to type it. It will automatically take it from its chip when you will select the device from the popup menu.

Physical surface? :w00t:
I am talking of the "unique" serial that is attributed to the stick with it's Mass Production Tool.


The number that (say) Chipgenius or Chipeasy reads, like:
 

Logical drive : R:\ Capacity: 0.1G
Device ID : VID = 8644 PID = 800B
Device SN : 00000000000197E0
Device version : 1.00

Device vendor : Jaclaz
Device model : USB Flash Disk
Protocol : USB2.0
Max power : 500mA

Partition type : FAT32 Device active : OK
Aligned state : Misaligned

Controller : (unknown)
Controller model: DM8261
Flash Vendor : SanDisk, Type: TLC, Page: 8K
Flash ID : 45D798B2
Score : 12 (Normal Score >= 30)

Tools : http://www.upan.cc/tools/mass/dm/
OS Version : Microsoft Windows XP Professional Service Pack 3
Update Status : The current version is the latest version!

 

In the above example, I left it, but I have more than one stick where I removed that info.

 

:cheers:

Wonko



#60 DavidB

DavidB

    Silver Member

  • Developer
  • 611 posts

Posted 14 June 2013 - 10:14 AM

Ok, I think that you're talking about the same thing as I was.
Yes, that's the SN.

Just curious, why do you remove it...? :D
Do you reset it to 0 or do you write another unique SN?
Anyway, I could use also other characteristics, like Capacity (total not volume), VID, PID, Controller model, Flash Vendor, Flash ID ad so on.
It will not be 100% unique but maybe it will be enough to find it in your computer, even if you have other ones connected...

Ok, I'm going out now to buy a video card... The colors that I'm seeing on screen are making me kill somebody :D Just kidding... Maybe... :D



#61 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 16066 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 14 June 2013 - 11:04 AM

I happen to do things just to annoy someone else :smiling9:  (not really-really :hypocrite: , only to prove a point if needed ;)).

 

I happened a few years ago to come across an app (BTW actually and IMHO useless and worthless) that recorded (besides and outside the Registry) the serials (and some other data) of any USB stick connected to the machine.

 

It was a "corporation" and the IT guy thought that it was "1337" to deploy that to all the PC's, and tried to "sell" this idea as a security enhancement/auditing fail-proof method :w00t:, so I made quite a few sticks with Serial 9341963742831649 to see how long it would have taken him to find about it.

 

Then took the habit when re-configuring a stick (I tend to "flip the Removable bit" on my sticks) to set it's serial to all 00's.

 

Why dont' you use the Disk Signature like anyone else? :unsure:

 

(I presume we are talking anyway of partitioned devices)

 

The good thing is that the Disk Signature:

  1. must be present for a NT system to identify the \\.\PhysicalDrive
  2. it must be "unique" on the system (and if there is a collision it is automatically fixed, if I recall correctly)

 

:cheers:

Wonko

 



#62 DavidB

DavidB

    Silver Member

  • Developer
  • 611 posts

Posted 14 June 2013 - 12:42 PM

Solved the video card problem, I bought an old GeForce 6200 FX with only 12 Euro.
I don't want to kill others anymore... Well, maybe with an exception  :whistling: 

Disk Signature could be the solution. Thank you for the idea.
Now I'm curious if there are others who will like it too...



#63 DavidB

DavidB

    Silver Member

  • Developer
  • 611 posts

Posted 14 June 2013 - 04:15 PM

I found a good way to retrieve the disk signature using Delphi code and I intend to implement it.

But if anyone wants the drive number implementation instead, speak now.

Like the priest says "If anyone objects to this marriage, let them speak now or forever hold their peace.." :D



#64 steve6375

steve6375

    Platinum Member

  • Developer
  • 7566 posts
  • Location:UK
  • Interests:computers, programming (masm,vb6,C,vbs), photography,TV,films
  •  
    United Kingdom

Posted 14 June 2013 - 04:22 PM

if the idea is to uniquely identify a device, why not use disk sig+USB serial no+drive capacity?



#65 DavidB

DavidB

    Silver Member

  • Developer
  • 611 posts

Posted 14 June 2013 - 05:33 PM

Yes, it will raise the "uniqueness" from, let's say, 99% to 99.9%.

The only downside is that the program will be a little more slower and a little more unstable.

But I think that's not such a big problem.



#66 DavidB

DavidB

    Silver Member

  • Developer
  • 611 posts

Posted 15 June 2013 - 07:03 AM

It seems like there is a problem retrieving the disk signature on GPT disks. I can't find any good and documented API function for this.
With MBR disks there are no problems. Currently the USB disks with MBR are far many than those with GPT but in the future this could change...



#67 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 16066 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 15 June 2013 - 09:37 AM

Cannot you just get 4 bytes with direct disk access to first sector of the device? :unsure:

 

I am not familiar with GPT style partitioning, but the signature should be in the same place, the first sector is anyway a "protective MBR", the function should be exactly the same:

http://msdn.microsof...7(v=vs.85).aspx

 

http://msdn.microsof...7(v=vs.85).aspx

 

I mean, a non-GPT aware NT based system has anyway to "identify" the disk using Disk Signature, so when you connect such a disk to a 2K (and possibly also XP) it must write to it a Disk Signature. :unsure: the issue might be a a GPT disk that was only mounted on newish Windows NT versions. :dubbio:

 

Yep, logic above confirmed :):

http://thestarman.pc...asm/mbr/GPT.htm

 

But if you use the "standard code" you can have Disk Signature on MBR disks and "DiskId" on GPT ones.

And as well if you use direct disk access you can get the 4 bytes on sector 0 offset 440 or the 16 bytes on sector 1 offset 56.

 

:cheers:

Wonko



#68 DavidB

DavidB

    Silver Member

  • Developer
  • 611 posts

Posted 15 June 2013 - 10:08 AM

Cannot you just get 4 bytes with direct disk access to first sector of the device? :unsure:

 

 

Disk signature for MBR yes, I'm already using this to take it from MBR:
 

const
  // Max number of drives assuming primary/secondary, master/slave topology
  MAX_IDE_DRIVES = 16;
var
   i: Integer;
   RawMBR: array[0..511] of Byte;
   btsIO: DWORD;
   hDevice: THandle;
   s: string;
begin
   s := '';
   for i := 0 to MAX_IDE_DRIVES - 1 do
   begin
      hDevice := CreateFile(PChar('\\.\PHYSICALDRIVE' + IntToStr(i)), GENERIC_READ,
         FILE_SHARE_READ or FILE_SHARE_WRITE, nil, OPEN_EXISTING, 0, 0);
      if hDevice <> INVALID_HANDLE_VALUE then
      begin
         SetFilePointer(hDevice, 0, nil, FILE_BEGIN);
         ReadFile(hDevice, RawMBR[0], 512, btsIO, nil);
         CloseHandle(hDevice);
         s := s + 'Disk ' + IntToStr(i) + ' = ' + IntToHex(RawMBR[443], 2) + ' ' +
            IntToHex(RawMBR[442], 2) + ' ' + IntToHex(RawMBR[441], 1) +
            ' ' + IntToHex(RawMBR[440], 2) + #13#10;
      end;
   end;
   ShowMessage(s);
end;

However the disk signature for a GPT disk is a GUID, not just a 4 bytes ID. And I'm not sure about the location...

 

I am not familiar with GPT style partitioning, but the signature should be in the same place, the first sector is anyway a "protective MBR", the function should be exactly the same:

http://msdn.microsof...f552637(v=vs.85).aspx

 

http://msdn.microsof...f561447(v=vs.85).aspx

 

Yes, I know this function, seems good, for the past 2 days I'm trying to make it work in Delphi and couldn't find a way because I can't find any documentation about setting it in Delphi.

 

 

I mean, a non-GPT aware NT based system has anyway to "identify" the disk using Disk Signature, so when you connect such a disk to a 2K (and possibly also XP) it must write to it a Disk Signature. :unsure: the issue might be a a GPT disk that was only mounted on newish Windows NT versions. :dubbio:

 

Yep, logic above confirmed :):

http://thestarman.pc...asm/mbr/GPT.htm

 

But if you use the "standard code" you can have Disk Signature on MBR disks and "DiskId" on GPT ones.

And as well if you use direct disk access you can get the 4 bytes on sector 0 offset 440 or the 16 bytes on sector 1 offset 56.

 

:cheers:

Wonko

 

 

Thanks for the tip, I will modify the code to see if I can get the GUID from the GPT disks (from that location).



#69 DavidB

DavidB

    Silver Member

  • Developer
  • 611 posts

Posted 15 June 2013 - 12:51 PM

I have a working code for both:

 



const
   // Max number of drives assuming primary/secondary, master/slave topology
   MAX_IDE_DRIVES = 16;
var
   i: Integer;
   DiskType: Byte;
   RawMBR: array[0..511] of Byte;
   btsIO: DWORD;
   hDevice: THandle;
   s: string;
begin
   s := '';
   for i := 0 to MAX_IDE_DRIVES - 1 do
   begin
      hDevice := CreateFile(PChar('\\.\PHYSICALDRIVE' + IntToStr(i)), GENERIC_READ,
         FILE_SHARE_READ or FILE_SHARE_WRITE, nil, OPEN_EXISTING, 0, 0);
      if hDevice <> INVALID_HANDLE_VALUE then
      begin
         SetFilePointer(hDevice, 512, nil, FILE_BEGIN); //sector 1 for GPT
         ReadFile(hDevice, RawMBR[0], 512, btsIO, nil);
         if (IntToHex(RawMBR[0], 2) + IntToHex(RawMBR[1], 2) +
            IntToHex(RawMBR[2], 2) + IntToHex(RawMBR[3], 2) +
            IntToHex(RawMBR[4], 2) + IntToHex(RawMBR[5], 2) +
            IntToHex(RawMBR[6], 2) + IntToHex(RawMBR[7], 2)) =
            '4546492050415254' then //EFI PART
            DiskType := 1 //GPT
         else
         begin
            DiskType := 0; //MBR
            SetFilePointer(hDevice, 0, nil, FILE_BEGIN); //sector 0 for MBR
            ReadFile(hDevice, RawMBR[0], 512, btsIO, nil);
         end;
         CloseHandle(hDevice);
         if DiskType = 0 then
            s := s + 'Disk ' + IntToStr(i) + ' = ' + IntToHex(RawMBR[443], 2) + ' ' +
               IntToHex(RawMBR[442], 2) + ' ' + IntToHex(RawMBR[441], 2) +
               ' ' + IntToHex(RawMBR[440], 2) + #13#10
         else
            s := s + 'Disk ' + IntToStr(i) + ' = ' + IntToHex(RawMBR[59], 2) +
               ' ' + IntToHex(RawMBR[58], 2) + ' ' + IntToHex(RawMBR[57], 2) +
               ' ' + IntToHex(RawMBR[56], 2) + ' - ' + IntToHex(RawMBR[61], 2) +
               ' ' + IntToHex(RawMBR[60], 2) + ' - ' + IntToHex(RawMBR[63], 2) +
               ' ' + IntToHex(RawMBR[62], 2) + ' - ' + IntToHex(RawMBR[64], 2) +
               ' ' + IntToHex(RawMBR[65], 2) + ' - ' + IntToHex(RawMBR[66], 2) +
               ' ' + IntToHex(RawMBR[67], 2) + ' ' + IntToHex(RawMBR[68], 2) +
               ' ' + IntToHex(RawMBR[69], 2) + ' ' + IntToHex(RawMBR[70], 2) +
               ' ' + IntToHex(RawMBR[71], 2) +
               #13#10;
      end;
   end;
   ShowMessage(s);
end;

 

Ok, now to implement this in the rest of the code  :hammer:



#70 DavidB

DavidB

    Silver Member

  • Developer
  • 611 posts

Posted 15 June 2013 - 05:41 PM

For now I implemented only with disk signature. It performed great in tests. But if, in the future, there will be duplicates problems, I will implement also with serial number and capacity.
With Qemu is adapting perfectly to any change in physical disks order.
With VirtualBox only half because the automatic change of usb.vmdk is not yet implemented.
But I'm thinking of doing this in the next few days...

Btw, should I detach usb.vmdk, rebuild it and attach it again (kind of slow but less potential problems) or should I just change the number of the \\.\PhysicalDrive inside the file (faster but rare problems may appear)?



#71 DavidB

DavidB

    Silver Member

  • Developer
  • 611 posts

Posted 18 June 2013 - 11:47 AM

In the past few days I've worked hard to implement the automatic modification of the vmdk HDD file(s) (pointing to the USB drive(s)).

What I've done so far:
- when the vmdk file is already attached to the VM it modifies its content: drive capacity, drive number and the number of cylinders. Btw, a curious thing about the number of cylinders: it seems that the maximum number (generated by the vboxmanage) is 16383 (although the VM doesn't show error if you increase it manually). And since it always uses 16 heads and 63 sectors it means that the maximum (geometric) size is 7.8745 GB.
- when it doesn't find the vmdk file attached, it creates one for that USB drive and attach it to a fixed port number (0).

What I have to implement more:
- the attach of the vmdk file to the first free/available port (so it shouldn't overwrite any other already attached drives);
- the removal of the vmdk for the second USB drive when it's removed also from the VMS entry.

It's gonna take some time to implement these too...
Well, back to work :D



#72 DavidB

DavidB

    Silver Member

  • Developer
  • 611 posts

Posted 19 June 2013 - 12:59 PM

It's all implemented now...

Here it is (1.0 Beta 1 version):

 

LE: link removed, you can find a better version in the next posts.



#73 steve6375

steve6375

    Platinum Member

  • Developer
  • 7566 posts
  • Location:UK
  • Interests:computers, programming (masm,vb6,C,vbs), photography,TV,films
  •  
    United Kingdom

Posted 19 June 2013 - 03:04 PM

Not quite sure how to use your GUI... How is one meant to set up the initial VM for VBox before selecting it in your app?

 

I had a few problems.

First - is it possible to copy the existing vbox rather than add drives and change an existing one - i.e. user selects existing vbox but you write a copy and temporarily change it and boot to that?

One of my vm's has a USB.vmdk+hdd1+hdd2 as storage, when I used your app and added first USB drive, it added USB1 to the other 3 drives and then wouldn't boot due to IDE drive config issues. The addition also meant that the VM that previously worked fine in VM Manager now doesn't (until I delete the extra added drive).

 

So I presume I need to create a VM without a USB vmdk already attached.

 

The First USB drive list is listing drives F and H, but F is my internal SSD HDD not a USB drive.

 

One of my VMs ("USB2 Boot") had a snapshot. The list of .vboxes in your app seemed to be not listing that one, but another vbox (called 'test iso') seemed to have the ID number of the "USB2 Boot" VM and it booted to the USB2 Boot VM when I ran 'test iso' from your app. Once I deleted the snapshot, your app listed the USB2 Boot VM.

 

Might be best if you gave unique names to the .vmdk's you create, just in case they clash with existing ones - e.g. VMStart_USB1.vmdk?

HTH

Steve



#74 DavidB

DavidB

    Silver Member

  • Developer
  • 611 posts

Posted 19 June 2013 - 04:11 PM

Not quite sure how to use your GUI... How is one meant to set up the initial VM for VBox before selecting it in your app?

 

 

I haven't implemented complete creation and/or control of the VirtualBox VM.
You just have to create a machine with default settings or use one that is already built. Most will work ok with my application and I will improve it when someone will show me a commonly used VM that doesn't work. Just send me the VM files and/or the VB configuration file(s) and I will try to solve the problem.

 

I had a few problems.

First - is it possible to copy the existing vbox rather than add drives and change an existing one - i.e. user selects existing vbox but you write a copy and temporarily change it and boot to that?

 

 

To make a copy of the VM is not such a bad idea but unfortunately not always practical because:
- it had to be physically copied elsewhere each time but when it has big files it will take too long time;
- I wouldn't call VirtualBox a bug free application so each time you will use the internal commands to copy the VM elsewhere, register it and so on, this will increase the chances that something would go wrong.
- it will take slightly more resources from the host computer;
- like I said, my application doesn't have complete control of the VM, I implemented as much control as it was needed to add/modify/delete the vmdk file(s), the rest requires user interaction with the VM (in the VirtualBox interface); some users will not like finding a duplicate of the VM.

 

One of my vm's has a USB.vmdk+hdd1+hdd2 as storage, when I used your app and added first USB drive, it added USB1 to the other 3 drives and then wouldn't boot due to IDE drive config issues. The addition also meant that the VM that previously worked fine in VM Manager now doesn't (until I delete the extra added drive).

 

So I presume I need to create a VM without a USB vmdk already attached.

 

Strange, it should of worked even with a vmdk previously attached. Could you please send me the this vmdk and the vbox file so I can try to duplicate the issue...? Maybe even some data about the USB drive (capacity).

 

The First USB drive list is listing drives F and H, but F is my internal SSD HDD not a USB drive.

 

 

Yes it's listing internal drives also because I haven't implemented a code that would differentiate USB drives from internal HDD's. Reason: not all the USB drives are seen as "removable" by the OS and not all the internal ones are seen as "fixed"; sometimes even the users "flip" the "removable" bit. Let me tell you something amusing: when I first installed Windows XP on my internal SATA WD, it was seen as "removable" but my external WD HDD was seen as not "removable". I fixed with my internal HDD but I let the external one to be seen as internal.
So I will have to make a code that it will search in specific computer ports which it is not so easy, especially when you try to make it 100% reliable...
Perhaps in the future, will see...

 

One of my VMs ("USB2 Boot") had a snapshot. The list of .vboxes in your app seemed to be not listing that one, but another vbox (called 'test iso') seemed to have the ID number of the "USB2 Boot" VM and it booted to the USB2 Boot VM when I ran 'test iso' from your app. Once I deleted the snapshot, your app listed the USB2 Boot VM.

 

Hmm, seems like a bug. Could you please recreate the problem and send me the vbox file and "c:\Documents and Settings\Username\.VirtualBox\VirtualBox.xml" (this is a Windows XP path, some of the folders in the "tree" are virtual in 7/8).

 

Might be best if you gave unique names to the .vmdk's you create, just in case they clash with existing ones - e.g. VMStart_USB1.vmdk?

HTH

Steve

 

 

My application uses usb0..usb99. It checks before creating another vmdk and uses another number in that range. So the probability of a clash is very little.
But I will use "VMSusb0..99".

Thank you for the ideas.

 

 

Later edit: using VMS to atart a VM with snapshot is generally not such a good idea anyway. The main gold of VMS is to make writes on USB drive visible; the snapshot is making the VM write into a file on HDD.



#75 DavidB

DavidB

    Silver Member

  • Developer
  • 611 posts

Posted 19 June 2013 - 05:21 PM

Two more things:
1. Are you using the latest VirtualBox version (4.2.12)?
2. I've seen that some programs (WinSetupFromUSB and probably your program too) use WMI to get data about USB drives. It's good but the problem is that is restricted to Windows (WMI = Windows Management Interface). Other OS's have very poor WMI implementation. For example I tried to use WinSetupFromUSB in wine (Linux), it's not even showing the interface before crashing.
I'm trying to find ways that can be used in any OS so I can port my application (instead of rewriting the entire code). But that's not so easy...






0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users