42 replies to this topic

[MedEvil]

I discovered this really weird problem in Win7PE_SE.
The USB hotplug does not work on every boot identical.

Please note i'm not talking about different builds, which should behave identical but about the same wim.

7 out of 10 runs, USB hotplug works absolutely flawless. Trayicon with device listed, autorun dialog and refresh of explorer view.

The other times one or more of the above are not working right from the start.

I could use some hints, why this could be happening and how to check for the problem.

[MedEvil]

The problem seems to be somehow connected to the compression of the wim.
With maximum compression, there are problems.
With normal compression, there are non.

No idea why. Any suggestions?

[Wonko the Sane]

Only very generic ones

What you describe could be well linked to a "timing problem" of some kind.

 

More "tightly" compressed data may take some little more time more in being accessed/loaded/executed/whatever during the initial booting.

Possibly (say) making a given service (or *whatever*) start before a required (and which loading was initiated earlier) *something else* has not finished loading (or creating a needed key in the Registry, etc.).

 

The old idea (that very few people EVER took into consideration) of having a very "basic"  main boot (or image or .wim or whatever) that being small need not high compression and a second separate volume (or image or .wim or whatever) seems to get to the surface again.

 

Wonko

[MedEvil]

But wouldn't a more compressed image affect all services / files equally and therefore not disturb the order?

[steve6375]

Does the same thing happen on a different system?

Are you testing on real h/w or virtual?

Have you tried preparing the wim file on a different system?

[Wonko the Sane]

But wouldn't a more compressed image affect all services / files equally and therefore not disturb the order?

No, not necessarily.

 

Lossless compression is very variable (in tightness) depending on the "original uncompressed" and consequently the time (or processor cycles) needed to restore/inflate/expand can vary.

Paradoxically (and as you might well know) if you try to compress already compressed data (by another algorithm) the compressed is bigger than the original, but more than that there is no real (AFAIK) "judgement" about balancing compression tightness with decompression speed (if not the rudimental, generic settings like "fast", "better", "maximum" and the like, which are normally "global" ones).

 

Now we are going over pure theory (and cannot say how this may  - or may not - apply to .wim compression and/or actual PE3.x booting), but let's imagine this situation (theoretical example), in an archive you have three files, all 1 Mb in size (uncompressed).

With "fast" you get:

file01  512 Kb (50% compression) 7 sec. expand time

file02  768 Kb (25% compression) 4 sec. expand time

file03  768 Kb (25% compression) 4 sec. expand time

the result will be an archive 2 Mb in size and thus an overall 33% compression and an expansion time of 15 seconds

 

With maximum you get:

file01 500 Kb (51% compression) 9 sec. expand time

file02 672 Kb (35% compression) 8 sec. expand time

file03 735 kb ( 28% compression) 6 sec. expand time

the result is an archive 1907 Kb in size and thus an overall 38% compression BUT with an expansion time of 23 seconds AND with file02 taking double the time (and thus possibly altering a sequence )

 

If we have "full control" on the compression and decompression (and have all the needed info and do some pretty heavy calculation in "real world" with hundred or thousands of files, taking into consideration execution times and what not) we may find out that a "custom" compression such as:

file01  512 Kb (50% compression) 7 sec. expand time

file02  768 Kb (25% compression) 4 sec. expand time

file03 735 kb ( 28% compression) 6 sec. expand time

resulting in an archive 2015 Kb in size and thus overall 34% compression with an expansion time of 17 seconds and keeping "relative sequence" the same (or actually increasing the gap between file02 and file03) will give more satisfactory results.

 

Wonko

[MedEvil]

Very valid points Wonko, didn't thought of it that way.

But since wim are not just an archive, but meant as a compressed drive, shouldn't the driver take care of such things? I mean, i have never heared that anyone had any trouble using a compressed NTFS drive.

That said, i agree with you that this seems to be a timing issue.

Any idea on how to pinpoint the problem?

[MedEvil]

Does the same thing happen on a different system?
Are you testing on real h/w or virtual?
Have you tried preparing the wim file on a different system?

- I have only 1 computer, on which i can run a rambooted Win7.
- Yes i'm testing in real hardware.
- No, i have not, but could do so. I however have build the PE under XP and Win7 on that computer. Didn't make a difference.

[MedEvil]

The conclusion from my tests was unfortunately wrong.
Compression has nothing to do with the problem.

Created yesterday evening a new build changed nothing, but added a few apps. And USB is off again!

There is clearly some timing issue. But how to find it?

[steve6375]

Can you explain a bit more clearly what you are seeing and how to reproduce this

 

e.g. Is this what you are doing

 

1. Boot to WinPE in RAM from USB stick

2. Connect another USB stick

3. Check if detected

4. If it is, use Safely Remove and then unplug

5. goto 2

 

 

Or

1. Boot to WinPE in RAM from USB stick

2. Remove boot USB

3. Re-insert boot USB

4. check if detected

5. If it is use Safely Remove and unplug

6. goto 3

 

Or does hotplug always work on one boot, but if you reboot it does not work at all, or sometimes works and sometimes not - or what???

[MedEvil]

I boot the wim file directly from HDD. (for tests this is the fastest method)
Once booted up, i attach a USB-Stick.

If everything works ok:
- the Hotplug tray icon will show up and list the name of the USB-stick to remove
- the autostart dialog will pop open
- should explorer be open the view is refreshed and the newly attached stick is shown.

In short everything works just like on a regular Win7.


If things don't work, one or more of the following happens:
- the Hotplug tray icon will show up, but the list of devices to remove will be empty
- no autostart dialog will pop open
- should explorer be open the view is not refreshed, USB-Sticks are both, not shown when attached and not removed from view when removed.

If things don't work. Opening and closing explorer a few times and pluging in and ut the sticks a few times , will mostly result in USB-Hotplug to become fully functional., but not always.

[lianzi2000]

I boot the wim file directly from HDD. (for tests this is the fastest method)
Once booted up, i attach a USB-Stick.

If everything works ok:
- the Hotplug tray icon will show up and list the name of the USB-stick to remove
- the autostart dialog will pop open
- should explorer be open the view is refreshed and the newly attached stick is shown.

In short everything works just like on a regular Win7.


If things don't work, one or more of the following happens:
- the Hotplug tray icon will show up, but the list of devices to remove will be empty
- no autostart dialog will pop open
- should explorer be open the view is not refreshed, USB-Sticks are both, not shown when attached and not removed from view when removed.

If things don't work. Opening and closing explorer a few times and pluging in and ut the sticks a few times , will mostly result in USB-Hotplug to become fully functional., but not always.

Could it be somehow the ShellHWDetection service is not running, or not started automatically? I seems to have a similar problem but more consistent -- the ShellHWDetection service can never start automatically, so the "Autorun" window will never popup. The Hotplug tray icon will not show up (just stay in systray), but the device will be listed for removal. explorer will not refresh to show the new drive. The only way to start ShellHWDetection is to do a "scan for hardware change" from device manager (devcon rescan will not work), after that everythign works.

[Agent47]

Hi MediEvil

 

Did you found the solution to USB problem. Is USB works fine in other PE3 builds like Multpe?

[MedEvil]

The problem is with timing.
Just adding or removing a program from the wim, even if it is not used during boot, affects how perfect USB works straight out of the box.

I suspect that explorer is not fully configured and needs to create some registry keys during start.
Depending on where the explorer is, when USB becomes available determines what features are available.

In case this wasn't clear, this is only a temporary problem! USB, no matter the state it is in when the first USB-Stick gets pluged in, will eventually become fully functional in Win7PESE!

No, i havn't yet encountered any other Win7 based PE which does USB support better.
Most start somewhat broken and will not improve with use. Sorry.

[Wonko the Sane]

The problem is with timing.
Just adding or removing a program from the wim, even if it is not used during boot, affects how perfect USB works straight out of the box.

I suspect that explorer is not fully configured and needs to create some registry keys during start.
Depending on where the explorer is, when USB becomes available determines what features are available.

Do you mean file order in the compressed image?

Maybe there is also for the contents of the wim some "better" setting not unlike what a pesort.txt can profduce on PE 1.x, BOTH on CD and on USB sticks.

Some initial reference:

http://www.911cd.net...showtopic=23863

 

Could there be ".wim fragmentation"?

 

Wonko

[MedEvil]

Since the wim is run from RAM, sorting will not speed things up noticeable.
And using sorting as a methode of fixing a timing problem is a bad idea. On another computer things will be profoundly different.

No the only real fix is, to not have two processes racing eachother.
We'll have to fix the race so explorer always wins. Guaranteed!

---------------

I actually meant, where explorer is, in the list of registry keys it has to create.

The more it manages to create, before USB starts, the more does USB and explorer integrate with eachother.

All problems are basicly of the same kind. USB does not know how to notify explorer of changes.

[MedEvil]

So i moved things around in the startup files and even introduced 60 second waits, in order to force a working / not working USB.
All without result. Nothing i did affected the USB timing issue at all.

If it was a build with working USB, it stayed working.
If it was a build with not working USB, it stayed not working.

I'm fresh out of ideas and open for suggestions, how to track this issue down.

[steve6375]

Did you try all the different USB ports on the system? Any difference?

Also try plugging in another USB device and the changing which ports each are plugged into.

It could be to do with enumeration of the USB ports, so changing the port you use for the USB drive and adding in other USB devices may affect the timing..??

[Agent47]

I tried several Chinese and Vietnamese PE3 discs and all are using some work around for USB ejection. So far i didn't see a single PE3 with fully working USB functionality (i mean native USB ejection and auto refreshed explorer). 

[MedEvil]

@Agent47
Win7PESE is also the only PE i know of, which does get USB fully functional under some circumstances.
Would be a lot easier to find the hickup, if there were a PE which has it working all the time.

@steve6375
Removing the cardreader is the only thing i have not tryed yet, everything else i already have.

The one difference i noticed about a fully functioning USB is, that the drive gets accessed 2 times.
1st access the hotplug icon appears in the system tray.
2nd access the autostart dialog appears.

If the USB does not fully work, there's only the 1st access to the drive, which may or may not include explorer views being updated.

[paraglider]

I sometimes see the same problem in full windows. Have to unplug the device and plug it into a different USB port.

[MedEvil]

I did a dozend boots with the same wim on the same computer and took each time a snapshot of the order the drivers were loaded in.

They are never identical!

There's always at least one driver, which changes order.

It seems, the later USBStor is started, the better USB functionality is working.

[MedEvil]

I sometimes see the same problem in full windows. Have to unplug the device and plug it into a different USB port.

This is a completely different problem.
What you describe is a problem of the hardware to detect the presence of a USB-device, because the resistance between the MB chip and the USB-controller is too high.

The problem i describe here is a software one. It seems to be related to some kind of timing or order issue.

[Wonko the Sane]

This is a completely different problem.
What you describe is a problem of the hardware to detect the presence of a USB-device, because the resistance between the MB chip and the USB-controller is too high.

Wow, your crystal ball is working very well if you can know from a distance what exactly happens on paraglider's systems
Remote Ohmeter/Oscilloscope, and entirely new frontier!

For NO apparent reason:

Spoiler

 

Wonko

[MedEvil]

I use the same crystal ball as Sherlock Holmes. It's called logic.