Jump to content











Photo
- - - - -

Strange Boot Issues from NTFS UFDs

flash.drive ntfs ufd bootable syslinux

  • Please log in to reply
11 replies to this topic

#1 mr-roboto

mr-roboto

    Newbie

  • Advanced user
  • 29 posts
  • Location:NYC
  •  
    United States

Posted 31 May 2013 - 05:54 PM

Folks:
 
I've made some puzzling observations recently and seek advise/comments/insight from those who might've encountered similar events.  I have a bootable DVD called Superdisc.  It's an ongoing project that integrates several DOS floppy images, Live Linux distros, and WinPE on a single bootable SYSLINUX-based platform.  The current incarnation works perfectly from DVD.  However, I just realized I didn't have a bootable USB tool kit, so I copied the contents of the current Superdisc to an existing NTFS-formatted flash drive.  I made the UFD bootable using syslinux.exe (4.06) and rebooted my PC.  The newly bootable UFD worked immediately as expected and all went well until I started testing the Live Linuxes.  Knoppix 7.05 worked normally.  PartedMagic (12_25_2012) worked but it took ten minutes to load initrd.img after which everything else behaved normally. Kaspersky Rescue Disk 10.3 took much longer than normal to boot, then froze w/ a kernel panic early in the initialization process !  PCLinuxOS 2.6.38 + 3.2.18 started to boot normally then failed, unable to locate the media containing the compressed file system.  Really freaky !
 
Using HashCheck, I made a checksum file using the Superdisc folders of the components (from the DVD) that failed and checked the UFD against that checksum file.  To my surprise, there were some random mismatches in the data files (ie. the virus defs) for the Kaspersky Rescue Disc, but the (critical) compressed images for the OSes test as sound, even after multiple tests.
 
Recently, I created a couple Windows 7 installation UFDs, using WinUSB 2.0 (it formats a UFD, transfers the Win7/Vista installation files from an ISO, and makes the UFD bootable.)  My Win7 distribution media is configured to perform an unattended install, the script for which has been in use for several years.  However, using these new UFDs has produced inconsistent results.  Running the distribution media from DVD under VMWare works every time, while an installation from one of the new UFDs (on a client PC) works normally for the 1st two phases of Windows 7 setup.  After entering the username at the start of the final (third) phase of Windows setup, my GUIRUNONCE script doesn't always execute !  The progress bar on the dialog keeps moving but the script itself doesn't launch (normally, the dialogs from the various programs in GUIRUNONCE would appear in no more that thirty secs.)  After five mins, no joy.  Upon reboot, the folder containing the script and programs are on the hard drive, they're just never executed.
 
I've had decent success with bootable UFDs in the past.  I used and swore by PE2USB for bootable BartPE/WinPE UFDs, but that was limited to 2GB FAT32 media.  Much more recently, I've been using PCLinuxOS UFDs, formatted as a ext2 partition for days at a time.  I can't be sure that the two aforementioned "problems" are caused by the same issue.  Obviously, I suspect some sort of data corruption, but I've distributed my activities across several UFDs (all Sandisks), most are less than six months old.
 
Anyway, I'm curious what are the experiences of others in this area.  I can't expect anyone to "come to the rescue" with a solution out of the blue.  A discussion and constructive suggestions are welcome.  TIA...
 


#2 ilko

ilko

    Silver Member

  • Advanced user
  • 500 posts
  •  
    Bulgaria

Posted 01 June 2013 - 01:42 AM

PartedMagic (12_25_2012) worked but it took ten minutes to load initrd.img

Does the system support boot in USB 2.0 mode?Try loading PLoP as first step, next boot your stuff.

 

 

Kaspersky Rescue Disk 10.3 took much longer than normal to boot

ditto

 

then froze w/ a kernel panic early in the initialization process

how is its file structure created? It requires some modifications to be launched from USB-HD media. Search reboot.pro, there are several topics.

 

 

PCLinuxOS 2.6.38 + 3.2.18 started to boot normally then failed, unable to locate the media containing the compressed file system

How is it placed in the UFD? Flat file structure, single ISO and kernel loaded with "cheat" parameters so the ISO is searched for, or...?

 

 

Using HashCheck, I made a checksum file using the Superdisc folders of the components (from the DVD) that failed and checked the UFD against that checksum file.  To my surprise, there were some random mismatches in the data files (ie. the virus defs) for the Kaspersky Rescue Disc

This should not happen obviously. Find out what's going wrong- faulty hardware, UFD, RAM, whatever...

 

After entering the username at the start of the final (third) phase of Windows setup, my GUIRUNONCE script doesn't always execute !

How are these commands executed? Do they expect a source or script on the UFD? Is the UFD involved at all at this step?



#3 mr-roboto

mr-roboto

    Newbie

  • Advanced user
  • 29 posts
  • Location:NYC
  •  
    United States

Posted 02 June 2013 - 06:47 PM

Does the system support boot in USB 2.0 mode?Try loading PLoP as first step, next boot your stuff.

how is its file structure created? It requires some modifications to be launched from USB-HD media. Search reboot.pro, there are several topics.

How is it placed in the UFD? Flat file structure, single ISO and kernel loaded with "cheat" parameters so the ISO is searched for, or...?
Problem 1 solution: SYSLINUX 4.06 (pos beyond, haven't tested beyond 4.06) appears to have issues with its NTFS support, at least that's my current hypothesis. By taking the same media and reformatting the drive to FAT32, everything works ! PCLinuxOS and Kaspersky Rescue both boot from FAT32 UFDs.

I "believe" whatever mechanism used to hand off the filesystem from the SYSLINUX bootloader to the selected OS has imperfections that cause some OSes to fail. Will refer this matter to the SYSLINUX mail list to pursue this further.

This is a snapshot of the main menu from the current incarnation of SuperDisc 2:
Super_Disc_2_1_Jun_Screen_Snapshot.jpg
 
After entering the username at the start of the final (third) phase of Windows 7 setup, my GUIRUNONCE script doesn't always execute !

Problem 2: This was caused by a different issue and a solution isn't immediately on the horizon. I have two customized Win 7 distribution DVDs, [obviously] x86 and x64. I can test my x86 media on both of my PCs, under VMWare Workstation, but not the x64 media (my P4 and Core 2 Duo don't support x64 guests.) Using custom INI files for something called WinToolkit, I was able to get a working setup for my x86 media and I manually edited the x64 INI files, to generate effectively identical x64 distribution media. Didn't work out that way.

My hypothesis explaining the problem is one or more files derails completion of the setup process bec of the NTFS alt data streams, which causes Windows to block recently downloaded programs from executing straight away. So I have to go back to square one and reconstruct my media from cleared sources, then find a x64 test bed to explicitly test the new media.

Anyway, that's it for now. Thanks for the use of the brain cells....

#4 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 16066 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 02 June 2013 - 07:00 PM

AFAIK NTFS support in Syslinux is still experimental, and AFAICR the 5.x version (that might have a better support for NTFS) "sucks BIG" for a number of othere reasons:

http://reboot.pro/to...ux-please-test/

http://reboot.pro/to...odern-syslinux/

 

BUT in your case it could be the actual distro's you used that have invalid (or poor) NTFS support.

You can try with grub4dos direct mapping the .iso to 4th partiton entry .... 

 

:cheers:

Wonko



#5 steve6375

steve6375

    Platinum Member

  • Developer
  • 7566 posts
  • Location:UK
  • Interests:computers, programming (masm,vb6,C,vbs), photography,TV,films
  •  
    United Kingdom

Posted 03 June 2013 - 09:01 AM

I have seen issues with some linux distros not liking NTFS USBs on booting (but can read NTFS once booted).

Try Easy2Boot. Most of these NTFS problems just disappear because Easy2Boot creates a new CDFS partition and so the 2nd stage boot loader sees the CDFS partition and loads the 2nd stage boot files from there and not from the NTFs USB drive.


  • mr-roboto likes this

#6 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 16066 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 03 June 2013 - 09:32 AM

Try Easy2Boot. Most of these NTFS problems just disappear because Easy2Boot creates a new CDFS partition and so the 2nd stage boot loader sees the CDFS partition and loads the 2nd stage boot files from there and not from the NTFs USB drive.

I.e. install a completely different menu system (BTW nice) in order to test if a single write to the UFD partiion table solves the problem.

 

Come on :), mr-roboto seems to be the kind of guy that wants to know what he is doing (as opposed to run an auto-magic app).

 

In any case, starting here is the relevant info about the method/approach:

http://reboot.pro/to...brided/?p=88531

 

:cheers:

Wonko



#7 halikus

halikus

    Frequent Member

  • Advanced user
  • 110 posts
  •  
    Canada

Posted 05 June 2013 - 01:29 AM

From either NTFS USB or CD i find  syslinux-4.06.zip  and grub4dos-0.4.5c-2013-05-21 both work.  After lots of testing, i found that specific combination worked the best for NTFS, although some things, like an extracted Linux distro, might only boot from fat32.



#8 mr-roboto

mr-roboto

    Newbie

  • Advanced user
  • 29 posts
  • Location:NYC
  •  
    United States

Posted 08 June 2013 - 06:45 PM

I.e. install a completely different menu system (BTW nice) in order to test if a single write to the UFD partiion table solves the problem.
 
Come on :), mr-roboto seems to be the kind of guy that wants to know what he is doing (as opposed to run an auto-magic app).
 
In any case, starting here is the relevant info about the method/approach:
http://reboot.pro/to...brided/?p=88531


:cheers:
Wonko

My dear Wonko, you are correct. I do favor the hands-on approach of "rolling my own" bootable media, as you probably recall my "stickied" Superdisc topic on MSFN.  However, this isn't the 1st issue I've encountered that's made me think of my own helper app, a la Easy2Boot, SARDU, etc.  Having said that, I'll definitely take up steve6375 on his suggestion to try E2B, as it has some capabilities that can't be realized using SYSLINUX at all.  My 1st impression the E2B web page is that regardless of doing it yourself (as in the case of SuperDisc) or using E2B, this stuff quickly demands the skills and experience of an expert.

Several years ago, you tried valiantly to get me up to speed w/ GRUB4DOS over on the 911CD forum. At that time, my development environment was much more limited than now, so it was easier to choose SYSLINUX over G4D and that effort ultimately led to the SuperDisc project on MSFN. Maybe it's time to revisit that decision.

@halikus: If I understand your post, you're using SYSLINUX and G4D integrated on your project(s) ? Definitely, not my preference. One or the other, but not both. But thanks for the heads-up.
 
Anyway, thanks to all for the use of their eyeballs+grey matter....

#9 steve6375

steve6375

    Platinum Member

  • Developer
  • 7566 posts
  • Location:UK
  • Interests:computers, programming (masm,vb6,C,vbs), photography,TV,films
  •  
    United Kingdom

Posted 08 June 2013 - 07:05 PM

My 1st impression the E2B web page is that regardless of doing it yourself (as in the case of SuperDisc) or using E2B, this stuff quickly demands the skills and experience of an expert.

 

I was afraid of it looking 'too complicated' when for most things it really isn't. For 99% of ALL linux ISOs you can just copy over the ISO.

Same for Windows Install ISOs (though granted XP install requires two boots and the use of F6 if you are using an unmodified ISO on an AHCI system).

 

The rest of the E2B documentation is how to get 'odd' things going like Hirens or Ophcrack or how to boot 6 different linux ISOs all with separate persistence files - something that I don't think any other method can do... and also how to modify the menu to get it how you want it (again something that I don't think the other contenders even document).



#10 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 16066 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 08 June 2013 - 07:43 PM

One or the other, but not both. But thanks for the heads-up.

Why? :w00t:

 

If you have both a set of spanners and some locking pliers in your toolbox you can undo MORE nuts! ;)

 

:cheers:

Wonko 



#11 mr-roboto

mr-roboto

    Newbie

  • Advanced user
  • 29 posts
  • Location:NYC
  •  
    United States

Posted 08 June 2013 - 09:40 PM

I was afraid of it looking 'too complicated' when for most things it really isn't. For 99% of ALL linux ISOs you can just copy over the ISO.

Same for Windows Install ISOs (though granted XP install requires two boots and the use of F6 if you are using an unmodified ISO on an AHCI system).

 

The rest of the E2B documentation is how to get 'odd' things going like Hirens or Ophcrack or how to boot 6 different linux ISOs all with separate persistence files - something that I don't think any other method can do... and also how to modify the menu to get it how you want it (again something that I don't think the other contenders even document).

 

@steve6375:  I think you misconstrue my comment.  I've done dozens of iterations of bootables like the project shown in the SuperDisc bootsplash above.  That's easy now.  I'm sure I can get up to speed w/ E2B and a comparable project in less than a day.  Experience has taught me that until you've got two or three projects in the bag, take your time, and try not to skip any steps....



#12 mr-roboto

mr-roboto

    Newbie

  • Advanced user
  • 29 posts
  • Location:NYC
  •  
    United States

Posted 09 June 2013 - 12:32 AM

Why? :w00t:

 

If you have both a set of spanners and some locking pliers in your toolbox you can undo MORE nuts! ;)

 

:cheers:

Wonko 

 

@Wonko:  I prefer the analogy of the utility knife vs the Swiss Army knife.  I have both, but only travel w/ the utility knife bec I never use anything beyond the primary blade.  Earlier incarnations of my SuperDiscs have included everything on the bootsplash shown above plus WinPE/BartPE and three different editions of XP.  And I've never perceived that SYSLINUX alone inhibited my work, in any way.  But I am committed to getting up to speed w/ G4D at last....







Also tagged with one or more of these keywords: flash.drive, ntfs, ufd, bootable, syslinux

0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users