Jump to content











Photo
- - - - -

Any experiences with ExFat on USB drives? I was pleasantly surprised.


  • Please log in to reply
19 replies to this topic

#1 Rootman

Rootman

    Frequent Member

  • Advanced user
  • 279 posts
  • Location:USA

Posted 02 January 2018 - 04:47 PM

Well, you learn something new every year. Since I am always hauling around large files in excess of 4GB I have been using NTFS on my USB drives forever. The only practical reason was for the large file sizes. I had no need of encryption or compression. When ExFat came out and started to get more popular I simply bypassed it thinking that it provided nothing that NTFS did. 

 

The prior 2 USB 3 drives I bought were having small file performance issues, as well as dismount issues.  So, on a lark over the recent holiday days off I setup one as ExFat.  I was pleasantly surprised. 

 

I found the same stick that was previously NTFS mounted a lot faster as ExFat.  I also typically put a VeraCrypt container on the sticks that is about 3/4 or more of the drive in size and launch it from the stick itself using a script in the root of the drive.  The VeraCrypt container mounted a lot faster (it was also formatted as ExFat) and most importantly, dismounted and allowed the stick itself to be unmounted cleanly without a reboot.  Prior to this, the last few USB 3 sticks I had refused to be dismounted citing an application had files open. Under ExFat it will now dismount the Veracrypt and then the USB drive just as I want it too. 

 

I just bought a SanDisk256G Extreme PRO oer Steve at Easy2Boots recommendation.  It seems amazingly fast.  I formatted it as NTFS and I am pondering swapping it over to ExFat as well. 

 

So, it may be old news to some of you guys, but it was a learning experience for me.

 

Anyone else have similar experiences?


  • Nuno Brito likes this

#2 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 14267 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 02 January 2018 - 07:14 PM

JFYI (only partially related):

http://www.msfn.org/...d-on-usb-stick/

 

If you are into experimenting (and only use newer OS's) UDF could be a good candidate.

 

The bootability of non-FAT12/16/32, non-NTFS filesystems with recent OS's needs to be tested anyway.

 

:duff:

Wonko



#3 Nuno Brito

Nuno Brito

    Platinum Member

  • .script developer
  • 10525 posts
  • Location:boot.wim
  • Interests:I'm just a quiet simple person with a very quiet simple life living one day at a time..
  •  
    European Union

Posted 02 January 2018 - 08:44 PM

I've been using ExFat for USB drives that need to work across Linux/Windows/Mac with bigger files.

 

Windows accepts ExFat (no wonder there), Mac also permits to read/write/format in ExFat.

 

Under Linux typically need to install the exfat-utils package and then it will work.

 

No complaints whatsoever, works good. :cheers:



#4 AnonVendetta

AnonVendetta

    Silver Member

  • Advanced user
  • 648 posts
  • Location:A new beginning.....
  • Interests:Self-development, computing

Posted 03 January 2018 - 03:17 AM

I don't see much of a reason to use exFAT on a USB, it's generally not suitable for booting. And NTFS is more widely accepted and supported. Not to mention that exFAT corrupts often. I use it as the filesystem on my MicroSD for my phone, but only because it is well supported by Android. With exFAT I don't have to worry about reformatting my MicroSD every time I flash a new ROM on the phone. I would much rather use a filesystem like ext4 or F2FS, because they are stable, faster, rarely corrupt, and support full Unix discretionary access control permissions, but some ROMs don't support those FSes. So I just settle for something that is recognized by most ROMs I try. And every time I insert it into a card reader in Windows, corrupt files are found when I run a filesystem check, and are placed into a folder called FOUND.000 in the root. There are usually anywhere ranging from several hundred MB to several GB placed in there. The file names are mangled beyond usability, and give no hint in regards to what they are, so they're pretty much unrecoverable. I suspect that they are portions of other files that somehow got corrupted while writing.



#5 steve6375

steve6375

    Platinum Member

  • Developer
  • 6736 posts
  • Location:UK
  • Interests:computers, programming (masm,vb6,C,vbs), photography,TV,films,guitars
  •  
    United Kingdom

Posted 03 January 2018 - 09:19 AM

corruption like that is usually a sign of not clicking on a 'safely remove hardware' (or linux equivalent) before removing the USB drive...



#6 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 14267 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 03 January 2018 - 10:01 AM

Still, ex-FAT is not "easily" bootable for Windows, you need grub4dos or similar (for booting images) and/or anyway at least the Windows 8 :ph34r: BOOTMGR for "flat" booting.

http://reboot.pro/to...ated-usb-drive/

 

As said there, most probably an XP Kansas City Shuffle is possible, but again it is not exactly easy-peasy.

 

:duff:

Wonko



#7 Rootman

Rootman

    Frequent Member

  • Advanced user
  • 279 posts
  • Location:USA

Posted 04 January 2018 - 12:35 PM

I boot all my USB drives with Grub4DOS, so no issues there with exfat.  I also have had a lot of corruption on them with NTFS, it seems that every few months I would eject one, pull it and have it hose.  So I am anxious to see if this issue is mitigated at all. 

 

I am impressed with how rapidly the mount and dismount, also it appears that the units that were previously extremely poor with small fie transfer  - almost to the point of being unusable - are now acceptable.

 

I brought this issue up to remind people that it's out there, and seems reliable and efficient.  I was  tried and true NTFS believer for USB, now I am going to use exfat whenever possible. 



#8 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 14267 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 04 January 2018 - 05:13 PM

I boot all my USB drives with Grub4DOS, so no issues there with exfat.  I also have had a lot of corruption on them with NTFS, it seems that every few months I would eject one, pull it and have it hose.  So I am anxious to see if this issue is mitigated at all. 

 

I will repeat (just in case of a misunderstanding):

  1. NTLDR is NOT compatible with exFAT
  2. BOOTMGR (of Windows Vista and 7) is not compatible with exFAT.

Most probably (but not necessarily) Vista and 7 can be booted using the 8 (or later) BOOTMGR, for XP there is NO known solution (though very likely, but again not necessarily an XP Kansas City Shuffle may work).

 

As a general rule of thumb filesystem corruption is due to (lack of) flushing of the in-cache data, using the Safe Removal button (or similar) or however flushing cache writing commits to the drive will eliminate (or however greatly reduce) those issues.

 

From a pure technical standpoint NTFS is ways more "robust" and reliable than *any* FAT based filesystem, and as a matter of fact the good MS guys know that and in Wndows Embedded  they use TexFAT (which essentially is an overlay to exFAT) that assures transaction-safe operations:

 

https://msdn.microso.../cc907929.aspx 

 

but TexFAT is NOT implemented in Desktop OS, so all you have is pure exFAT.

 

:duff:

Wonko



#9 Rootman

Rootman

    Frequent Member

  • Advanced user
  • 279 posts
  • Location:USA

Posted 12 January 2018 - 01:32 PM

I will repeat (just in case of a misunderstanding):

  1. NTLDR is NOT compatible with exFAT
  2. BOOTMGR (of Windows Vista and 7) is not compatible with exFAT.

Most probably (but not necessarily) Vista and 7 can be booted using the 8 (or later) BOOTMGR, for XP there is NO known solution (though very likely, but again not necessarily an XP Kansas City Shuffle may work).

 

As a general rule of thumb filesystem corruption is due to (lack of) flushing of the in-cache data, using the Safe Removal button (or similar) or however flushing cache writing commits to the drive will eliminate (or however greatly reduce) those issues.

 

From a pure technical standpoint NTFS is ways more "robust" and reliable than *any* FAT based filesystem, and as a matter of fact the good MS guys know that and in Wndows Embedded  they use TexFAT (which essentially is an overlay to exFAT) that assures transaction-safe operations:

 

https://msdn.microso.../cc907929.aspx 

 

but TexFAT is NOT implemented in Desktop OS, so all you have is pure exFAT.

 

:duff:

Wonko

 

I hear what you are saying, it's just not panning out for me.  I've been a strict NTFS user for everything for years. As I also said, I don't boot anything with NTLDR or BOOTMGR off my USB drives, I boot everything off my USB sticks using GRUB4DOS and it works fine under ExFat. Everything I've launched on my sticks using Easy2Boot has worked just as it did before. 

 

The corruption was often, and I always ejected my sticks properly. On a LOT of occasions with NTFS the stick would simply not eject stating that something was in use on the drive, despite everything I checked it with never revealing anything.  I was forced to reboot the PC in those cases - OFTEN.  A few times I have no idea what the heck corrupted it, it apparently was working fine and ejecting fine and then Windows announced on the next try that the volume was not dismounted cleanly and did a chkdsk - often much of the content was hosed.  I have taken to backing up my sticks with a sync tool to my HDDs in order to facilitate rapid recovery. 

 

2 of my sticks were especially irritating to work with.  I repeatedly reformated them, in Windows an RMPrepUSB and always had issues with mounting, unmounting and corruption.  These 2 sticks would often take 30 seconds or more just to mount and show in Explorer. Since I've reformatted to ExFat they pop up in seconds and I've yet to have any issues in a few weeks of daily use. They always dismount cleanly and quickly. The speed, especially for small files, is an incredible boost.  I was about to toss one stick in particular, it took so long to copy any small files to it that it was virtually useless. With ExFat it works just fine.

 

Yes, I was in the NTFS camp for USB drives for years.  Now I'm sticking with ExFat.  For me at least it just works. 



#10 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 14267 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 12 January 2018 - 03:02 PM

What I meant was that you cannot boot NT 4.00, 2000, XP and 2003 without NTLDR, and you cannot boot Vista and later Windows OS without BOOTMGR.

 

You can initiate the boot with grub4dos (or with *whatever else*) but you have to go through NTLDR or BOOTMGR to boot a "flat" Windows NT based system.

 

A Windows NT inside an image (hosted on exFAT) may work (some methods may work and some won't).

 

By the same token,  SETUPLDR.BIN or BOOTMGR are needed to boot a PE (unless - and again it depends on the specific method - an image is used).

 

When you use "safe removal" and you get the message that a USB device is in use - surprisingly enough - it actually means that the device is in use.

 

Detecting (and stopping/closing) the *whatever* is actually is using it may be non-trivial, still it is possible.

 

Now, as with any FAT based system the number of accesses to the device (both reads and writes) is definitely less than what NTFS *needs* and that may well explain the dramatic change (in better :thumbup: ) of behaviour on your "particular" two sticks[1], and, as hinted on the thread on MSFN it is likely that the good MS guys have not "dumbed down" exFAT as much as they have dumbed down FAT32

 

:duff:

Wonko

 

[1] If you could provide make/model and possibly also Vid/Pid it might be useful to know them



#11 Rootman

Rootman

    Frequent Member

  • Advanced user
  • 279 posts
  • Location:USA

Posted 12 January 2018 - 04:15 PM

Sorry, we're never going to agree on this.  ExFat, since I switched all 4 of my main USB sticks, surprisingly works real well for me, boots everything on all my sticks like multiple version of Win PE, Linux, Rescue utils etc - all from Easy2Boot. The sticks are snappy, mount quicker, eject every time, I can browse it faster and the performance is FAR better.  If you have never suffered from crappy performance on an NTFS formatted USB stick or issues with ejecting it then good for you.  I've increasingly had more and more issues, until, on a lark I switched one over to ExFat. It's got exactly the same content on it - I restored it from a backup of the same stick while formatted as NTFS, and works splendidly. I am simply NOT booting anything that won't boot on exfat through grub4dos / easy2boot so your repetition of your points about NTLDR and BOOTMGR are moot. I've run through all 23 ISOs and imgptns I have on my sticks - all work.  The Linux's won't read the stick once booted to, but I've yet to have this be a problem.  I seldom use them anyway. 

 

I thought perhaps some other poor souls out there might be having similar issues, if so, I suggest they give ExFat a try.  If it fails to boot, won't boot their images / ISOs / etc.  Then switch back to NTFS. It's been a good move for me.  Not having to plan on rebooting my PC because the same NTFS stick refused to eject yet again is well worth it, besides the remarkable performance boost and quick mounts. ExFat is not for everything, and perhaps not for much of anything, but is sure as hell works well for me on my USB sticks. 



#12 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 14267 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 12 January 2018 - 05:15 PM

Easy2boot uses images , not "flat" setups usually, which is exactly the point I was trying to make clear (or clearer):

 

"flat" is different from an image, an image is different from "flat"

 

... but it ends up being "m00t"[1] ... 

 

:duff:

Wonko

 

[1] note to self: never engage again in talks or discussion with Rootman



#13 Rootman

Rootman

    Frequent Member

  • Advanced user
  • 279 posts
  • Location:USA

Posted 12 January 2018 - 06:48 PM

Yes, my point since the beginning.  I use easy2boot / grub4dos to boot images, ISOs and ImgPTN files. I also use the USB sticks to store data as backups and have utilities and portable apps on them.  I was not booting from flat images and likely never will, at least from these USB sticks. I tried a Windows 10 install from an ISO converted to a ImgPTN and it works fine too.  The only gotcha I have seen against ExFat is the inability for most Linux distro ISOs to see the stick once booted as LIVE from it via Easy2Boot / Grub4ODS. No biggie, I seldom need to write back to the disk anyway, I never do persistence either.  I don't know if installing Linux from ExFat will work either.  Maybe I'll goof with it some slow afternoon. 

 

NTFS on USB was just aggravating as hell, I had messed with ExFat before and thought "Meh", nothing here that NTFS can't do and it's more universal. I was also concerned with the lack of journaling it lacked.  NTFS rights, compression or encryption never concerned me on USB sticks. I was just amazed at how much better, faster and cleaner my USB sticks worked when I switched to ExFat.  It's worth a look for anyone having issues with their sticks, inability to dismount, slow mounts, slow file transfers etc. 

 

Had someone posted something like my original post I would have tried it long ago. 



#14 steve6375

steve6375

    Platinum Member

  • Developer
  • 6736 posts
  • Location:UK
  • Interests:computers, programming (masm,vb6,C,vbs), photography,TV,films,guitars
  •  
    United Kingdom

Posted 13 January 2018 - 10:21 AM

I was doing some more tests on exFAT with E2B and found that VBox+VMUB did not seem to boot from a USB drive after installing grub4dos to the MBR of a drive prepared with RMPrepUSB (even though a real system seemed to boot OK). It turned out that I need to use the exFAT-compatible grub4dos 0.4.6a boot code, so the latest version of RMPrepUSB is now v2.1.740 which now works with exFAT better when installing grub4dos.

 

P.S. SWITCH_E2B.exe does not work with exFAT drives because the utility GetFileExtents.exe by Mark Roddy does not support exFAT drives.

 

If anyone with sufficient Visual C++ Fu would like to update the app to also work with exFAT, I would be very grateful?



#15 Rootman

Rootman

    Frequent Member

  • Advanced user
  • 279 posts
  • Location:USA

Posted 14 January 2018 - 08:59 PM

Thanks Steve, I had not noticed that the SWITCH_E2B didn't work, you can still switch back to the E2B menus from an IMGPtn by booting to the stick and choose to return to the E2B menus.  That's good to know though.  Everything else worked for me, with the exception of any Linux persistence, which I happen to never use anyway. 

 

I'm still leaving my HDD USB drives as NTFS, if for no other reason than the switch_e2b issue and the fact that it takes too long to recreate them after a format :)



#16 steve6375

steve6375

    Platinum Member

  • Developer
  • 6736 posts
  • Location:UK
  • Interests:computers, programming (masm,vb6,C,vbs), photography,TV,films,guitars
  •  
    United Kingdom

Posted 14 January 2018 - 09:12 PM

linux isos + persistence should work?



#17 Rootman

Rootman

    Frequent Member

  • Advanced user
  • 279 posts
  • Location:USA

Posted 15 January 2018 - 12:52 AM

linux isos + persistence should work?

 

I haven't messed with it much, but the couple that I used couldn't see the USB stick, I am assuming that they did not support ExFAT.  I have never used persistence so I am not concerned with it.  I am just glad I tried ExFat out, it has made working with my USB sticks pleasurable again, they mount faster in Windows, allow me to explore faster and unmount like a charm now. 



#18 steve6375

steve6375

    Platinum Member

  • Developer
  • 6736 posts
  • Location:UK
  • Interests:computers, programming (masm,vb6,C,vbs), photography,TV,films,guitars
  •  
    United Kingdom

Posted 15 January 2018 - 09:59 AM

Well once booted to linux they may not be able to see the USB drive, but you should at least be able to boot to a linux ISO and with persistence (if a persistence file + mnu is used).

 

For booting to linux, grub4dos partnew is used, so the stage 1 boot process loads the initial kernel and initrd using the BIOS and grub4dos. Stage 2 should get the files from the iso9660 partition that E2B has written to the partition table using partnew, so linux just needs to understand the cdfs and does not access the exFAT fs.

 

Similarly for persistence, the ext persistence file is mapped as a new partition entry using partnew, so again no exFAT access is required.



#19 Rootman

Rootman

    Frequent Member

  • Advanced user
  • 279 posts
  • Location:USA

Posted 15 January 2018 - 06:19 PM

Well once booted to linux they may not be able to see the USB drive, but you should at least be able to boot to a linux ISO and with persistence (if a persistence file + mnu is used).

 

For booting to linux, grub4dos partnew is used, so the stage 1 boot process loads the initial kernel and initrd using the BIOS and grub4dos. Stage 2 should get the files from the iso9660 partition that E2B has written to the partition table using partnew, so linux just needs to understand the cdfs and does not access the exFAT fs.

 

Similarly for persistence, the ext persistence file is mapped as a new partition entry using partnew, so again no exFAT access is required.

 

Cool to know, I may just goof around with it on some slow afternoon.  



#20 Rootman

Rootman

    Frequent Member

  • Advanced user
  • 279 posts
  • Location:USA

Posted 09 July 2018 - 07:06 PM

Just a 6 month update.

 

I'm still using eXFat on my USB thumb drives, and still pleased.  It just seems a lot faster and dismounts every time.  I still keep my larger USB HDDs as NTFS and just use eXFat on my smaller thumbdrives.  It works fine for me with Easy2Boot and general data storage.  I've not had ONE BIT of corruption on these sticks.  I typically had to rebuild a stick every month or 2 with NTFS.  Even though I was forced to reboot or shutdown the PC to dismount the stick the NTFS file system should still corrupt.  IN fact I just had one of my new USB HDDs using NTFS totally hose the filesystem and I had to use a recovery tool to get it back.  Strange that a format that usually gets such a bum rap actually turns out to be pretty good. 

 

If you boot Windows 'flat' images from a eXFat drive you will have issues as Wonko repeatedly brought up. If you just use them for data storage, or just use Easy2Boot they will work fine. I seldom use imgPRTNs so not being able to use Switch_E2B is not an issue, I can just reboot to the drive and use the Easy2Boot menu to flip it back. 






0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users