What happened to this experiment? I downloaded RMPrepUSB, and even the docs belonging to version 2.1.647 (the one referenced in this thread) do not mention the alignment question. Actually formatting a USB stick with the current version seems to produce XP-friendly formatting, offset of 32,256 bytes. I'm good with the Command Line, if that's the most direct way to do it. Yes, I'm willing to sacrifice most of a megabyte of 8 GB, to gain 20% in file transfer speed.
RMPrepUSB - faster FAT32 write access on Flash memory drives
#26
Posted 21 January 2013 - 08:20 AM
#27
Posted 21 January 2013 - 08:50 AM
See Version History on web site
#28
Posted 21 January 2013 - 05:07 PM
Version History, of course. Sorry, I didn't look there. I downloaded the latest version 2.1.654, which formatted a USB stick with an offset of 32,256 bytes. I don't see 2.1.647, the one announced to offset by 1 MB, on your download page. Would the intermediate version 2.1.648 work for my purpose of getting a 1 MB offset? I did find the interface confusing. When it failed to do what I wanted with XP (NTLDR) formatting, I tried Win7 formatting (1 MB offset, right?), but in fact RMPrepUSB did something quite different, creating two partitions, the main one having an offset again of 32,256 bytes. I guess my trouble starts with trying to use a sophisticated tool for a simple (but valuable!) purpose. The operation takes seconds, but learning how to do it takes hours. I did not mention it in the original question, but the computer runs XP, SP3.
#29
Posted 21 January 2013 - 05:40 PM
Any version after 647 has the FAT32 optimisation change.
You get 2 parititions if you tick the 'Boot as HDD (C: 2PTNS)' checkbox (not surprisingly)! This is recommended as some BIOSes won't boot from a USB flash drive that has a single partition - the 2nd partition is just a dummy one to force those BIOSes into thinking it is an HDD and booting it!
NTFS optimisation is already done by Windows (the Windows DLL is used to format the drive) - speed optimisation in RMPrepUSB is done for FAT32 only - it is not needed for NTFS as it should be already fairly optimum. I just start the partition at 2048 which may improve things for a flash drive.
#30
Posted 21 January 2013 - 05:47 PM
Thanks. So is there a way for me to format the USB stick to FAT32 with a 1 MB offset?
#31
Posted 21 January 2013 - 05:49 PM
It should be doing this. The 1st cluster should be at a modulo 1Mb offset (not the partition) - how are you checking the format and what are you expecting?
#32
Posted 21 January 2013 - 05:56 PM
I deleted the partition, then ran RMPrepUSB 2.1.654 :
msinfo32 for Kingston.
Description Disk drive
Manufacturer (Standard disk drives)
Model Kingston DataTraveler 2.0 USB Device
Bytes/Sector 512
Media Loaded Yes
Media Type Removable media other than	floppy
Partitions 1
SCSI Bus Not Available
SCSI Logical Unit Not Available
SCSI Port Not Available
SCSI Target ID Not Available
Sectors/Track 63
Size 7.45 GB (8,003,197,440 bytes)
Total Cylinders 973
Total Sectors 15,631,245
Total Tracks 248,115
Tracks/Cylinder 255
Partition Disk #1, Partition #0
Partition Size 7.46 GB (8,010,809,856 bytes)
Partition Starting Offset 32,256 bytes
#33
Posted 21 January 2013 - 06:11 PM
"Partition Starting Offset 32,256 bytes"
I was expecting a number 1,xxx,xxx rather than 32,256. Like on the computer's msinfo32 info for the main hard drive, a SSD:
Partition Disk #0, Partition #0
Partition Size 107.42 GB (115,341,262,848 bytes)
Partition Starting Offset 1,048,576 bytes
Am I barking up the wrong tree?
Edited by Jonathan Berry, 21 January 2013 - 06:12 PM.
#34
Posted 21 January 2013 - 06:13 PM
That does not show it!
Use RMPrepUSB - Drive Info - type in P1.
See early posts in the discussion.
e.g.
First FAT begins at LBA 1130
Second FAT begins at LBA 16945
Root Directory begins at LBA 32760
First file data (cluster 0) begins at LBA 32768
The first file data (cluster 0) should be a multiple of 2048 (1MB) - this is the change to speed up FAT32 writes on a USB flash drive.
- Motasem likes this
#35
Posted 21 January 2013 - 06:25 PM
Thanks. I click "Drive Info", put "P1" in the textbox and press OK. A file USBInfo.txt is created and opened. At the bottom is this:
First FAT begins at LBA 180
Second FAT begins at LBA 15446
Root Directory begins at LBA 30712
First file data (cluster 0) begins at LBA 30720
Is that a desired result? 32768 (your post above) = 30720 (my result) + 2048 (1 MB) so I would guess that it is what was wanted. I was succeeding when all the time I thought I was failing !?
Edited by Jonathan Berry, 21 January 2013 - 06:28 PM.
#36
Posted 21 January 2013 - 06:26 PM
Yep - 2048 * 15 = 30720
#37
Posted 22 January 2013 - 12:01 PM
Does this utility has any benefits for an SDXC 64GB microsdcard? or for the lower sized 32GB microsd ? thanks for any replies.
#38
Posted 22 January 2013 - 12:04 PM
Does this utility has any benefits for an SDXC 64GB microsdcard? or for the lower sized 32GB microsd ? thanks for any replies.
Probably, although I can't say how much. I depends on what you used to format it with previously and what filesystem it has (FAT32/NTFS???)
#39
Posted 22 January 2013 - 12:39 PM
POPULAR
For anyone new to this discussion, here is the theory.
When writing to a USB Flash drive, the flash drive contoller circuitry must do a read/erase/write cycle on a whole page as individual sectors cannot be erased in flash memory. Let us assume a 16k erase page size for this example.
Files consist of groups of clusters - e.g. 4k clusters. Now consider a filesystem that accesses a disk in clusters (CL). What if the clusters are arranged like this:
[Erase page block 1][Erase page block 2][Erase page block 3]
[CL1][CL2][CL3][CL4][CL5][CL6][CL7][CL8]
Notice that CL3 overlaps an erase page boundary. So when the OS writes to cluster 3, the flash controller has to perform a read/erase/write cycle on Erase page block 1, and then another read/erase/write cycle on Erase page block 2 for the 2nd half of the cluster. The same is true of CL7. Thus we have caused two extra time-consuming read/erase/write cycles.
Now what if we ensure that a cluster ends on a page boundary - we get:
[Erase page block 1][Erase page block 2][Erase page block 3]
[CL1][CL2][CL3][CL4][CL5][CL6][CL7][CL8]
If each cluster is written separately we get 8 r/e/w cycles instead of 10.
To be honest I was expecting only a marginal (if any) improvement in file write times (especially as the above case is greatly simplified and exaggerated!), but when I measured the difference under Windows in practice, it was quite significant!
Result in post #2
Edited by steve6375, 23 April 2017 - 04:00 PM.
- Brito, ilko, Porfyr and 4 others like this
#40
Posted 22 January 2013 - 08:47 PM
[Erase 1 page block][Erase 2 page block][Erase 3 page block] [ CL1 ][ CL2 ][ CL3 ]
Here is russian dicussion about cluster aligning on block boundary. There is also dfb64 utility for block size searching.
#41
Posted 22 January 2013 - 09:19 PM
Probably, but the erase page size is not easily known!
#42
Posted 23 January 2013 - 12:28 AM
Probably, although I can't say how much. I depends on what you used to format it with previously and what filesystem it has (FAT32/NTFS???)
It was formatted with Ubuntu Gparted app, as FAT32 for an Android ICS tablet. it has 2 primary partitions, the first and biggest is the FAT32 (49GB) and the second smaller as EXT4 (13GB). the total after formatted is around 59GB out of the 64GB that it's advertised (after converting 64gb to the real formatting)
#43
Posted 23 January 2013 - 09:20 AM
OK, well try doing some write benchmarks and then reformat using RMPrepUSB Fat32
#44
Posted 23 January 2013 - 12:07 PM
This is a great finding. Always creating my partitions 'aligned' with commercial tools, I have actually wondered about this for a long time knowing that partitions store metadata at the beginning but not previously knowing a way to check the "first file data sector" of a partition. I think this means that all the current partition utilities out there like "Paragon Alignment Tool" - and even all the partition alignment advice on the forums about using tools like Gparted is bunk and wrong??? Because everyone is telling you to use partition tools such as Gparted or Partition Guru, etc. to start your partition at a sector divisible by 2048 or 4096. However - it does not take into account the beginning file tables and log tables which are NOT sized in multiples of sector size or page size. So although the partition starts at an aligned sector - because of the beginning partition metadata - the data ends up all being un-aligned!
Following are the results from RMPrepUSB "Drive Info" - P1 for an "aligned" partition created with a commercial partition program vs. the partition created with RMPrepUSB new version.
Created with Partition Guru 4.20 - FAT32 at aligned sector 4096 (Gparted would give similar results):
First FAT begins at LBA 4134
Second FAT begins at LBA 7939
USB Sector 63
So the "aligned" partition created with Partition Guru 4.20 starts at sector 4096 - but the first file cluster starts at 11752 which is unaligned.
Edited by robertcollier4, 23 January 2013 - 12:29 PM.
- Motasem likes this
#45
Posted 23 January 2013 - 12:22 PM
The proof of the pudding is in the eating.
I tested using Windows Xcopy to write lots of small files to the target flash drive (and repeated 10 times and then repeated this each 3 times after a fresh partition/format each 10 times - with AV off).
i.e.
wipe/partition/format
Xcopy > target SSD
(repeat Xcopy 9 times thus overwriting files)
wipe/partition/format
Xcopy > target SSD
(repeat Xcopy 9 times thus overwriting files)
wipe/partition/format
Xcopy > target SSD
(repeat Xcopy 9 times thus overwriting files)
Maybe you can try some SSD tests and post results?
#46
Posted 23 January 2013 - 12:42 PM
NTFS optimisation is already done by Windows (the Windows DLL is used to format the drive) - speed optimisation in RMPrepUSB is done for FAT32 only - it is not needed for NTFS as it should be already fairly optimum.
Windows Vista and later create partitions at starting sector 2048 (at 1 MB boundaries). However - then you have the partition metadata for NTFS.
Do you know of a way to check "First file data sector" for a NTFS drive? I tried RMPrepUSB > Drive Info > P1 and got:
USB Sector 4096
Edited by robertcollier4, 23 January 2013 - 12:56 PM.
#47
Posted 23 January 2013 - 01:09 PM
Also, just thought I would point out, Partition Guru 4.20 does display an entry for "Data start sector" for FAT32 drives but it is a different value of 8121 than the RMPrepUSB>"Drive Info">P1 "First file data" of 8192.
For the same drive, RMPrepUSB says:
COMMAND LINE: DRIVE=1 USBINFO USBSTART=P1 SURE Accessing Drive 1 - "SanDisk SDDR-113" (2,002,255,872 bytes) USB Sector 63 First FAT begins at LBA 558 Second FAT begins at LBA 4371 Root Directory begins at LBA 8184 First file data (cluster 0) begins at LBA 8192
and Partition Guru 4.20 says:
Edited by robertcollier4, 23 January 2013 - 01:13 PM.
#48
Posted 23 January 2013 - 01:15 PM
Root Directory seems to be 63 out - perhaps they forgot to add on the partition offset?
They use a Data Start Sector = Root Directory, however the first two clusters are unused, so the first file data sector will be two clusters past the Root Dir sector
#49
Posted 23 January 2013 - 01:42 PM
The MFT is usually not at the beginning of the volume, on a large volume it is at the 3GB position. NTFS will use any free cluster for a file, so it could be before the MFT or after.
The main thing would be to ensure that any file starts at a sector number (LBA number) that is divisible by 8.
If you install grub4dos to the NTFS formatted drive and then go to the grub4dos console and type
blocklist /$MFT
you can see where it starts and how many sectors it takes up. Or use it on any file on the drive (so you can just put one file on the drive and use blocklist on that?).
Note that blocklist seems to have a problem giving a result with some files (may be their sector position is many GB onto the drive and the number causes an overflow?).
for blocklist /$mft I get 6291456+39936, so 6291456 is divisible by 8.
[Edit] the grub4dos blocklist command returns the logical offset, so you need to add the start of the partition to 6291456. In most cases the first partition, if NTFS, will start at 2048 (which is divisible by 8).
Edited by steve6375, 23 April 2017 - 04:12 PM.
#50
Posted 23 January 2013 - 01:55 PM
.
Firefox Lazarus Form Recovery Extension
.
https://addons.mozil...-form-recovery/
(damn I just hit backspace and my focus was apparently off editing my post and Firefox went back to the previous page, then when I went forward all my typing was lost )
Edited by vocivoci, 23 January 2013 - 01:56 PM.
1 user(s) are reading this topic
0 members, 1 guests, 0 anonymous users