Jump to content











Photo
- - - - -

How to multi-partition MicroSD card for fast speed and good performance?


  • Please log in to reply
13 replies to this topic

#1 Zoso_The_Internet_Tard

Zoso_The_Internet_Tard

    Silver Member

  • Advanced user
  • 536 posts
  • Interests:An investigation is underway to determine whether Trump has any ties to America.

Posted 17 October 2015 - 06:26 PM

I just bought a Patriot 128GB class 10 MSD from Fry's Electronics a few days ago. It is exclusively for use in my Note 4, which Samsung states to be compatible with 128GB MSDs. I need to have 3 partitions:

 

1.90GB exFAT (advantage over FAT32 is larger file sizes over 4GB, which my stock modded TouchWiz ROM recognizes out of the box). I would love to use Ext4 for this partition, but it seems to be only mountable by AOSP/CyanigenMod-based ROMs, whereas TouchWiz ROMs cannot mount Ext4 on MSD without questionable/unreliable hacks. By default Android will only mount the 1st primary partition on an MSD, which is why it needs to have the largest size of the 3.

 

2. 2nd partition, 25GB formatted as ext4, which is for use with an app called Link2SD.FS must be ext4 because permissions and file ownership must be supported, which exFAT cant handle. L2SD allows to "link"/"bind" apps, odex/deodex files, app data/caches, etc to the 2nd partition, via symbolic links. This is mainly useful for games and other large apps.

 

3. 1.5GB Linux swap partition, not for use by the ROM/Android itself, but for use with running chrooted/jailed Linux installs (think of it as running a full Linux distro in a VM on Android).

 

Theoretically, this setup should be fine, but in practice I've found it be slow. Copying files to/from the 1st primary partition can take awhile, when I first set this layout up it took several hours to copy about 12GB of files from the phone's internal storage to Sd. Speeds ranged from less than an Mbyte per sec to 20. Apps linked to the 2nd partition oftentimes run sluggishly, hang for short durations, etc.

 

Partitions were created with a card reader on a PC via Gparted in Parted Magic. MBR was used. I also checked the alignment of the partitions via parted (CLI version of Gparted). All seemed fine. I dont think the SD is a counterfeit/bum, the Patriot brand is known for reliability, and Fry's isnt known to willfully sell knockoffs/faulty goods. Speed tests with various benchmark tools show the card runs at/near advertised speeds (with a margin above or below rated speeds, of course). I'm thinking this is a matter of improper alignment, not enough space at/before partitiin table start, I'm just not sure. I went with Gparted's default alignments when creating partitions.
 

I'm basically looking for MSD-specific partitioning advice, perhaps recommendations to use other filesystema if need be.

 

Thanks!

 

More info, probably not relevant:

Bootloader is unlocked, so I can install custom KitKat/Lollipop ROMs,AOSP ROMs, etc.

Latest TWRP custom recovery installed

Device is rooted with CF-AutoRoot and latest SuperSU

Rom is stock odexed TouchWiz 4.4.4 KitKat firmware released by Samsung, but with minor community mods like busybox preinstalled, zipaligned, etc. All stock Google/carrier/Samsung bloatware is present but can be easily removed

Custom kernel is installed, with support for execution of init.d scripts on boot (otherwise Link2SD wouldnt work), overclocking/underclocking, loop device support, SELinux security status is set to Permissive rather then Enforcing (Latter is KK default value as mandated by Google)



#2 Nuno Brito

Nuno Brito

    Platinum Member

  • Team Reboot
  • 10441 posts
  • Location:boot.wim
  • Interests:I'm just a quiet simple person with a very quiet simple life living one day at a time..
  •  
    European Union

Posted 18 October 2015 - 03:05 PM

Hello,

 

I've recently did similar and started using MicroSD for running the operating system as base. My impression using it for a normal desktop is plain normal. Just used plain formatting (Ext4) without additional partitions and the Linux desktop is nowhere slow. Previously, had the OS running directly from the SSD media where it would (at least in theory) be faster but I don't really notice much difference. Advertised speed on the card is 40Mb/s for read operations.

 

Didn't had the opportunity to test yet with Android or cellphones to see if there is an impact.



#3 Zoso_The_Internet_Tard

Zoso_The_Internet_Tard

    Silver Member

  • Advanced user
  • 536 posts
  • Interests:An investigation is underway to determine whether Trump has any ties to America.

Posted 22 October 2015 - 05:58 PM

I'll post the card I got a bit later, I did manage to squeeze a bit better speed by repartitioning the card according to some instructions I found on XDA (something about aligning the partitions to sector boundaries, maybe I can find the link). However, I noticed an anomaly, which is that the card's benchmarks are quite high (from my own testing), approx.matching manufacturer's claims. But real life speed is much slower. I think maybe it's because the card's first primary partition is exFAT rather than my preferred Ext4. What else might account for this huge discrepancy?

 

I found a script which allows TouchWiz ROMs to mount ext4 partitions on boot, but it has permissions issues (I have to chmod and chown a bit when first acccessing the partition or it is unwritable with lots of "permission denied" errors). Android also doesnt believe the partition is mounted (in the Storage menu) and so doesnt make it available to any PC OS for MTP access so I cant transfer files.

 

Another thing, when formatting an SD ast Ext4 for a mobile, you may want to consider creating the FS without journaling, and not reserving any of the FS for root ( reducing the number to 0 or something really low). This can be done with the mkfs.ext4, command, the specific switches for omitting these are readily available on the Web.I think the standard % reserved for root is 5. Leaving these out gives you more usuable space, and these arent any really necessary on mobile. I wouldnt consider doing this when creating an ext4 partition for a PC though (mainly for reasons of data integrity, and preventing non-root accounts from using all available space).

 

If you're into Android and want to get the most out of an MSD, you should look into F2FS (Flash Friendly File System). It's a relatively new FS developed by someone closely associated with Samsung, designed for mobile devices. It supports Linux-like permissions and ownership and other attributes associated with *nix FSes. It also benchmarks quite high and outperforms ext4 in many respects. It may turn out to be the new FS standard of the future for Android. The only thing holding it back is its' corporate associations, the community tends to be distrusting of this, and so Samsung is working to make the code as open as possible, to prove they can be trusted. On the other side of things, most people cant yet use it on their devices, because it requires both support from the ROM (whether stock or custom), as well as kernel support. If you can find a ROM that will let you format all your device's partitions as F2FS (mainly /system, /data, /cache, dalvik cache) then you wont regret it.



#4 gbrao

gbrao

    Frequent Member

  • Advanced user
  • 350 posts
  •  
    India

Posted 23 October 2015 - 03:45 AM

http://reboot.pro/to...-a-sdsdhc-card/

 

http://wiki.laptop.o..._Storage_Device



#5 Zoso_The_Internet_Tard

Zoso_The_Internet_Tard

    Silver Member

  • Advanced user
  • 536 posts
  • Interests:An investigation is underway to determine whether Trump has any ties to America.

Posted 09 December 2015 - 06:05 PM

I found some instructions that made sense at http://forum.xda-dev...592&postcount=2. But I ran into a few problems putting it into practice. I had previously partitioned my MSD using his instructions but still got bad performance and low speed (despite high benchmarks, which aren't necessarily indicative of actual everyday results). I'm still using the formula of exFAT for the first primary partition (which is what Android reads and mounts by default), ext4 for the 2nd partition for use by Link2SD, and a swap partition at the end for use by my chrooted/jailed Linux installs. I must use exFAT as opposed to FAT32 for the first partition if I wish to store files larger than 4GB, although I prefer ext4 or F2FS there instead.

 

To summarize the instructions in Pepoluan's post (for those who don't want to read it), his instructions are to put 1MB before the first partition (to align on sector 2048) and to align on the block erase size, and he gives 256 as a nice, common sense whole number by which to divide. So:

 

Last partition sector minus first partition sector plus 1 divided by 256......or:

 

X - Y + 1 / 256 = Z

 

His explanation is that the result should be a whole integer/number for optimal alignment purposes. Excuse my math, but it's one thing I thoroughly failed in school, despite acing every other subject, so if my equation is mathematically incorrect, then whatever. Even now I have no real understanding of anything beyond very basic arithmetic. Fractions/percents/decimals etc are out.

 

So....the issue I seem to be encountering is that different calculators are giving slightly different results. I really do believe my abysmal speed/performance thus far are due to incorrect alignment or failure to understand good practice partitioning concepts due to lack of knowledge.

 

I created 3 partitions in MiniTool, following the formula and checking it twice to ensure a whole number was the result, then inputting the info into Windows Calc for results. Everything seemed fine. Then I rebooted into Linux-based Parted Magic and checked the results there. PM'S Calc gives a slightly different result that has a decimal followed by number, i.e. not a whole number. I didn't change the partition scheme at that time. I then checked the results with 2 different Calc apps on my phine, one gave a whole number and the other a number with a decimal. The numbers before the decimal (to its' left) matched my previous answers.

 

Infuriated by what appears to be contradictory answers, I wiped the SD'S partition scheme and created partitions that most closely approximated my desired sizes. I have idea how to lay out partitions by sectors, etc, I only use desired size in an attempt to design my layouts. And I ideally don't want any MB wasted (unallocated) on the SD, except the 1MB at the beginning and end.

 

Last, I created partitions with Rod Smith's gdisk utility, but got similar results.

 

This is driving me nuts, I would appreciate a bit of help here.



#6 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 13576 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 09 December 2015 - 07:17 PM

To summarize the instructions in Pepoluan's post (for those who don't want to read it), his instructions are to put 1MB before the first partition (to align on sector 2048) and to align on the block erase size, and he gives 256 as a nice, common sense whole number by which to divide. So:

 

Last partition sector minus first partition sector plus 1 divided by 256......or:

 

X - Y + 1 / 256 = Z

 

His explanation is that the result should be a whole integer/number for optimal alignment purposes. Excuse my math, but it's one thing I thoroughly failed in school, despite acing every other subject, so if my equation is mathematically incorrect, then whatever. Even now I have no real understanding of anything beyond very basic arithmetic. Fractions/percents/decimals etc are out.

That would actually be:

(X - Y + 1) / 256 = Z

 

But, given that Y1, Y2 and Y3 are the start offsets of partition #1, #2 and #3 respectively, it would be easier to check that:

Y1/256=Z1=INT(Z1)

Y2/256=Z2=INT(Z2)

Y3/256=Z3=INT(Z3)

...

It doesn't really-really matters that the last partition size is a multiple of 256, as since it is a last partition there is no risk that the following partition (which doesn't exist) will start non-aligned.

The given post assumes (as most tools will behave like that) that a partition will start immediately after the end of the preceding one, but what actually matters for speed access is that the first sector of a volume is correctly aligned, no matter where the previous one ends...

 

It works for your first partition as 2048/256=8, what are the values of the start LBA address of your other partitions?

 

:duff:

Wonko



#7 Zoso_The_Internet_Tard

Zoso_The_Internet_Tard

    Silver Member

  • Advanced user
  • 536 posts
  • Interests:An investigation is underway to determine whether Trump has any ties to America.

Posted 09 December 2015 - 07:55 PM

@ Wonko: The instructions were that *every* partition should be evenly divisible by 256, not just the last one. Does this mean that he is entirely incorrect?

 

What do you mean by Y1, Y2, Y3? I was only using letters to symbolize numbers, like in an equation .

 

It does make sense the the starting sector of a partition matters, but why not the last sector?

 

What is Int (Z23)?

 

Does choice of files stem really make a big difference?

 

Should the end of the disk also have 1MB unallocated (Gparted appears to do so by default, not sure about MiniTool and gdisk).

 

I'm also debating on MBR vs GPT. Since the card isn't bootable, it shouldn't be a consideration. I only need 3 partitions. But GPT has better support for error correction/reduncancy, etc. Android is perfectly capable of reading/mounting an SD that is GPT, however, I use many root tools (mainly terminal emulators, benchmarking apps, etc) that appear to only work with MBR. Obviously this is a matter of choosing the right apps/tools.

 

I have no idea of the starting LBA addresses of each partition, what tool should I use to obtain that info? I keep changing up the sizes because of seeming contradictory results.



#8 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 13576 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 09 December 2015 - 08:13 PM

Let's see if we can abstract from math/symbolism.

 

The post you referred to is largely correct, but let's simplify with just two partitions.

The second partition (when created by most automated tools) will begin immediately after the end of the first partition.

So, if the first partition starts aligned and it's size is a multiple of 256 sectors, it is the same thing to use the size of the partition + 1 sector than using the beginning of the second partition, the latter besides being easier is correct even in the case (logical volumes inside extended as an example) when the second partition does not start immediately after the first one.

 

What may slow down access is if the start of a partition is not a multiple of the SD "page size".

If the end of the partition is not a multiple of the SD "page size" it might only make a difference when accessing the very end of the partition (which is something never happens in normal operation).

 

LBA start address or "partition start offset" is what you referred to as "first partition sector", in the case of your first partition it is 2048.

What are the "first partition sectors" of the 2nd and 3rd partition?

 

:duff:

Wonko



#9 Zoso_The_Internet_Tard

Zoso_The_Internet_Tard

    Silver Member

  • Advanced user
  • 536 posts
  • Interests:An investigation is underway to determine whether Trump has any ties to America.

Posted 09 December 2015 - 08:42 PM

Perhaps you misinterpreted Pepoluan's post. What he meant is that you need to subtract the starting sector of each partition from the end sector of the same partition, not the first sector of the next partition. His calculations are on a per-partition basis and doesn't take the next partition into consideration.

 

It seems the starting sector of a partition must be even rather than odd, since 256 won't divide evenly into an odd number.

 

The Arch Wiki states to divide by 8 rather than 256, is 256 necessarily a better choice? 8 divides into 256 evenly, so it would be a prime of 256? What is the reasoning behind what number to divide by?

 

If Google is correct then the page size is one and the same as the 'allocation unit'.............

 

I don't have the LBA info, I'll have to create test partitions first.



#10 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 13576 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 10 December 2015 - 02:49 PM

Perhaps you misinterpreted Pepoluan's post. What he meant is that you need to subtract the starting sector of each partition from the end sector of the same partition, not the first sector of the next partition. His calculations are on a per-partition basis and doesn't take the next partition into consideration.

Let me assure you that I did not misinterpret it.

 

As said his approach will work but is more complicated than needed and is not accurate in all cases, as said it will be accurate ONLY if each partition starts immediately after the previous one.

 

All tools will provide the info as two values:

Start LBA (which you can and should read as "sectors before")

Num Sectors (which is the number of sectors of the partition).

 

Let's see if I can give you a couple examples, imagine two partitions, each exactly 10 megabytes or 20480 sectors (20480*512=.10,485,760)

First partitition Start LBA 2048

First partition number of sectors 20480

First partition last sector 2048+20480-1=22528-1=22527

Second partition Start LBA 2048+20480=22528

Second partition number of sectors 20480

Second partition last sector 22528+20480-1=43008-1=43007

Third partition Start LBA 22528+20480=43008

...

The calculation in that post for first partition:

(22527-2048+1)/256=20480/256=80 Ok, it is an integer

 

The calculation in that post for second partition:

(43007-22528+1)/256=20480/256=80 Ok, it is an integer

 

The proposed calculation:

First partition: 2048/256=8 Ok, it is an integer

Second partition:22528/256=88 Ok, it is an integer

 

In the above case both calculation are valid :), but now imagine that there is a gap between first and second partition:

 

First partitition Start LBA 2048

First partition number of sectors 20480

First partition last sector 2048+20480-1=22528-1=22527

Here there is a gap of - say - 63 sectors.

Second partition Start LBA 2048+20480+63=22591

Second partition number of sectors 20480

Second partition last sector 22591+20480-1=43071-1=43070

Third partition Start LBA 22591+20480=43071

 

 

The calculation in that post for first partition:

(22527-2048+1)/256=20480/256=80 Ok, it is an integer

 

 

The calculation in that post for second partition:

(43070-22591+1)/256=20480/256=80 Ok, it is an integer <- THIS DOES NOT MEAN that the partition is correctly aligned.

 

 

The proposed calculation:

First partition: 2048/256=8 Ok, it is an integer

Second partition:22591/256=88,24609375 It is NOT an integer (the partition is not correctly aligned)

 

I hope that now the concept is more clear.

 

 

About the size to divide by as mentioned in the given post, the 256 is on the cautious side, it correspond to an internal SD card "page size" of 128 Kb, the largest that has been observed, a specific card may have a smaller one.

If you prefer 256 will be ok on *any* device while on some devices 128, 64, 32, 16 or 8 might do as well.

 

The 8 you refer to is connected to the 4096 bytes size (8*512=4096 or 1*4096=4096) which is a value which is a good one for hard disks and SSD's and that - coincidentally - is commonly used also as cluster size on NTFS.

 

The 2048 to which you align the first partition is what is used by default in modern windows systems but it is arbitrary and even more conservative than 256, and "automated" tools will normally create partitions that are multiple of 2048 or 1 Mb in size.

 

So, if you prefer it, 8 is a "bare minimum", 256 is the max observed specifically on SD cards, 2048 is a (much larger) value that will be "good on anything".

 

:duff:

Wonko



#11 Zoso_The_Internet_Tard

Zoso_The_Internet_Tard

    Silver Member

  • Advanced user
  • 536 posts
  • Interests:An investigation is underway to determine whether Trump has any ties to America.

Posted 25 August 2016 - 04:08 PM

So, I just tried using the SD Formatter tool on my 200GB MicroSD. It is the official tool for formatting MSDs. I noticed that it placed a 32MB unallocated space before the 1st partition. I can only imagine that this must have something to do with alignment or block size. It really agitates me to see unallocated space, it feels as if my space isn't being used to the fullest. However, I still need a minimum of 3 partitions. SD Formatter does not support creating more than one partition. Should I place a 32MB unallocated space in between each partition, or is this only necessary for the first partition? I plan to shrink the 1st partition with GParted. It also formatted the 1st partition with exFAT as the filesystem, I plan to reformat it as Ext4 (since my CyanogenMod-based ROM can mount that just fine), the 2nd partition will be Ext4 as well, and the 3rd will be swap. There may also later be a 4th Ext4 partition in which I will try to install a chrooted Linux OS with Linux Deploy. I cannot have more than 4 partitions unless I use GPT (and really have no need for more than that), SD Formatter uses MBR as the partition table.

 

Thanks!



#12 steve6375

steve6375

    Platinum Member

  • Developer
  • 6597 posts
  • Location:UK
  • Interests:computers, programming (masm,vb6,C,vbs), photography,TV,films,guitars
  •  
    United Kingdom

Posted 29 August 2016 - 12:24 AM

When dealing with solid state devices, any partition alignment (in terms of cylinder/heads, etc) is meaningless.

 

SSD devices use flash memory and to write to flash memory, the controller needs to perform 'page erase cycles' on blocks of memory before it can overwrite that memory. So although 'access times' are negligible, the  page-read+page-erase+page-write times are not.

 

When a file is written to media, the OS writes a file in clusters. These clusters therefore need to be aligned nicely on the flash memory - if possible all clusters should be within the same erase-page block for optimum efficiency.The erase-page size is normally in the order of MBs (e.g. 8MB) but it varies from device to device.

 

For instance, an exFAT filesystem may have 64 sectors per cluster (=32K clusters). So what is important, is that when the clusters are written to the flash memory, each cluster is nicely aligned. So it is the position of clusters in a filesystem that is important, and not (necessarily) the position of the start of the partition (although the start address of a partition may affect the cluster positions). 

 

The efficiency of the OS also comes into play and this may vary depending of the type of filesystem being used. An OS could write each cluster of a file separately - e.g.

File1=96K - OS commands to flash memory are:

Write 1st cluster (32K)  (controller performs page-read+page-erase+page-write cycle)

Write 2nd cluster (32K) (controller performs page-read+page-erase+page-write cycle)

Write 3rd cluster (32K) (controller performs page-read+page-erase+page-write cycle)

 

If one of these 32K clusters crosses an erase page boundary however, two page-read+page-erase+page-write cycles will be needed.

 

Alternatively, an OS could buffer the 96K and send just one write command to the flash memory controller

Write 96K (controller performs page-read+page-erase+page-write cycle)

 

Again, if the 96k crosses an erase page boundary however, two page-read+page-erase+page-write cycles will be needed.

 

So, depending on how the OS is buffering writes (or not) and what the actual cluster LBA address is, you can get variations in performance.

 

In particular, writing lots of small files will give poor times.

 

https://blogofterje....-fs-on-sd-card/

 

P.S. You can use RMPrepUSB - Drive Info - P1  to view the file cluster start position, etc. for FAT32, NTFS and exFAT volumes (but not ext volumes).


Edited by steve6375, 29 August 2016 - 12:29 AM.


#13 naruto1979

naruto1979
  • Members
  • 6 posts
  •  
    Philippines

Posted 4 weeks ago

very nice discussion. Thanks guys 



#14 Zoso_The_Internet_Tard

Zoso_The_Internet_Tard

    Silver Member

  • Advanced user
  • 536 posts
  • Interests:An investigation is underway to determine whether Trump has any ties to America.

Posted 4 weeks ago

I've done my best so far to ensure that my MSD is properly aligned, but write speeds still suck. My MSD's first partition is ext4, which my chosen Android ROMs support. The 2nd is F2FS, also supported, for linking apps to MSD. The 3rd is Linux Swap.




0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users