Jump to content











Photo
- - - - -

imdisk: can't format in exFAT


Best Answer Olof Lagerkvist , 14 August 2019 - 02:54 PM

Disk volumes cannot have any kind of size in bytes like that. It needs to be a multiple of at least the sector size. So, 197916303 is not a valid size for a disk volume. Try for example 197914624 instead.

Go to the full post


  • Please log in to reply
12 replies to this topic

#1 Azev

Azev

    Member

  • Members
  • 34 posts
  •  
    Brazil

Posted 14 August 2019 - 02:32 PM

Hi!

 

I'm trying to create a ramdrive with the command bellow but when I specify a custom size in bytes the drive cannot be formated in exFAT:

imdisk -a -s 197916303 -m U: -o rem -p "/fs:exFAT /q /y"

cannot format in exFAT: "The disk is too large to format for the specified file system."

imdisk -a -s 200M -m U: -p "/fs:exFAT /q /y"

will format in exFAT

 

 

why exFAT:

 

- NTFS will use extra space for $MFT and uses more CPU

- FAT: filesize limitation issue

 

 

 



#2 Azev

Azev

    Member

  • Members
  • 34 posts
  •  
    Brazil

Posted 14 August 2019 - 02:43 PM

The purpose is to load a large movie file (~4Gb) into ram to avoid constant disk I/O and HDD temperature increasing.

External HDDs tend to increase temperature above 45ºC very quickly.

With this technique, after the movie is loaded into RAM the HDD can spin down and cool off.


Edited by Azev, 14 August 2019 - 02:48 PM.


#3 Olof Lagerkvist

Olof Lagerkvist

    Gold Member

  • Developer
  • 1448 posts
  • Location:Borås, Sweden
  •  
    Sweden

Posted 14 August 2019 - 02:54 PM   Best Answer

Disk volumes cannot have any kind of size in bytes like that. It needs to be a multiple of at least the sector size. So, 197916303 is not a valid size for a disk volume. Try for example 197914624 instead.


  • Azev likes this

#4 Azev

Azev

    Member

  • Members
  • 34 posts
  •  
    Brazil

Posted 14 August 2019 - 02:58 PM

Hmm interesting. Is there a formula to get the right number? must be divisible by 1024 or something?

 

Say the movie file have 197916303 bytes. How can I find the size in bytes that can hold the movie file int it?


Edited by Azev, 14 August 2019 - 02:58 PM.


#5 v77

v77

    Silver Member

  • Team Reboot
  • 602 posts
  •  
    France

Posted 14 August 2019 - 03:12 PM

As the default sector size of imdisk volumes is 512 bytes, it should be something like
volume_size = (file_size + 511) & (!511)

This syntax is used in both C language and batch (see the help for set /A ...).
But you may have to add some KB because even exFAT requires a few sectors by default.


  • Olof Lagerkvist likes this

#6 Azev

Azev

    Member

  • Members
  • 34 posts
  •  
    Brazil

Posted 14 August 2019 - 03:47 PM

I code in NodeJS (javascript).

some math:

 

197914624 (Olof's) / 512 = 386552

197916303 (filesize) / 512 = 386555.279296875

 

Then I tried:

386560 * 512 = 197918720 (exFAT format ok)

386561 * 512 = 197919232 (exFAT format ok)

386555 * 512 = 197916160 (exFAT format ok)

 

So I'm guessing it can be any number of bytes as long it is multiple of 512 (sector size)

 

I will make some tests with this formula:

(int(fileSizeInBytes/512) + 2048) * 512

the "+2048" is to add an extra 2048 bytes extra just in case


  • Olof Lagerkvist likes this

#7 Azev

Azev

    Member

  • Members
  • 34 posts
  •  
    Brazil

Posted 14 August 2019 - 04:02 PM

Perfect. The formula works.

 

Thank you so much team!

 

The script creates a ramdisk to fit exactly the movie file, then copy the movie file to it, then start the playback, then unmount the the ramdisk when the player is closed.

 

Copying the whole movie (~4Gb) takes under 20s.


  • Olof Lagerkvist likes this

#8 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 16066 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 14 August 2019 - 04:13 PM

@v77

 

It would be interesting to know how much is "some KB".

 

I.e., you can't have an exact formula and "some KB" in the same post.

 

And the formula is anyway "wrong", in the sense that what a file occupies on a filesystem is clusters (and not sectors).

 

@Azev

The size of 197916303 is the actual file size, but if you right click on the file-> Properties you will find two sizes listed, one is file size, the other one is file size on disk, you need to use the latter as base (no need for formulas).

In the best case the result will be "exact" (if the volume it resides on is a "normal" NTFS with 4 KB clusters) in the worst case it will be a little smaller than needed (if the volume has smaller clusters), but even in the worst case it will be off at the most of "a few KB", not a problem since you have to add "some KB" anyway to get the parameter to pass to IMDISK.

 

BUT, there is another problem exFAT (by design) is intended for large files/large volumes/whatever, so essentially it is "wasteful" tending to have "largish" cluster sizes:

https://support.micr...-fat-and-exfat 

your file is below 256 MB so it is fine with 4 KB clusters, but if another video crosses the 256 MB "boundary" you will need a bigger than 256 MB volume, and then the default cluster size will become 32 KB.

 

So, an exact formula will depend also on cluster size (which in itself depends on size of the filesystem, which depends on size of the file).

 

You need to make a formula based on clusters (minimal unit addressable by the filesystem) and not on sectors (minimal unit addressable of a block device).

 

The 2048 * 512 = 1048576 bytes (not 2048 bytes) extra are not "enough KB" however, even if they work in this single example.

 

But then, unlike FAT 12/16/32 File Allocation Tables in exFAT are not  really-really fat's but some form of pointers to (very like NTFS) a bitmap.

 

So, unlike FAT12/16/32, it is not just a matter of counting the clusters and calculate the size of the FAT(s) and - and this may depend on the OS you are running - it has to be seen if you will be making exFAT or TexFAT volumes (basically 1 or 2 copies of the FAT) see :

https://www.sans.org...le-system-33274

 

all in all, if you want an exact formula you will have to do a lot of research and tests.

 

If you are OK with an approximate formula, I would try with:

 

(int(fileSizeInBytes/4096) * 1.03) * 4096

 

The 1.03 (which basically means "consider roughly 120 bytes of indexing for each cluster in the filesystem") can of course be tuned by making a few tests, it's only a starting point, probably a 1.02 (i.e.roughly 80 bytes per cluster) will be enough, possibly even less.

 

But since all in all we are talking of 200-300 MB volumes, it would make little sense to go very tight, the difference would be in the order of magnitude of 2 or 3 MB.

 

If it is 4 GB sized files (and volumes) it goes in the 40 MB approximation, which might start to have some (very little) relevance.

 

:duff:

Wonko



#9 Azev

Azev

    Member

  • Members
  • 34 posts
  •  
    Brazil

Posted 14 August 2019 - 04:24 PM

@Wonko

 

Indeed. Instead of

(int(fileSizeInBytes/512) + 2048) * 512

I will use

int(fileSizeInBytes/512) * 512 + 2048


#10 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 16066 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 14 August 2019 - 04:35 PM

 

@Wonko

 

Indeed. Instead of





(int(fileSizeInBytes/512) + 2048) * 512

I will use





int(fileSizeInBytes/512) * 512 + 2048

Well, than it cannot possibly work.

 

Read my post above.

 

Only the VBR on exfat is 12 sectors i.e. 12*512=6144 bytes.

 

In the case of 197918720 bytes, 197918720 / 4096 = roughly 50,000 clusters, even the "sheer" 2 bytes per cluster of a single FAT will be 100,000 more bytes needed

 

:duff:

Wonko



#11 Azev

Azev

    Member

  • Members
  • 34 posts
  •  
    Brazil

Posted 14 August 2019 - 04:42 PM

It IS working. But there is 1% free extra space.

 

I tested with different files size

~300mb

~4gb

~700mb

~2gb

 

ydccBp1.png

All of them work as expected. As soon the player is closed the drive is removed.

imdisk -D -m U:

Edited by Azev, 14 August 2019 - 04:58 PM.


#12 v77

v77

    Silver Member

  • Team Reboot
  • 602 posts
  •  
    France

Posted 14 August 2019 - 04:54 PM

That will remove all the fun, but there is another solution: install the ImDisk Toolkit, and create a dynamic ramdisk (large enough to hold any video file).



#13 Azev

Azev

    Member

  • Members
  • 34 posts
  •  
    Brazil

Posted 14 August 2019 - 05:04 PM

@v77 I though of that but it must be all a command line process.

If imdisk command line had this option I would use it.






1 user(s) are reading this topic

0 members, 1 guests, 0 anonymous users