Jump to content











Photo
- - - - -

Dokan + 7zip = mount many possible archives as a logical drive

dokan 7zip

  • Please log in to reply
39 replies to this topic

#1 erwan.l

erwan.l

    Platinum Member

  • Developer
  • 3041 posts
  • Location:Nantes - France
  •  
    France

Posted 17 July 2019 - 05:16 PM

Hi Gents,

Following this post where I used Dokan + Libfns to allow one to mount a logical drive actually pointing to a nfs export, I have now decided to give it a go with Dokan + 7zip.

Indeed 7zip supports many formats (7z, zip, tar, iso, wim, gz, jar, squashfs, etc), I though it would be nice to marry the two and allow one to mount an archive as a logical drive (x:\) or a folder (c:\mount\).

For now, only read operations are supported.

sevenzip_dokan run without arguments will give the possible options.

Below a simple command line to mount an archive on X:
Sevenzip_DOKAN.exe /r test.zip /l x

 

Below a simple command line to mount an archive on c:\mount\
Sevenzip_DOKAN.exe /r test.zip /l c:\mount\

Both the source and the binary are available here.

Cheers,
Erwan

kgvjGiX.png


  • Hunter23071985 likes this

#2 Guest_AnonVendetta_*

Guest_AnonVendetta_*
  • Guests

Posted 18 July 2019 - 01:03 AM

Nice idea, but what good is this if it's read-only?



#3 Blackcrack

Blackcrack

    Frequent Member

  • Advanced user
  • 458 posts
  •  
    Germany

Posted 18 July 2019 - 05:56 AM

+1 also good for an idadmin.exe as snapin in the wl-nfs.dll or maybe a extra snapin folder, for wl-7zip.dll what i have speak about here :

http://reboot.pro/to...ndows/?p=212189
(updated the post/Idea/suggestion)

because you can have also id's in the 7zipfs ..
therewith have you the possible to have 2 snapins as start ..

therewith it's it set, id you developing the id system + plugins/snapins,
for the community to developing more plugins, like wl-sftp.dll and other fs-drivers for different
filesystems in WinNT to use.. as example also wl-ext4.dll to use also Linux fs like ext1-4 aaand...
if later maybe exist, an wl-drive.sys as driver for loading in the WinNT system at boot the whole id-system
for Linuxuser, therewith could it be possible to become more important.. just as small explain why i stay behind so permanently for you ;)
this can be a big thing !

so back to snapin wl-7zip.dll for idadmin.exe :) love the idea !

Nice idea, but what good is this if it's read-only?


i guess not....
but in other hand, let run setups or so, without decompressing, like a CD or an other ro only disk/space..
think on Mac - dmg' ro'ly, maybe it is also given to open mac disk images therewith with the UID/GID if i think on Apple Macintosh too :)


therewith it's the name setting up LinuxWindows Tools this makes possible to use this tool for Linux systems (LinuxtoWindows)
or maybe L2W-Tools and Mac is today also a Linuxdistro... therewith it is given to be the name right ..
and the "oldschool"-Name L2W is for all Old-School'er readable and know what's mean ..


@erwan.l different questions because i be not a programmer :

  • is this Sevenzip_DOKAN.exe or *DOKAN* possible to make a dll with it ?
  • it is possible to use the 7zip *.dll's with a headprogramm, who names maybe idadmin.exe or L2Wadmin.exe with a wll-7zip.dll snapin...

best regards
Blacky



#4 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 16066 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 18 July 2019 - 11:33 AM

XP (SP2, and quite proud of it):

dokanctl.exe is not a valid Win32 application

 

:duff:

Wonko



#5 erwan.l

erwan.l

    Platinum Member

  • Developer
  • 3041 posts
  • Location:Nantes - France
  •  
    France

Posted 18 July 2019 - 11:38 AM

XP (SP2, and quite proud of it):

dokanctl.exe is not a valid Win32 application

 

:duff:

Wonko

 

Ouch, XP SP2...You are tough player...

I must confess that i usually start my tests with Win7.

 

Have a look here : https://github.com/d...hoose-a-version .

Looks like you should install version 0.6 (i use a version 1.x).

But the page also states that after 0.8 code, compatibility is not supported with previous versions.

 

I'll have revive an old XP VM and see how it goes.



#6 erwan.l

erwan.l

    Platinum Member

  • Developer
  • 3041 posts
  • Location:Nantes - France
  •  
    France

Posted 18 July 2019 - 12:01 PM

About XP : I took a look at version 0.6.0 which does install fine on XP.

 

But... Dokan was a complete different solution back then which means I myself will only be able to develop Dokan solutions starting with Windows 7 and using Dokan version >= 0.8.0.



#7 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 16066 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 18 July 2019 - 02:41 PM

About XP : I took a look at version 0.6.0 which does install fine on XP.

 

But... Dokan was a complete different solution back then which means I myself will only be able to develop Dokan solutions starting with Windows 7 and using Dokan version >= 0.8.0.

Sure, not a problem at all :) , but as always, how §@c#ing difficult is to write "compatible with windows 7 + ONLY"?

 

People using XP are anyway in the phase of extinction, but think at all the guys still using Vista ;).

 

Only for the record, changing OS major subsystem from 6 to 5  the dokanctl.exe runs just fine (until it finds a call to Wow64DisablWow64FsRedirection in kerenel32.dll and throws an error).

 

:duff:

Wonko



#8 erwan.l

erwan.l

    Platinum Member

  • Developer
  • 3041 posts
  • Location:Nantes - France
  •  
    France

Posted 18 July 2019 - 03:15 PM

Sure, not a problem at all :) , but as always, how §@c#ing difficult is to write "compatible with windows 7 + ONLY"?

 

People using XP are anyway in the phase of extinction, but think at all the guys still using Vista ;).

 

Only for the record, changing OS major subsystem from 6 to 5  the dokanctl.exe runs just fine (until it finds a call to Wow64DisablWow64FsRedirection in kerenel32.dll and throws an error).

 

:duff:

Wonko

 

I actually have a lot more sympathy for ppl still using XP (i had a hard to let go as well) than ppl using Vista (why on earth someone would stick to Vista ?).

 

About this hack of yours (subsystem from 6 to 5), nice one !

Would be interested to know how you do that.



#9 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 16066 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 18 July 2019 - 04:26 PM

I actually have a lot more sympathy for ppl still using XP (i had a hard to let go as well) than ppl using Vista (why one earth someone would stick to Vista ?).

About this hack of yours (subsystem from 6 to 5), nice one !
Would be interested to know how you do that.

Really ? I thought it was "common knowledge".

A number of programs tagged "Vista or later" work just fine on XP (some even on 2K), the only reason why the Major Operating System Version and Major Subsystem Version are set to 6 is because - starting with some release of Visual Studio - the good MS guys (cannot say if maliciously/intentionally or just out of chance/incompetence) set the default setting to 6.

A number of programmers simply use it "as is" and even if the code is in itself perfectly 2K or XP compatible, the setting causes the error "This application is not a valid Windows 32 bit ...

Anyway you open the .exe in CFF explorer:
https://ntcore.com/?page_id=388
select "Optional Header" and change the relevant values, then save the PE.

An example of an executable that has this issue:

http://code.kliu.org/misc/elevate/

You will notice how the Author explicitly says that it is for Vista+ only, yet it works just fine, see starting here for a few posts:

https://msfn.org/boa...comment=1092424

The context in the above is that Dave-H needed to access a same external USB hard disk on two different PC's, one with 8 and one with XP through different interfaces that exposed different geometries.
Even if the use case is very, very narrow I am still rather proud of having devised at the time a dual 512/4K compatible partitioning scheme, even if I can probably count the people appreciating that without need to taking my shoes off, JFYI:
https://msfn.org/boa...comment=1095185
https://msfn.org/boa...ent-interfaces/

:duff:
Wonko

#10 erwan.l

erwan.l

    Platinum Member

  • Developer
  • 3041 posts
  • Location:Nantes - France
  •  
    France

Posted 18 July 2019 - 06:59 PM

 

@erwan.l different questions because i be not a programmer :

  • is this Sevenzip_DOKAN.exe or *DOKAN* possible to make a dll with it ?
  • it is possible to use the 7zip *.dll's with a headprogramm, who names maybe idadmin.exe or L2Wadmin.exe with a wll-7zip.dll snapin...

best regards
Blacky

 

Hi Blacky,

 

This is my first round on Dokan.

On round 2, there will be a generic executable taking 3 main parameters : the "source", the logical drive letter and a dll/plugin which will have to expose a few functions (_findfiles, _createfiles, _readfile, etc ...).

This way other dev guys will be able to make their own filesystem with a few lines of codes, using whatever langage they wish (as long as they can produce a c-like dll).

 

Long term wise, it could even be possible to interface it with script like langages like autoit.

 

Regards,

Erwan



#11 Blackcrack

Blackcrack

    Frequent Member

  • Advanced user
  • 458 posts
  •  
    Germany

Posted 18 July 2019 - 08:33 PM

it's ever the first step to try crawl.. the next Step is to stand, maybe by helping hand, walking and then if works, running ..

but at first must ever make the first steps to make the next steps *nod* :)



#12 Hunter23071985

Hunter23071985

    Newbie

  • Members
  • 19 posts
  •  
    Russian Federation

Posted 27 July 2019 - 11:18 PM

erwan.l, thanks for Dokan + 7zip mount!
When we can see Read-Write and mount to folder with support of Windows XP SP3?


#13 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 16066 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 28 July 2019 - 11:43 AM

 

When we can see Read-Write 

 

Hopefully never. :w00t:

Strangely enough, I am serious.

Only a subset of the formats supported by 7-zip are actually read-write, and some that can be written cannot be "updated".

Allowing read/write operation (unless some serious checks are made) is likely to open a (huge) can of worms.

 

 

 

 

... mount to folder with support of Windows XP SP3?

 

Unfortunately never :(

As we have already discussed, the issue is in the actual Dokan/Dokany that in its actual form is not compatible with XP (and earlier versions that are compatible with XP have seemingly different functions/interfaces).

In theory it may still be possible but it would mean for erwan.l to develop (and maintain) two different code-bases, one if which woulf be used by you, me and the handful of resisting XP users.

 

:duff:

Wonko



#14 Hunter23071985

Hunter23071985

    Newbie

  • Members
  • 19 posts
  •  
    Russian Federation

Posted 28 July 2019 - 12:22 PM

This is very sad, I hope the author will pay attention to my post and links.
This programs can mount archives as a folders: 7Z NSE, WIN ARCHIVER, WINMOUNT and PISMO FILE MOUNT.
PISMO can save changes and this is very convenient for creating portable programs in one executable file.
But the compression ratio in PISMO is extremely low - it squeezed the test sample to 887 MB, while the 7Z weighs 558 MB.
We need a more efficient free tool, and I hope on Dokan + 7zip mount!
At the expense of XP, I understand the situation, but I hope for a separate build that will not receive updates and maintenance.

Edited by Hunter23071985, 28 July 2019 - 12:28 PM.


#15 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 16066 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 28 July 2019 - 03:34 PM

Yep :), I know and PISMO FIle Mount has been already mentioned (as opposed to Dokan):

http://reboot.pro/to...es/#entry203351

 

and, JFYI, there is alsop this bazislib thingy:

http://reboot.pro/to...ws/#entry212188

 

The link you posted is to PISMO PTISO, the link to File Mount is:

https://pismotec.com/pfm/

 

Still, there are issues in read/write access to some compressed formats, see (only as an example):

http://reboot.pro/to...edisk/?p=120035

 

No idea if 7-zip format is suited to read/write actions.

 

:duff:

Wonko



#16 erwan.l

erwan.l

    Platinum Member

  • Developer
  • 3041 posts
  • Location:Nantes - France
  •  
    France

Posted 28 July 2019 - 04:19 PM

 

This is very sad, I hope the author will pay attention to my post and links.
 

 

Yep, the author does pay attention to your posts and links :)

 

Currently I am using 7zip library (7z.dll) to read archives.

I went for 7zip because it supports a long list of formats.

 

However, as Wonko rightfully outlined, not all formats support "write".

Furthermore, 7zip library does it make it easy to create a new archive but does not make it so easy to update.

And here, updating is actually what I need.

I could come with dirty hacks playing with streams and temp file(s) but that would be ugly and I wont go that road.

 

Now, about 7zip library itself, I could easily replace it by any other similar library as long as it can provide 3 features :

-list files/folders within an archives

-read an archive

-update an archive

 

These 3 features are enough to create a filesystem with Dokan and mount the archive to a logical drive or folder.

 

Regards,

Erwan



#17 Hunter23071985

Hunter23071985

    Newbie

  • Members
  • 19 posts
  •  
    Russian Federation

Posted 28 July 2019 - 04:57 PM

Wonko the Sane, erwan.l, thanks for the informative answers, useful links and remarks.
Based on them, I came to a conclusion that many people and you too are interested in mounting an RW archive.
The problem in the absence of a suitable library...
 
I'm not a programmer, so I'm interested to know your opinion, what we can do in this situation.
How hard is it to write such library? Can we jointly contact the author of 7-zip or something else?
I'm ready to provide any support, if I can do something...


#18 erwan.l

erwan.l

    Platinum Member

  • Developer
  • 3041 posts
  • Location:Nantes - France
  •  
    France

Posted 28 July 2019 - 05:39 PM

I might give a shot at https://libzip.org/documentation/ or https://bitbucket.or...iplib/wiki/Home .

 

While losing the advantage of reading many possible formats (7z), these 2 libraries might make it possible to come with a solution allowing one to mount a zip in RW.



#19 Hunter23071985

Hunter23071985

    Newbie

  • Members
  • 19 posts
  •  
    Russian Federation

Posted 28 July 2019 - 06:31 PM

Great, a large number of formats isn't a determining factor for us. But we can lose the compression level.
Libzip includes zlib.h and I assume that it uses zlib (i.e. minizip), which doesn't give high-quality compression.
Also in the documentation of Libzip I haven't found anything about compression parameters/tests/etc.
But if it's the only one possible solution, I'll try to test it tomorrow in more detail...


#20 erwan.l

erwan.l

    Platinum Member

  • Developer
  • 3041 posts
  • Location:Nantes - France
  •  
    France

Posted 29 July 2019 - 07:05 PM

 

Great, a large number of formats isn't a determining factor for us. But we can lose the compression level.
Libzip includes zlib.h and I assume that it uses zlib (i.e. minizip), which doesn't give high-quality compression.
Also in the documentation of Libzip I haven't found anything about compression parameters/tests/etc.
But if it's the only one possible solution, I'll try to test it tomorrow in more detail...

 

 

Libzip is a nice library, nicely documented and easy to build on windows x86/x64.

Libzip supports zlib and bzip2, with 9 compression levels (more here).

I have put a windows build there .

The repo also includes a basic delphi/freepascal demo (adding a file to a zip and listing files in a zip).

Everything is there to eventually come with a new Dokan proxy to handle zip archives in RW mode.

 

I might give it a try in the coming days or if you have some dev skills, give it a try (any langage being able to generate a c like dll will do).



#21 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 16066 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 30 July 2019 - 08:39 AM

Yep, but that (even if fun :)) would expose an algorithm/compression (ZIP/PKZIP) that has - all in all - a very low compression level, and that can already be managed just fine by 7-zip.

 

What would be interesting would be - in a perfect world - to find one of the newish, high performance (compression wise) formats and interface that (or those) RW.

 

Which of course bring us back to lz4:

http://reboot.pro/to...lz4-compressor/

 

which still it is not - I believe - particulaely suited to "update writes".

 

:duff:

Wonko



#22 erwan.l

erwan.l

    Platinum Member

  • Developer
  • 3041 posts
  • Location:Nantes - France
  •  
    France

Posted 30 July 2019 - 11:19 AM

 

 

Yep, but that (even if fun  :)) would expose an algorithm/compression (ZIP/PKZIP) that has - all in all - a very low compression level, and that can already be managed just fine by 7-zip.

 

Indeed, zlib/bzip algo/compression is not the best (and there are better algo/compression with 7zip) but libzip does offer a more convenient/safe way to add/update items in a zip container.

pros and cons...

What would be interesting would be - in a perfect world - to find one of the newish, 
high performance (compression wise) formats and interface that (or those) RW.

Which of course bring us back to lz4: http://reboot.pro/to...lz4-compressor/
 
which still it is not - I believe - particulaely suited to "update writes".

LZ4 does indeed store one stream only (AFAIK) and therefore is not a container.

And we do need a container to emulate a filesystem.

Now, we could use zlib container with no compression and stores in that container our files compressed by LZ4.

We could then have best of both worlds I guess : a RW filesystem stored in one archive with high compression (lz4 applied on each file).

 

If I put up a dokan proxy using libzip handling RW, it would be easy enough to move to the next stage and add LZ4 compression on each file.



#23 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 16066 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 30 July 2019 - 12:04 PM

Now, we could use zlib container with no compression and stores in that container our files compressed by LZ4.

We could then have best of both worlds I guess : a RW filesystem stored in one archive with high compression (lz4 applied on each file).

 

If I put up a dokan proxy using libzip handling RW, it would be easy enough to move to the next stage and add LZ4 compression on each file.

Or we could use an actual container, i.e. a volume image, containing a known, tested since years filesystem (curiously - unlike an archiver - actually written and fine-tuned to be read/write/update), mount it directly with IMDISK and call it a day.

 

Nothing against reinventing the wheel :), but the new wheel needs to be rounder than the old ones ;).

 

:duff:

Wonko



#24 erwan.l

erwan.l

    Platinum Member

  • Developer
  • 3041 posts
  • Location:Nantes - France
  •  
    France

Posted 30 July 2019 - 12:21 PM

Or we could use an actual container, i.e. a volume image, containing a known, tested since years filesystem (curiously - unlike an archiver - actually written and fine-tuned to be read/write/update), mount it directly with IMDISK and call it a day.

 

Nothing against reinventing the wheel :), but the new wheel needs to be rounder than the old ones ;).

 

:duff:

Wonko

 

Correct !

With one comment thus : you miss the automated part then :)

 

I'll stick to the original idea : possibly enable RW on a mounted (as a logical drive or folder) archive file.

 

One thing which was not really marketed as well : mounting a container as a folder (i.e not as a logical drive) can be handy in some scenario as well.

Something which Dokan can do.



#25 Hunter23071985

Hunter23071985

    Newbie

  • Members
  • 19 posts
  •  
    Russian Federation

Posted 30 July 2019 - 12:31 PM

Libzip is a nice library, but makes restrictions to the list of formats: only zip with bzip2 and deflate(zlib) compression.

I did more complex tests:

952 MB - source (955 MB in test.tar)

263 MB - 092.210 sec., ptime cmd /c "7za.exe a -r -t7z -m0=lzma2 -mx=9" > LZMA.txt

287 MB - 079.436 sec., ptime cmd /c "WIMLIB-IMAGEX capture Test Test.wim --compress=lzx:400 –solid" > LZX.txt

287 MB - 078.865 sec., ptime cmd /c "WIMLIB-IMAGEX capture Test Test.wim --compress=lzms:100 –solid" > LZMS.txt

312 MB - 175.498 sec., ptime cmd /c "ptiso.exe create -z lzma Test.cfs Test" > PTISO_LZMA.txt

361 MB - 286.312 sec., ptime cmd /c "7za.exe a -tzip -m0=BZip2 -mx=9 BZip2.zip ./Test" > BZIP1.txt

366 MB - 082.025 sec., ptime cmd /c "bzip2.exe -k -c -9 test.tar > test.9.bz2" > BZIP2.txt

386 MB - 139.419 sec., ptime cmd /c "7za.exe a -tzip -m0=Deflate -mx=9 Deflate.zip ./Test" > DEFLATE.txt

396 MB - 091.878 sec., ptime cmd /c "gzip.exe -c -9 test.tar > test.9.gz" > GZIP.txt

397 MB - 027.481 sec., ptime cmd /c "IMAGEX /capture T: IMAGEX.esd ""IMAGEX"" /compress maximum" > Imagex.txt

397 MB - 031.140 sec., ptime cmd /c "DISM /Capture-Image /ImageFile:D.esd /CaptureDir:T\ /Name:"Dm" /Compress:maximum" > D

431 MB - 080.025 sec., ptime cmd /c "COMPACT /c /f /s /a /i /exe:lzx I:\*.* " > Comp.txt

437 MB - 138.937 sec., ptime cmd /c "lz4.exe -16 test.tar test.lz4" > LZ4.txt

437 MB - 066.005 sec., Pismo PFO (test performed 2018-08-10)

 

Startup and closing time for portable:

- 06/10 s. - nsis-portable (original)

- 10/10 s. - imagex-portable (problems - sometimes files and folders stop opening even for reading)

- 18/10 s. - pismo-portable (problems - Adobe Acrobat, TreeSize and other software don't work)

 

Now the leaders are compact, but not reliable IMAGEX and reliable, but cumbersome and slow PISMO.

Dokan+bzip2 is our hope (compression 2.6X), because Dokan+LZ4 is not impressed, Dokan+7Z is not writable

and Dokan+WIMLIB is not possible. ))) Or not?






2 user(s) are reading this topic

0 members, 2 guests, 0 anonymous users