Jump to content











Photo
- - - - -

Minimal fileset for installing an ADK Optional Component?


  • Please log in to reply
46 replies to this topic

#26 erwan.l

erwan.l

    Platinum Member

  • Developer
  • 3041 posts
  • Location:Nantes - France
  •  
    France

Posted 05 January 2018 - 12:07 PM

TinyDism has been mentionned a few times now, if anyone cares to try, it is here.

 

A few words thus :

-That for now this is purely a "POC / exercice for fun" with Misty: this is a very early alpha, dont shoot me for bugs, lack of versioning, etc ...

-The GUI is ugly as hell (or shall I say uglier that the ones I usually come with :) ).

-There is a command line version (which uses the same code as the GUI).

 

On my win8.1 x64 system, where I dont have adk/waik installed (and I dont recall installing it), I can perform basic tasks such as add-driver, add-package, mount/unmount, ...

 

On a win7x64 (with adk/waik installed) where I have used getwaiktools to download ADK5 (win8.1), this POC works fine as well.

 

For now I am using dismapi.dll which I believe (but to be confirmed) is there by default since windows 8.x.

Ultimately I would like to drop dismapi.dll and go straight to dismcore.dll which again would get us closer to MS original functions.

Unfortunately dismcode.dll is a COM object (and I hate these....) and is undocumented (which makes it therefore very tempting to use).

 

That POC may, or may not, lead us to some interesting ideas or new tools.

Overall dismapi is very well documented and extremely easy and quick to implement so this is not much efforts wasted in worse case scenario.

I am probably not the first one to look at dismapi.dll (dismcore.dll on the other side has very little results on google).



#27 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 16066 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 05 January 2018 - 03:24 PM

@Misty

Yep :), what I was trying to say is that DISM and its subsystem most probably can manage correctly a Registry extracted to (say):

C:\myniceextractedirectory\withinsideastupidlylongnamebuthtisisjustanexample\Windows\System32\config\

as opposed to a "replicated" Windows structure like (still say):

Z:\Windows\System32\config\

but I suspect that other possible files might be not managed as nicely (but of course it is well possible that *everything* is "correct" and "relative").

 

@erwan.l

Anything will be anyway a small or large progress, so, good work! :)

 

:duff:

Wonko



#28 erwan.l

erwan.l

    Platinum Member

  • Developer
  • 3041 posts
  • Location:Nantes - France
  •  
    France

Posted 07 January 2018 - 06:16 PM

Some findings (on my part) :

 

  • starting with windows 8.x, dism components are there by default - you should not need to download any extra MS files.
  • dismapi.dll was introduced with windows 8 (i.e "winpe4" i.e "ver 6.2.x" i.e "adk4" )
  • dismapi "session oriented" functions (add-driver, add-package, etc) are downward compatible but not upward compatible : dismapi 10.x can work with a 6.x image, not the other way around
  • dismapi "wim oriented" functions (mount, unmount, get-imageinfo, etc) are downward and upward compatible : any dismapi.dll should be able to mount any wim file.
  • the host itself does not matter much and you can download adk (4,5 or 6) thru getwaiktools and handle wim files from there.

And note for later, dismapi.dll is only the visible part of the iceberg.

There is a whole lot more in dismcore.dll to discover/play with/tweak, etc.

Because dismcore.dll is a COM object, I dont expect much findings there unless some dev guys with COM knowledge on this forum are willing to spend some time there.



#29 misty

misty

    Gold Member

  • Developer
  • 1066 posts
  •  
    United Kingdom

Posted 09 January 2018 - 06:23 PM

Thanks to a hard drive failure on my current test system (and lots of distractions with project work!) I've not gotten any further with my original goal.

I have however had lots of contact with erwan.l via PM and want to say a big thank you to Erwan for all of the work he has put into TinyDISM. He has released 11 builds in approximately 4-5 days! And that's just the ones that I'm aware off!!!!

Thank you Erwan :worship:

For other people wanting to play, I would suggest using the command-line version included in the download package that Erwan included in post #26 (TinyDISM).

Current supported swithes include -
 
tinydism 0.1 by erwan2212@gmail.com
tinydism /get-imageinfo c:\temp\boot.wim
tinydism /mount-image c:\temp\boot.wim c:\mount [/ro]
tinydism /unmount-image c:\mount [/discard]
tinydism /add-package c:\mount c:\temp\package.cab
tinydism /add-driver c:\mount c:\temp\driver.inf
tinydism /add-capability c:\mount Language.Basic~~~en-US~0.0.1.0
tinydism /get-drivers
tinydism /get-packages
Erwan - you are amazing. Thank you my friend.

:cheers:

Misty

P.s. This really has been fun. And a great distraction during a time of stress!

#30 erwan.l

erwan.l

    Platinum Member

  • Developer
  • 3041 posts
  • Location:Nantes - France
  •  
    France

Posted 09 January 2018 - 06:44 PM

Erwan - you are amazing. Thank you my friend.


:cheers:

Misty

P.s. This really has been fun. And a great distraction during a time of stress!

 

 

Always a pleasure and fun to work with you!

We share the same goal on this one : get some distration during stressful times :)



#31 misty

misty

    Gold Member

  • Developer
  • 1066 posts
  •  
    United Kingdom

Posted 13 January 2018 - 10:33 AM

TinyDISM has been added to my new ADK project in the latest update.
 

Project updated.

Additional options added to the main project script -

  • Select between DISM or erwan.l's TinyDISM (see here and here) to apply packages and carry out mount/unmount operations.
  • Option to mount the source .wim file instead of fully extracting it has also been added.
A new post processing script has been added for creating a bootable .iso file after the build has completed.

If you have already downloaded the project then click on the Update Project button in the Advanced Options script.

Comments/feedback welcome.

:cheers:

Misty

P.s. Much of the code has been copied from MistyPE and there may be some unintentional references to MistyPE remaining.



#32 misty

misty

    Gold Member

  • Developer
  • 1066 posts
  •  
    United Kingdom

Posted 13 January 2018 - 10:47 AM

Due to the amount of time I've been spending coding and testing, I will be (attempting) to take a break for a week or two. I will be checking in on the forum and I can't rule out being sucked back in by interesting discussions :whistling:

Before taking a break I thought I'd share some thoughts related to WinPE / Packages / (Tiny)DISM.

Some background
At this point in time, in order to add certain functions to a Windows PE, Optional Components are a necessary evil. Whilst it is theoretically possible to add a feature without an Optional Component, it's not always practicable. Think about the WinPE-NetFx package for example - who wants to trace all of the dependencies for that. And where would the source files be obtained. And each Windows PE build may require different dependencies.

The most convenient method remains downloading and installing one of the Windows ADK's. For the majority of people this means downloading the full ADK using the adksetup.exe installer - with lots of features that are likely to be unnecessary. For WinPE only functions we can narrow it down to just the following installers (and their .cab dependencies - which vary between ADK versions) -
  • Windows Deployment Tools-x86_en-us.msi
  • Windows PE x86 x64 wims-x86_en-us.msi
  • Windows PE x86 x64-x86_en-us.msi
That's still a lot of bloat if only -
  • A few Optional Components are required
  • Only a 32-bit WinPE is required
  • Only a 64-bit WinPE is required
Also, it might not be desirable to install the Assessment and Deployment Kit. Or more than one ADK for Windows 10 may be required.

So what about extending JFX's excellent Get WAIK Tools (discussed and available here) to download just the required element(s) - for example DISM (and dependencies), winpe.wim (32-bit or 64-bit or both), individual Optional Components, etc.

Get WAIK Tools already has an option to download part of a file. Get WAIK Tools uses (or appears to use) this approach to facilitate downloading individual elements from some ADKs - presumably using hardcoded links in the program.

I'm not sure whether Get WAIK Tools is redistributable - I'll ask on the support topic on MSFN. CURL may be a viable alternative if not.

So we have Erwan's nice command-line (and very easy to script) TinyDISM.

And we have my new little PEBakery/WinBuilder ADK project.

If I can find a means of identifying which bits (offsets) of the relevant .cab contains the required file then it may be possible to -
  • Download just the part of the .cab containing the required file
  • Decompress the bit of the file downloaded above (remember it's been partially extracted from a compressed .cab and not decompressed during the download)
  • Use it as required - e.g. in a PEBakery/WinBuilder project.
Any thoughts on the viability of this approach?

And suggestions re finding the file offsets in a .cab and decompressing it afterwards. Looks like Get WAIK Tools may already handle the decompression during the extraction.

:cheers:

Misty

#33 erwan.l

erwan.l

    Platinum Member

  • Developer
  • 3041 posts
  • Location:Nantes - France
  •  
    France

Posted 13 January 2018 - 12:49 PM

About extracting on the fly some files from an ISO which happens to be online (i.e over HTTP), I have been tempted to trigger some developpement there for a while.

As GetWaikTools has been around for a while and does an excellent job, I put any dev on my side on hold.

 

However, if JFX has no time or interest to add extra features to his tool, let me know and I could volunteer to support you with your idea.

 

Downloading the first few bytes of an iso and retrieve its file directory (and therefore offset/length of files in the iso) is fairly easy.

I could think about a simple command line tool like : getfile.exe http://url/my.iso myfile.ext .

No need then to worry about the offset/length : the command line tool would handle that internally.

 

Just like you, I believe downloading and/or installing a whole ADK could be a show stopper.

Anything we can do to take some distance from MS tools will make the process/redistribution easier.


  • misty likes this

#34 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 16066 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 13 January 2018 - 02:12 PM

To be fair, GetWaiKTools is a bit more sophisticated than that, there are no problems with actual .iso's, there is httpdisk:

http://reboot.pro/to...ole-waik-first/

 

What GetWaikTools does (largely thanks to the work that JFX put on it, besides the actual scripting/programming, in the maintaining) is to download and rebuild - whenever possible - partial .cab files and rebuild them.

 

This is not AFAICR a "fully automated" feature, it needs manual tests to find suitable ranges and what not.

 

BTW the newer versions of GWT also accept command lines with an URL and a range to download (so it can also act like a curl replacement) and - as at the time was demonstrated by Homes32 AutoIt already has this feature via WinHttp, see:

http://www.msfn.org/...&comment=999488

 

So - with reference to the "freely explore" point you made elsewhere - I am all for it :thumbup:, and for re-inventing the wheel, as long as the new wheel is rounder, more silent, provides better handling or reduced attrition, etc. 

Or - if you prefer - you should be very aware of the current "state of the art" before attempting to make something better than what is already available, before spending the (scarce) resources available on re-creating solutions to already solved problems it would be probably more useful to the community to provide new soutions for unresolved problems. :dubbio:

Of course if it is just for your personal fun and interest it is more than OK. :)

 

:duff:

Wonko 



#35 erwan.l

erwan.l

    Platinum Member

  • Developer
  • 3041 posts
  • Location:Nantes - France
  •  
    France

Posted 13 January 2018 - 02:24 PM

Or - if you prefer - you should be very aware of the current "state of the art" before attempting to make something better than what is already available, before spending the (scarce) resources available on re-creating solutions to already solved problems it would be probably more useful to the community to provide new soutions for unresolved problems. :dubbio:

 

Absolutely aggree, as much I as like to explore, or have selfish fun, I am not into duplicating existing features.

This is a waste of time and efforts and in some way a lack of respect for original dev guys.

Usually, I try to (but i am not perfect) to look at all other existing solutions before i come with my own.

I have started to look at GWT these last days and indeed the command line options are very handy and possibly covers it (all or mostly).

 

What I did not try yet : lets say I know where my iso is on internet, I know the flle I want to "extract", can I do it in one go? as in "sometool.exe some_online_iso somefile_in_my_iso > my_local_file".

 

Obviously I am conscious there are many constraints in the way : cab, zip, msi, etc which are included in iso files.

Thats probably where the bigger constraint is as extracting a plain file from an iso is rather straight forward.

So by the end, one may have to still download a few MB's to finallt get a few KB's file - still better thus than having to download a few GB's iso...


  • Atari800XL likes this

#36 misty

misty

    Gold Member

  • Developer
  • 1066 posts
  •  
    United Kingdom

Posted 13 January 2018 - 02:35 PM

@everyone
Thanks for the feedback.

Reply from JFX from the Get WAIK Tools thread

Hi misty,

yes Get WAIK Tools is redistributable, feel free to use it as you like.

The range command of GWT is actually only thought for ISO (or other uncompressed) files.
Extracting files from inside a cabinet, without loading the complete file, is much more difficult.

From the topics, I take it you are looking for the 2 big cab file that hold all the winpe component packages?
If so, I don't think it's possible to reduce the download much.

They are packed with MSZip compression, with many files in separate blocks.


Unfortunately it look's like the impressive Get WAIK Tools is not suitable for this task. At least not at the moment!

 

About extracting on the fly some files from an ISO which happens to be online (i.e over HTTP), I have been tempted to trigger some developpement there for a while.
As GetWaikTools has been around for a while and does an excellent job, I put any dev on my side on hold.

However, if JFX has no time or interest to add extra features to his tool, let me know and I could volunteer to support you with your idea....


Unfortunately it's not .iso files that the required files need extracting from - it's the ADK .cab files.

I have also looked at Nuno's re7zip, however the Java dependencies make this unsuitable.

Misty
  • Atari800XL likes this

#37 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 16066 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 13 January 2018 - 02:37 PM

 

What I did not try yet : lets say I know where my iso is on internet, I know the flle I want to "extract", can I do it in one go? as in "sometool.exe some_online_iso somefile_in_my_iso > my_local_file".

Again, if it is a "random" .iso, you use httpdisk instead, do read the given thread:

http://reboot.pro/to...ole-waik-first/

 

Now what would be useful would be a http (and while you are at it ftp also) interface/proxy to 7-zip.

 

Why 7-zip?

Because it is open source, freeware and it can open almost any of the "common" formats, both filesystems and compressed files.

 

EDIT: I see that Misty :thumbup: found and referenced Nuno's re7zip which is an attempt to something similar to what I described. 

The actual "description" url is:

http://reboot.pro/fi...ile/224-re7zip/

 

:duff:

Wonko


  • misty likes this

#38 misty

misty

    Gold Member

  • Developer
  • 1066 posts
  •  
    United Kingdom

Posted 13 January 2018 - 02:41 PM

@Erwan.l

Just like you, I believe downloading and/or installing a whole ADK could be a show stopper.
Anything we can do to take some distance from MS tools will make the process/redistribution easier.

Amen to that!

Now whilst I appreciate your offer of developing a new application, I agree with Wonko (and you) about not reinventing the wheel. If on the other hand you were to come up with a tool for handling .cab files (direct download like in the ADK files - not from within .iso like in the WAIK) then that appears to be something that is currently missing - at least in a Windows tool without java dependencies. And I mean no disrespect to Nuno and re7zip - it's just that the java dependencies make it impractical for me.

Misty

#39 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 16066 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 13 January 2018 - 02:44 PM

Additional thought.

 

Another thing that - last time I checked at least - is amissing is a valid tool to manage the MS "new" compression format, some hints references are given here:

http://www.msfn.org/...mpression-tool/

 

:duff:

Wonko



#40 erwan.l

erwan.l

    Platinum Member

  • Developer
  • 3041 posts
  • Location:Nantes - France
  •  
    France

Posted 13 January 2018 - 03:01 PM

@Erwan.l
Amen to that!

Now whilst I appreciate your offer of developing a new application, I agree with Wonko (and you) about not reinventing the wheel. If on the other hand you were to come up with a tool for handling .cab files (direct download like in the ADK files - not from within .iso like in the WAIK) then that appears to be something that is currently missing - at least in a Windows tool without java dependencies. And I mean no disrespect to Nuno and re7zip - it's just that the java dependencies make it impractical for me.

Misty

 

Do you mean a tool to retrieve a file from a online cab similar to re7Zip who handles retrieving a file from an online iso?

 

If I can find the CAB specs somewhere, that should be doable.

Quickly looking at a cab file header, it looks like file are listed and I guess next to it are the offset and length in the cab.

If so, that could be a quick one.

 

EDIT :

cab file specs here.

 

As a whole we need command line tools that easily download files from "containers" such as iso, zip, file etc without having to know the offset/length of the files we want to extract/download.


  • Atari800XL and misty like this

#41 misty

misty

    Gold Member

  • Developer
  • 1066 posts
  •  
    United Kingdom

Posted 13 January 2018 - 03:06 PM


Do you mean a tool to retrieve a file from a online cab similar to re7Zip who handles retrieving a file from an online iso?

Yes I do. And re7zip is reported to handle more than .iso files. Note from the re7zip download page -

All the archives supported by 7-zip are supported by this tool, this includes:


zip, tar, split, rar, lzma, iso, hfs, gzip, cpio, bzip2, 7z, z, arj, cab, lzh, chm, nsis, deb, rpm, udf, wim, xar


  • Atari800XL likes this

#42 misty

misty

    Gold Member

  • Developer
  • 1066 posts
  •  
    United Kingdom

Posted 13 January 2018 - 05:00 PM

Some discussion regarding re7zip and some points raised above has taken place on MSFN with JFX. Please read from here.

:cheers:

Misty

P.s. What happened to that break I mentioned :whistling:
  • Atari800XL likes this

#43 erwan.l

erwan.l

    Platinum Member

  • Developer
  • 3041 posts
  • Location:Nantes - France
  •  
    France

Posted 13 January 2018 - 05:28 PM

Some discussion regarding re7zip and some points raised above has taken place on MSFN with JFX. Please read from here.

:cheers:

Misty

P.s. What happened to that break I mentioned :whistling:

 

Just updated the MSFN thread.

Have been a registered user for a while but never posted anything there : my 1st post there !


  • misty likes this

#44 osfixer

osfixer

    Member

  • Validating
  • 35 posts
  •  
    United Kingdom

Posted 06 March 2019 - 07:30 PM

Interesting project..

I have a question about this, how does Dism++ work? They apparently use CBS Servicing not

dism dlls.



#45 erwan.l

erwan.l

    Platinum Member

  • Developer
  • 3041 posts
  • Location:Nantes - France
  •  
    France

Posted 06 March 2019 - 08:15 PM

Interesting project..

I have a question about this, how does Dism++ work? They apparently use CBS Servicing not

dism dlls.

 

Lots of tools out there are using dismapi.dll which is a C dll : easy to interface, use, etc.

 

Dism++ (which is quite a nice tool by the way) uses dismcore.dll which is a COM dll.

Undocumented and much harder to interface compared to a "classic" C dll.

This dll is on my list of "to open" but this is not going to happen in a near feature...

discome.dll is a bit of "one ring to rule them all"



#46 osfixer

osfixer

    Member

  • Validating
  • 35 posts
  •  
    United Kingdom

Posted 06 March 2019 - 08:24 PM

Seems like they use CbsHost.dll also..



#47 NeoBeum

NeoBeum
  • Members
  • 3 posts
  •  
    Australia

Posted 22 June 2019 - 02:35 PM

@misty

 

How often do you update the dynamic/static libraries for wimgapi and dismapi to use in your PE Image?

 

 

Side note:

I'm currently in the middle of creating an updated library to hopefully interface the new Windows Terminal into WinPE. Along the way I've encountered issues with the Windows Servicing Stack - one of the recent issues was that it totally stripped the installed components and left the Winre.wim bare. I'm trying to contact the Microsoft Windows Setup Development team so I can work on some DISM API.

 

I'm also looking at adding some Linux File System support to the PE library that I'm working on. There's alot of crap that I'm in the middle of adding.

The main one is having a disk identifier that uses all possible binaries available to identify the disk. This is for a safetynet on laptops - Windows had issues a few years ago identifying NVME, SATA, (SCSI, IDE) and still has issues reporting "BusType" for MSFT_Disk - depending on what program is used to identify.

 

 






1 user(s) are reading this topic

0 members, 1 guests, 0 anonymous users