Jump to content











Photo

NTFS now supported in ReactOS LiveCD

reactos ntfs

  • Please log in to reply
24 replies to this topic

#1 Pierre Schweitzer

Pierre Schweitzer

    Newbie

  • Advanced user
  • 14 posts
  •  
    France

Posted 03 November 2014 - 09:48 PM

Dear all,

I'm Pierre Schweitzer, one of the ReactOS developers. This is a free operating system that aims to re-implement Windows, but this time with an open source license.

ReactOS now supports reading files from NTFS volume. This was a long awaited feature people were asking for. And here it is.

You can see what I'm talking about on the three pictures attached with this post.

On this one, you see ReactOS displaying NTFS information about a volume and reading a file from it.

 

post-65664-0-66107100-1415051048.png

 

 

 

On this screen, you have ReactOS running the welcome.exe binary from the NT4 installation, on the NTFS volume and also display a bitmap from said installation.

 

post-65664-0-25659100-1415051060.png

 

 

 

On this one, you have the NTFS Information tool from M. Russinovich running on ReactOS and giving information about the NTFS volume.

 

post-65664-0-43433900-1415051087.png

 

I was happy to share the current state of the work with you for two reasons.

1) First reason is that you are intensive Windows users, and you may have a need for a highly modified Windows-compatible OS which can read NTFS. At that point, ReactOS can be of help. Especially if you also need PE support!

2) The second reason is also that we need help. Your help. File read implementation is done but we need some help with NTFS writing portion.

Enough words! If you want to test this initial support for NTFS, you can grab an ISO from here: http://iso.reactos.o...S-NTFS-Live.iso

If you encounter any problem, feedback is welcome. Or best, help us and provide a patch! :)

:cheers:
Pierre

Attached Files


  • NetworkPro likes this

#2 betrand

betrand

    Frequent Member

  • Advanced user
  • 467 posts
  •  
    France

Posted 04 November 2014 - 10:52 AM

Great news! :thumbsup:

Welcome Pierre!

I think you will have quite some interest here as anyway a lot of people here know about and follow Reactos for a long time.

Good it has achieved the "read" state.

 

As you are here,

not a dev myself, but as you mention "our" use of Windows, I used an old Reactos Explorer.exe which didn't need as many dlls,

 which was OK on XP. There was a problem with subfolders though, limited to two. Would be nice to have that Reactos/XP Explorer.exe 

possibility.

Also the Explorer.exe had the Ntobj (registry). Any development on imports?

 

Cheers,

 

Another step forward!


  • Dr.NHTT likes this

#3 Pierre Schweitzer

Pierre Schweitzer

    Newbie

  • Advanced user
  • 14 posts
  •  
    France

Posted 04 November 2014 - 07:38 PM

Dear betrand,

 

First of all, thanks for your warn welcome.

 

Regarding the explorer.exe from ReactOS, I've got to admit this is not my area of competence. Basically, what I can tell you about this is that it is a fast moving target. We are currently employing a full-time dev who's working on re-implementing the explorer ReactOS is using (to drop the old & buggy we were using up to know). If you have interest in testing this new one, I can point you to the right ISOs.

 

Cheers,

Pierre


  • Dr.NHTT likes this

#4 joakim

joakim

    Silver Member

  • Team Reboot
  • 912 posts
  • Location:Bergen
  •  
    Norway

Posted 04 November 2014 - 10:47 PM

You say you need help with the NTFS writing portion, but what is the current status of this part? And, are there any known issues with the reader part of it?



#5 Pierre Schweitzer

Pierre Schweitzer

    Newbie

  • Advanced user
  • 14 posts
  •  
    France

Posted 05 November 2014 - 06:22 AM

Dear joakim,

 

Regarding the writing portion of the driver, it's totally unexistant. This is a complete open area of work.

 

Regarding the reading portion, let's say honestly that it is pretty raw. It does its job, and that's it. We ignore many things. To make it short, we read MFT, directories, files, their attributes (resident or not), and that's it. This is enough for getting read support for NTFS.

One of the clearly identified issue so far is also that we appear to have issues with reading Win7 volume (NTFS 3.X). Most of our previous tests were based on NTFS 1.2 (NT4). So, there's bugfixing/required features needed here as well.

 

Cheers,

Pierre


  • Dr.NHTT likes this

#6 Icecube

Icecube

    Gold Member

  • Team Reboot
  • 1063 posts
  •  
    Belgium

Posted 05 November 2014 - 01:02 PM


As you are here,

not a dev myself, but as you mention "our" use of Windows, I used an old Reactos Explorer.exe which didn't need as many dlls,

 which was OK on XP. There was a problem with subfolders though, limited to two. Would be nice to have that Reactos/XP Explorer.exe 

possibility.

Also the Explorer.exe had the Ntobj (registry). Any development on imports?

 

Cheers,

 

Another step forward!

 

gigaherz is working on the new ReactOS Explorer.

 

To follow the progress, you can read his blog posts:

https://reactos.org/blog/2924



#7 Pierre Schweitzer

Pierre Schweitzer

    Newbie

  • Advanced user
  • 14 posts
  •  
    France

Posted 05 November 2014 - 09:24 PM

Hello!
 
I've updated the ISO on the first topic. It brings in the most recent fixes added to NTFS.
 
What would be interesting is to get Mark Russinovich's NTFS tool fully working. It requires to properly compute the MFT zone first and last cluster. We're kind of stuck on getting it to work.
 
Also, we need more feedback on mounting volumes created in Windows 7. Our tests were done mostly against NT4 but if someone could test in Win7 to see if it is still compatible, this would really help.
 
Anyone?
 
Thanks! :cheers:


  • Dr.NHTT likes this

#8 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 16066 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 07 November 2014 - 11:08 AM

@Pierre Schweitzer

Good news. :)

 

As a side note, I will just throw this on the table :w00t: (and quickly hide my hand behind my back :whistling:).

 

Besides the future development of the Write enabled filesystem driver, you should consider seriously the idea of continuing developing a SURELY Read ONLY driver, which may be much more useful than you might think in some "niche" fields, as an example in Forensics and in "properly and safely" making images and backups.

 

As it was already suggested some time ago here by gonzoMD:

https://reactos.com/...cbbca523#p93864

it is very likely that you can find useful code in ScroungeNTFS:

http://thewalter.net...tware/scrounge/

 

Keep up the good work :thumbsup:

 

:duff:

Wonko


  • Brito and TheHive like this

#9 TheHive

TheHive

    Platinum Member

  • .script developer
  • 4198 posts

Posted 08 November 2014 - 06:32 AM

@Pierre Schweitzer

Good news. :)

 

you should consider seriously the idea of continuing developing a SURELY Read ONLY driver, which may be much more useful than you might think in some "niche" fields, as an example in Forensics and in "properly and safely" making images and backups.

 

 

Keep up the good work :thumbsup:

 

:duff:

Wonko

I was going to mention the same idea for support of Read only or Read/write options for the same reasons mentioned by Wonko.



#10 Pierre Schweitzer

Pierre Schweitzer

    Newbie

  • Advanced user
  • 14 posts
  •  
    France

Posted 08 November 2014 - 08:11 AM

Hi,

 

Pure read-only driver was actually a discussion we had a while back with another developer (because it has several advantages, indeed). We came to the conclusion that the best option to do so was not to do it in the file system driver directly, but to provide a filter driver that forces read-only.

 

So that, it could be applied to NTFS volumes, but also to FAT volumes.

 

This is also an open area for development and patches ^_^.

 

Cheers,

Pierre


  • Dr.NHTT likes this

#11 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 16066 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 08 November 2014 - 01:57 PM

We came to the conclusion that the best option to do so was not to do it in the file system driver directly, but to provide a filter driver that forces read-only.

Sorry to say so :(, but while your conclusion may be very correct from a programming standpoint: :)  - besides NOT going successfully through Occam's razor :w00t: - it is ENTIRELY "wrong" :ph34r:

The issue in Forensics is that you need to guarantee/validate that NO WRITES are POSSIBLE, and if you put an envelope around something to prevent it from "being contaminated", there is aways *somehow* the possibility that *something* CAN (even if only in theory) go through this outer layer.

It has (even if only once recorded, and in a very peculiar case) ALREADY happened that a hardware write blocker (a dedicated piece of hardware, usually considered the best possible safeguard as it is physically inserted in the device BUS) missed some write commands, noone would probably fully trust a filter driver (whilst a driver containing NO writes commands would be much more reliable and more easily validated/accepted by the community).

 

:duff:

Wonko



#12 Pierre Schweitzer

Pierre Schweitzer

    Newbie

  • Advanced user
  • 14 posts
  •  
    France

Posted 10 November 2014 - 01:20 PM

Dear Wonko,

 

I indeed didn't think about such test case and thus, the critical need for total read-only security.

 

This would indeed be doable without much effort. We can think about a registry setting to disable write support. FSD generally have one function to read & one to write (redirecting to the lower device). This is where you can block writing for instance.

 

But then, you have to think even "bigger". You can override this by bypassing the FSD actually. Just open the disk itself (\Device\HarddiskX\Partition0) and you can write/read whatever you want. So, this unlocks other "write" possibilities...

Perhaps acting directly in the disk driver might be a solution...

 

This is open for comments, suggestions. There's no obvious solution.

 

Cheers,

Pierre


  • Brito and Dr.NHTT like this

#13 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 16066 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 10 November 2014 - 01:56 PM

Yes, but No.

What is needed (IMHO) is an open source NTFS filesystem driver that contains NO Write commands.

Of course direct access to physical disk is possible, that is not normally a problem/issue, the behaviour of the "rest of" ReactOS system would be anyway need to be validated, but still a setting in the Registry is (while more elegant) NOT what is *needed*.

It is possible that you are unaware of some of the background of the reasons why hardware writeblockers are commonly used by forensic investigators, and why, now that we have in newer MS windows based PE's (please read as WinFE's) some Registry settings that allow at least to carry a part of the forensic work in "non-Linux" (which BTW in a number of common not specifically made for forensics use ALSO writes to NTFS) it is still so difficult to validate these procedures, partly because they depend on a few Registry settings.

The point is not about "voluntary" writes initiated by the user or by a user initiated program (that as said are always possible with direct disk access) but rather about "unwanted" writes initiated (automatically and in "background") by the Operating System or by any of the normally running services.

 

JFYI, I have used in the past (for some "particular" needs) "plain" (i.e. WITHOUT NTFS access at all) ReactOS using in it a Freeware program (that is intended for Windows 9x/Me):

http://www.diskinter...om/ntfs-reader/


:duff:
Wonko



#14 Pierre Schweitzer

Pierre Schweitzer

    Newbie

  • Advanced user
  • 14 posts
  •  
    France

Posted 11 November 2014 - 09:36 AM

Well, in the end, this is easily doable. Some #ifdef in the code can make any driver read-only just by rebuilding it ;-).

 

But for ReactOS, our goal is still, in the end, to provide RW support for NTFS. This would require that you rebuild it yourself to have RO.

Hence my proposal for registry setting which avoid rebuilding. But yeah, it's way less reliable... A setting can be changed!

 

The best solution here would be to make a modified ReactOS-LiveCD with modified FSDs, only RO.



#15 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 16066 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 11 November 2014 - 10:24 AM

The best solution here would be to make a modified ReactOS-LiveCD with modified FSDs, only RO.

Yep :), even better ideally (in my perverted mind ;)) there should be a (say) ntfs.sys and ntfsro.sys and the IFS detector should attempt to load ntfs.sys when a NTFS filesystem is detected, but if the ntfs.sys file is NOT found in the directory where it should be (like %Windir%\System32\drivers\) it should attempt (like a failsafe switch) attempt to load ntfsro.sys instead, this way the only need would be to delete the ntfs.sys file. (or if filenames are important) find a similar way where you delete ntfs.sys and replace it with a ntfsro.sys renamed to ntfs.sys.

 

Sorry for bringing your topic on a slightly different ground, but is so rare to have the possibility to communicate with any of the good ReactOS guys that I had to took this occasion to point out this request for feature.

 

:duff:

Wonko


  • Brito likes this

#16 Pierre Schweitzer

Pierre Schweitzer

    Newbie

  • Advanced user
  • 14 posts
  •  
    France

Posted 11 November 2014 - 12:48 PM

While I understand your need, this would be way more difficult to get into ReactOS trunk, because this is really specific need that doesn't really match the mainline of ReactOS. Other devs wouldn't support such a change.

 

This would be interesting to think about a "ReactOS Forensics" distribution or whatever ;-).



#17 esc1010

esc1010
  • Members
  • 1 posts
  •  
    United States

Posted 11 November 2014 - 04:13 PM

This is a great feature, but can it read an external hard drive formatted with NTFS?



#18 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 16066 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 11 November 2014 - 05:00 PM

.... that doesn't really match the mainline of ReactOS. 

 

... whatever that mainline is. :whistling:

 

(I know that you are not at all involved in this :), but I have my own perplexities on this "mainline" and how it has been followed - or completely failed to be followed - in the past, I know how all the reactOS are good guys - and often extremely competent programmers :thumbup: - but over the years the set of priorities has been changed several times, always "targeting for the stars" and delivering instead - for many reasons including poor support by the community and very limited resources - invariably "almost something". :()

IMHO (as I stated several times before) it would have been "better" if they chose some smaller/easier goal and complete that one fully, as an example, JFYI:

http://reboot.pro/to...-again/?p=43184

 

Sorry :blush: again for the OT (the Devil made me do it ;))

 

DevilTextversion5A.gif

 

 

:duff:

Wonko



#19 Pierre Schweitzer

Pierre Schweitzer

    Newbie

  • Advanced user
  • 14 posts
  •  
    France

Posted 14 November 2014 - 08:00 AM

This is a great feature, but can it read an external hard drive formatted with NTFS?

 

In theory, yes.
In practice, your external HDD is likely to be USB. Then, it depends on how your HDD is handled by our USB stack. The best here is to test. There are USB devices which are reported to work, some others no.

This support is still experimental & WIP.

 

... whatever that mainline is. :whistling:

 

(I know that you are not at all involved in this :), but I have my own perplexities on this "mainline" and how it has been followed - or completely failed to be followed - in the past, I know how all the reactOS are good guys - and often extremely competent programmers :thumbup: - but over the years the set of priorities has been changed several times, always "targeting for the stars" and delivering instead - for many reasons including poor support by the community and very limited resources - invariably "almost something". :()

IMHO (as I stated several times before) it would have been "better" if they chose some smaller/easier goal and complete that one fully, as an example, JFYI:

http://reboot.pro/to...-again/?p=43184

 

Sorry :blush: again for the OT (the Devil made me do it ;))

 

DevilTextversion5A.gif

 

 

:duff:

Wonko

 

Well, the point in "almost something" is that you can demonstrate it can be done (even if not perfect - yet). This is a way to asses the project is viable.

This is something really important to gather support and people following us.

 

Regarding your link, it is kind of old (2008) and the situation changed on our side, exactly in the direction pointed by the author of the post. The USB stack I was talking about previously is developed for Windows. The idea being that if you replace Windows files by ReactOS files, your Windows still boots and works.

This is also true for the new explorer being developed.

Some parts of the network stack are also being worked on at the moment, and a similar call has been made by ReactOS developers.

 

This has of course a really nice side effect which is that in order to get your driver working in ReactOS when it has been developed in Windows you have to fix broken behavior in ReactOS. This greatly improve our compatibility with Windows drivers.

 

As a teasing... One of the developer who worked with me on NTFS driver is currently working on getting the Windows NT4 NTFS driver working in ReactOS and already fixed several critical bugs in our FS API.


  • Brito likes this

#20 Wonko the Sane

Wonko the Sane

    The Finder

  • Advanced user
  • 16066 posts
  • Location:The Outside of the Asylum (gate is closed)
  •  
    Italy

Posted 14 November 2014 - 05:36 PM

Regarding your link, it is kind of old (2008) and the situation changed on our side, exactly in the direction pointed by the author of the post.

Somehow I am happy :) that the suggestion/idea by the "author of the post", even if it took quite a few years, has got some followers among the good ReactOS guys .
I believe that jaclaz is very often right :smiling9: (though in a grumpy manner), JFYI ;):
http://reboot.pro/to...-all-the-bytes/

:duff:
Wonko

#21 TheHive

TheHive

    Platinum Member

  • .script developer
  • 4198 posts

Posted 14 November 2014 - 07:35 PM



#22 Pierre Schweitzer

Pierre Schweitzer

    Newbie

  • Advanced user
  • 14 posts
  •  
    France

Posted 14 November 2014 - 08:57 PM

Yep :), even better ideally (in my perverted mind ;)) there should be a (say) ntfs.sys and ntfsro.sys and the IFS detector should attempt to load ntfs.sys when a NTFS filesystem is detected, but if the ntfs.sys file is NOT found in the directory where it should be (like %Windir%\System32\drivers\) it should attempt (like a failsafe switch) attempt to load ntfsro.sys instead, this way the only need would be to delete the ntfs.sys file. (or if filenames are important) find a similar way where you delete ntfs.sys and replace it with a ntfsro.sys renamed to ntfs.sys.

 

Sorry for bringing your topic on a slightly different ground, but is so rare to have the possibility to communicate with any of the good ReactOS guys that I had to took this occasion to point out this request for feature.

 

:duff:

Wonko

 

Let's get back to this post first.

In order to try to make your ideas reality to turn ReactOS in something "forensics" capable, I've created a git repository on github: https://github.com/reactos/RosFE

So that it's possible to create such a ReactOS without breaking mainline features.

 

We can now start looking at your needs. And github makes social integration easier, such as accepting patches! ;)

 

Somehow I am happy :) that the suggestion/idea by the "author of the post", even if it took quite a few years, has got some followers among the good ReactOS guys .
I believe that jaclaz is very often right :smiling9: (though in a grumpy manner), JFYI ;):
http://reboot.pro/to...-all-the-bytes/

:duff:
Wonko

 

Didn't know about it! What a surprise!
 


  • Dr.NHTT likes this

#23 Blackcrack

Blackcrack

    Frequent Member

  • Advanced user
  • 458 posts
  •  
    Germany

Posted 16 November 2014 - 07:13 PM

Hi,

Woh quite nice if i see the whole steps what the ReactOS and the developers does.
And i am looking forward when it is in direction testable alfa-》beta :)
and with the explorer_new :)))))

best regards
Blacky

#24 doobleshaft

doobleshaft
  • Members
  • 2 posts
  •  
    United Kingdom

Posted 16 November 2014 - 09:58 PM

Great to see! I'll test it this week though probably in a virtual machine.

 

My development skills are somewhat out of date but it does make me want to get involved. Would take me ages to do anything as its been so long...

 

Certainly happy to do testing.



#25 Pierre Schweitzer

Pierre Schweitzer

    Newbie

  • Advanced user
  • 14 posts
  •  
    France

Posted 17 November 2014 - 08:44 PM

Testing is already a great way to get involved. We cannot cover all the test cases.

 

And when you stumble on bugs, you'll want to have a look in code to fix them. And you'll slowly come back to dev :).

 

In any case, you're much welcome, bet it for testing first, deving then :cheers:







Also tagged with one or more of these keywords: reactos, ntfs

1 user(s) are reading this topic

0 members, 1 guests, 0 anonymous users