Queer NTFS and/or XP behaviour

Foreword:
This topic is extremely "narrow", it is actually more "forensics" than "Team Reboot", and possibly the only person on Earth that may actually be interested, even marginally, to this is Joakim.
 
Preamble:
While doing a few "crazy" experiments with re-creating a "fake filesystem" from a FTK imager .csv (which BTW is a TAB delimited text file) I found some queer behaviour of the NTFS (or of the XP I tested this in or of the Explorer in it).
Everything began here:
http://www.forensicf...wtopic/t=11359/
Chad131 had what seemed to me a nice idea for a test:
http://www.forensicf...571281/#6571281
Though the "generic" (as opposed to "specific FTK imager .csv") has been IMHO nicely solved with the tool mentioned here.
http://www.forensicf...72194/#6572194 
http://www.primitive...sk-indexer.html
 
I decided to do a few tests and Francesco kindly posted a "test" .csv:
http://www.forensicf...572932/#6572932

Main Content:
 
So I started looking for some tools that I could use in one of my "primitive batches".
To create the sparse files I initially tried the re-known tool by Bo Branten mksparse.exe, but soon noticed that it has an issue, i.e. it *needs* a filesize that is multiple of 512 bytes, so I digged a bit more in my toolbox and came out with makesparsefile.exe, a nice, lesser known tool:
Copyright © 2008 Jérôme Hodé (he also provides a nice makesparseVDI tool and more):
https://forums.virtu...p?t=8048#p31276
http://jerome.hode.free.fr/opensource/
 
Then I checked first thing the Nirsoft Nircmd (which is a very nice tool and IMHO a "must have" one):
http://www.nirsoft.n...ils/nircmd.html
BUT it seemingly has not the "nanosecond" kind of precision in time that I needed (and BTW it misses a simple way to get the file time/date)-
Then I checked the (another extremely nice set of tools), the Swiss File Knife:
http://stahlworks.co...file-knife.html
(which also seemingly misses a way to get the file date/time but that has the needed "nanosecond" precision).

EDIT:

I take this back.

The sfk filetime -all -flat or sfk filetime -all flat2 works nicely.
 
So I put together one of my as usual half-@§§ed batches to test the approach.
Some of the "filesystem structures" described in the FTK imager .csv miss (obviously) the Created/Modified/Accessed timestamps, so I thought I could set them to "0".
First "surprise", notwithstanding the "theoretical" date limit of NTFS of datres going back to 1601:
http://msdn.microsof...0(v=vs.85).aspx
I did not manage to get the sfk to accept dates earlier than 01-01-1980 (this is most probably an "internal check" of the sfk tool), anyway, I set those elements (rendered in the fake filesystem as directories) to 01-01-1980 00:00:00.000000.
Second "surprise".
In an Explorer view of the mounted filesystem those had NO Created/Modified/Accessed AT ALL. <- puzzling, but actually exactly what I was looking for (whilst using another file-manager, i.e. 7-zip or doing a DIR in command prompt gave the "right" 01-01-1980 date and time).
 
The other evident issue is that since my batch created the structure in the same order of the .csv, when I used the sfk to "touch" a file, the directory in which it was contained had it's Accessed time changed.
So one needs to either "touch" the directories after having set all files or re-process the dirctories in a "second pass", only a minor issue.
 
But I decided to go further and - just for the fun of it - "0date" all elements in the filesystem, as I was trying to understand how to deal with the OS changing - *somehow* and/or *automagically* - Accesssed dates/times when performing actions on the filesystem.
 
Third surprise.
I did a couple "DIR /S" and Windows Searches on the volume and left it alone for some 24 hours with the machine running, the reason being explained here:
http://www.forensicf...wtopic/t=9329/ 
 
The ONLY file that had it's access date changed was \P1\[root]\$Bitmap <- queer.
So, I redid from scratch, and noticed that almost immediately after the "0date" touching, let's say within "a few" minutes delay, and WITHOUT any command given in the console, nor in GUI, still the \P1\[root]\$Bitmap Accessed time changed.
A few more tests gave other *random* files accessed, like $Upcase, or $I30.
Guessing that, even if made as "plain" files, filenames like $Bitmap, $I30, etc. had some "special" meaning, I tried "renaming them" to §Bitmap, §I30, etc.
Fourth surprise, it didn't change anything in the "queer" behaviour.
 
So I thought, of course, by default the indexing service is ON, so I turned it OFF for the volume and re-did the file/folder creation, with same results.
 
So, I decided to avoid completely those names and just get a bunch of "random" files/folders, from one of my other files, copy them to the volume and re-0date all of them, with same result.

Another "queer" thing is that while if I right click in explorer on a file and choose "Properties" the Accessed date/time is (obviously) changed, the same thing does NOT happen if I do the same thing on folders, that however do change their timestamp if you (say) delete a file which is inside them.

 
Conclusion (temporary) is:

• Windows XP, or Explorer (or *whatever* in the running system) accesses at least one file in a mounted volume "by it's own will".

So, I re-did from scratch, but this time running FIlemon to "watch" specifically the "*" (everything) on the M: volume.
I cannot make heads or tails of the result, it *seems* like Explorer accesses a number of files, seemingly looking for Alternate Data Streams but these accesses do not result (always) in a changed Accessed times.
 
 
Instructions to replicate/setup the test environment.
Get the mentioned tools makesparsefile.exe and Swiss File Knife, put them in a directory together with the attachment, which contains, the couple half-@§§ed batches I used and a copy of the "example.csv" from Francesco.
To get FileMon (yes the "old" version, that was merged with Regmon and resulted into Procmon), go to Softpedia:
http://www.softpedia...s/Filemon.shtml
 

The batches are "hardcoded" to drive M: (and to "example.csv").
What I used in my tests was a "Virtual Memory" IMDISK of 10 Mb in size (since they are all "sparse" files and directories there is no *need* for a larger filesystem.
 

Warning:

Those that do not know EXACTLY where their towel is are kindly asked to ignore this thread/post and DO NOT EVEN THINK of playing with this kind of matters, botching an existing, good drive because of even a small mistake is a concrete risk.
Seriously: kids, DO NOT DO THIS at home. 
 
As always, ideas, different paths to experiment with, etc. are welcome.
 

Wonko

Found some references :

http://forum.sysinte...0248_page1.html

http://blogs.technet...it-records.aspx

http://www.osronline....cfm?link=65283

But still not any real "explanation".

Wonko

I seem to recall that dates are encoded in 'seconds since 01-01-1980' somewhere - I haven't checked though.

re. Explorer, wouldn't Explorer look inside certain files for icons embedded within it, so it can display the correct icon for it? Also there would be file associations on various file extensions which would cause Explorer to go to the associated app and get the icon from there??  If the files were read to get the icons, it might cause the access date to be modified??

@Steve

No. (in the sense of, thanks, but am not looking for speculations or guesses, but rather to explanations to the activity monitored by FileMon and to the effects it has - or *something else* has on file and folder Accessed timestamps).

The (already given) reference states how NTFS times are made:

A file time is a 64-bit value that represents the number of 100-nanosecond intervals that have elapsed since 12:00 A.M. January 1, 1601 Coordinated Universal Time (UTC). The system records file times when applications create, access, and write to files.

 

Do not confuse them with FAT dates/times.

The article by Mark Russinovich explains most of it:

http://blogs.technet...it-records.aspx

and pinpoints what I overlooked, though the "flush" of interrogations seems to happen when opening a folder (double clicking).

But still it makes little sense.

I prepared a new batch, that creates a directory structure (to be applied to a newly formatted filesystem) like:

M:\Folder0
M:\Folder1
M:\Folder2
M:\Folder0\Folder0File0.ext
M:\Folder0\Folder0File1.ext
M:\Folder0\Folder0File2.ext
M:\Folder0\SubFolder0
M:\Folder0\SubFolder1
M:\Folder0\SubFolder2
M:\Folder0\SubFolder0\File0.ext
M:\Folder0\SubFolder0\File1.ext
M:\Folder0\SubFolder0\File2.ext
M:\Folder0\SubFolder1\File0.ext
M:\Folder0\SubFolder1\File1.ext
M:\Folder0\SubFolder1\File2.ext
M:\Folder0\SubFolder2\File0.ext
M:\Folder0\SubFolder2\File1.ext
M:\Folder0\SubFolder2\File2.ext
M:\Folder1\Folder1File0.ext
M:\Folder1\Folder1File1.ext
M:\Folder1\Folder1File2.ext
M:\Folder1\SubFolder0
M:\Folder1\SubFolder1
M:\Folder1\SubFolder2
M:\Folder1\SubFolder0\File0.ext
M:\Folder1\SubFolder0\File1.ext
M:\Folder1\SubFolder0\File2.ext
M:\Folder1\SubFolder1\File0.ext
M:\Folder1\SubFolder1\File1.ext
M:\Folder1\SubFolder1\File2.ext
M:\Folder1\SubFolder2\File0.ext
M:\Folder1\SubFolder2\File1.ext
M:\Folder1\SubFolder2\File2.ext
M:\Folder2\Folder2File0.ext
M:\Folder2\Folder2File1.ext
M:\Folder2\Folder2File2.ext
M:\Folder2\SubFolder0
M:\Folder2\SubFolder1
M:\Folder2\SubFolder2
M:\Folder2\SubFolder0\File0.ext
M:\Folder2\SubFolder0\File1.ext
M:\Folder2\SubFolder0\File2.ext
M:\Folder2\SubFolder1\File0.ext
M:\Folder2\SubFolder1\File1.ext
M:\Folder2\SubFolder1\File2.ext
M:\Folder2\SubFolder2\File0.ext
M:\Folder2\SubFolder2\File1.ext
M:\Folder2\SubFolder2\File2.ext

 

So, I double click on M:\Folder0 and the access date of M:\Folder0\Folder0File0.ext is updated to current. <- queer, but "fair enough", as it is first file

Then I go one directory up and double click on M:\Folder1 and the access date of M:\Folder1\Folder1File1.ext is updated. <- Why?

Then I go one directory up and double click on M:\Folder2 and the access date of M:\Folder2\Folder2File2.ext is updated. <- Why?

So I try to do this in a different order (after having reset Access dates/times:

M:\Folder2 and the access date of M:\Folder2\Folder2File2.ext is updated.

M:\Folder1 and the access date of M:\Folder1\Folder1File1.ext is updated.

M:\Folder0 and the access date of M:\Folder0\Folder0File0.ext is NOT updated

So, I try again in a different order and now M:\Folder1\Folder1File0.ext is updated, then M:\Folder0\Folder0File0.ext is, then M:\Folder0\Folder0File2.ext, and finally M:\Folder2\Folder2File2.ext is, then I open at arandom, several timaes all folders and subfolders, and after some serious double clicking on folders the Accessed files remain these four.

I can find no "pattern", let alone a "logical" one.

When I select a file, right click and Copy (without pasting anywhere) understandably the selected file Accessed time is modified, but if I do the same on a Subfolder, the Accessed time of the Subfolder and of all the files in it remain untouched.

Attached the batch to create the structure and the one to check for modified access times, that allows to create a similar structure on a FAT volume, so that one can test the different behaviour.

Wonko

I almost feel obliged to answer, but have troubles finding enough marginal interest in this very narrow topic.

I did find the information of the excessive accessing of objects on the filesystem by explorer, both new and interesting. But I will accept that explorer just behaves "a bit strange" when accessing file/folders (or update of LastAccess time of certain Windows version), rather than spending an extensive amount of time trying to get to the bottom of this behaviour.

That said, I don't have an answer for the observed behaviour of explorer. But maybe some day..

Slightly OT, but would not procmon work (you can deactivate the reg, proc and network stuff if you like)?

Sure, I also "accept" it, though what I find "irrational" is that a "random" file is accessed and that is accessed without actually "clicking" on it, but rather clicking on the directory that contains it.

There is also a form of "timing" issue, if I open the folder and quickly exit it, nothing happens, then if I open it and "linger" on it some time a "random" file is accessed, but not always and not always the "same" file in repeated attempts.

I believe that this "peculiar" behaviour is *somehow* triggered by the "peculiar" timestamps of January 1st 1980, 00:00.0000000, and while it has "in itself" no particular meaning/relevance, I find it (if needed) yet another proof that you cannot trust anything behaving for sure "rationally".

It is simply a "queer" behaviour that apparently makes no sense.

I am happy anyway that it is not something "peculiar" to my environment, if you can reproduce it. 

Thanks.

Wonko

About procmon, yes, of course it can replace filemon, it's just a matter of what one is used to/familiar with.

Speaking of tools, maybe you should have checked FileTest as well to get the date. At least it's open-source

http://www.zezula.ne...s/filetest.html

Speaking of tools, maybe you should have checked FileTest as well to get the date. At least it's open-source

 

http://www.zezula.ne...s/filetest.html

BUT, last time I checked, it was not command line.

Wonko

BUT, last time I checked, it was not command line.

 

Wonko

Not sure I understood it all but...here attached an open source and command line exe that will use the native MS API NtQueryAttributesFile to retrieve its CreationTime,LastAccessTime,LastWriteTime,ChangeTime .

I have no credit there : took me 2 mns googling and compiling some bits found here and there.

It should display something like that :

CreationTime:01CC7FB2C9EBEAB8
LastAccessTime:01CDC10EE3931D64
LastWriteTime:01CB88B010294A50
ChangeTime:01CDC10EE3931D64
FileAttributes:00000027

/Erwan

Thank you erwan.
But I have no issues whatsoever in using the tools I am using.

For NO apparent reason florin91 posted an OT about a (btw nice tool from the same author of RunAss and of BellaVista ) and this tool, besides NOT being needed, would not also do for this specific task as it is not command line (but I don't need a command line tool, as I already found one that works fine).


Wonko