22 replies to this topic

[Wonko the Sane]

So, there is a Unix tool that exists since the dawn of time:

dd

Why is it called dd?

From it's man page (verbatim):

dd(1) - Linux man page
Name
dd - convert and copy a file
Synopsis
dd [OPERAND]...
dd OPTION
Description


Copy a file, converting and formatting according to the operands.

WHY is it called "dd" and not "cc"?
http://www.djmnet.or...e/dd-origin.txt

Is there a port for Windows NT based systems?

Yes, several of them.

Where can I find a "minimal" one?

There are UNXUTILS:
http://unxutils.sourceforge.net/
the whole package is around 3 Mb (binaries) and 36 Mbytes (source)
the "current" dd is 40960 bytes.
There is an older version (that probably does not do physical devices) here:
ftp://ftp.internat.f...ocustworld.com/
that is sized 32768 bytes.

Since it is an OPEN SOURCE project it should mean that you get the SOURCE code, right? (yes you can get it ) AND that this source is compilable, right? (NO this seemigly won't happen ).

Half the internet is full of normal peeps that ask "How can I recompile UNXUTILS?" and that get replies like:

• UNXUTILS is bad, get GnuWin32.
• UNXUTILS is old and not mantained.
• UNXUTILS use a downhill library that doesn't work.
• UNXUTILS is old, badly mantained, get CYGWIN port.

GNUWIN32 is a similar project:







http://gnuwin32.sourceforge.net/
So you want dd and what happens?
You need to download the WHOLE STUFF, which is around 232 Mb of size.
Then you start looking for the stupid dd, and finally you find it in \GetGnuWin32\packages\coreutils-5.3.0-bin.zip\bin\ yes you could have got just coreutils-5.3.0-bin.zip, but noone told you that dd was actually in coreutils, that is anyway good, since dd won't run, asking for a libintl3.dll, 101,888 bytes in size, that you will find in \GetGnuWin32\packages\coreutils-5.3.0-dep.zip\bin\ together with another .dll, libiconv2.dll 898,048 bytes in size.

So you go to CYGWIN, and AGAIN you need to get an "automated" SETUP.EXE.
After you get it and it has connected to the internet, you try to find dd, and you find:

• Ddd (which is another thing)
• ddrescue (which is another thing)

thank goodness you somehow know that dd is normally inside "coreutils", so you uncheck EVERYTHING, and you ONLY check:

• coreutils
• cygwin

you need anyway to download "something" (a whole lot of things you don't actually need as dependencies, the setup will add them for you)

Once finished you will have downloaded some 14 Mb of stuff.
dd.exe, 70158 bytes in size, will be in:
\release\coreutils\coreutils-8.10-1.tar.bz2\coreutils-8.10-1.tar\usr\bin\
Now you have to find it's dependencies, which are in:
\release\cygwin\cygwin-1.7.9-1.tar.bz2\cygwin-1.7.9-1.tar\usr\bin\ :
cygwin1.dll 2,666,500 bytes
cygintl-8.dll 31,232 bytes (which is OBVIOUSLY NOT in the above) but rather in:
\release\gettext\libintl8\libintl8-0.17-11.tar.bz2\libintl8-0.17-11.tar\usr\bin\
cygiconv-2.dll 1,003,022 bytes from
\release\libiconv\libiconv2\libiconv2-1.13.1-2.tar.bz2\libiconv2-1.13.1-2.tar\usr\bin\
cyggcc_s-1.dll 44,558 bytes from
\release\gcc4\libgcc1\libgcc1-4.3.4-4.tar.bz2\libgcc1-4.3.4-4.tar\usr\bin\

So you think that the above is a piece of bloat and try to get the MINGW version, by going to MINGW:
http://www.mingw.org/
obviously you cannot get a dd.exe from there and there is no easy way to know where it is among the zillion files here:
http://sourceforge.n...ts/mingw/files/
so you have once again to get an "installer" mingw-get-inst-20110802.exe thiss one is much worse than the Cygwin one (I didn't thought it was possible) as it has NO granularity you are basically forced into downloading a whole package, around 80 Mb.
Actually what you get is the actual programming environment, so you have to start again from scratch.
Maybe you want to get MSYS and NOT MINGW.
Of course dd is NOT in "coreutils":
http://sourceforge.n...r.lzma/download
nor in MSYS "CORE":
http://sourceforge.n...r.lzma/download
At this point you search a bit on the internet and find out that probably there is no compiled dd.exe in MINGW/MSYS, hey, wait, maybe it is in the fileutils:
http://sourceforge.n...iewmonth=200506
too bad that "fileutils" does not exist (anymore) there is "file" (which is ANOTHER thing)

Time to move on.

Some of the smart peeps around the board will have probably said by now:

Ha, there is dd for windows:
http://www.chrysocome.net/dd

Which of course I already know, and that is NOT a dd port but a re-write of it (in Delphi) with a few additions (nice ) and a few missing features (not important, I guess last time dd was used with the "convert" switch was 1987 or 1988 ).
Problem is that executable is 355,328, not at all "minimal".

Now, some even smarter peeps will say:

Ha, have you not checked the FAU page?
http://gmgsystemsinc.com/fau/

Which of course I already know, it is a "forensic oriented" dd.exe which by the way I often use.
dd.exe 901,120 bytes
NEEDED at least :
fauerror_410.dll 110,592
BTW if you don't have an English system (or a system set to English the FAU dd will try to translate it's usage in your language with results - at least for Italian - that calling "appalling" would be an euphemism).

Of course, when the going gets tough ... :
http://en.wikipedia....Tough_Get_Going
actually more like:
http://www.imdb.com/...es?qt=qt0479926

Bluto: And it ain't over now. 'Cause when the goin' gets tough...
[thinks hard of something to say]
Bluto: The tough get goin'! Who's with me? Let's go!
[Bluto runs out, alone; then returns]

Meet PLDD :
http://home.comcast....csi/tools/pldd/
self-standing, biggish (pldd.exe 63,488 bytes) BUT includes a "timeout" and more than that the equivalent of a /dev/zero.

So we have several different morals from this story:

• When a "mainly DOS/NT" guy attempts to touch the "linux" world he is probably doomed to failure.
• Finding things (and the things made like you would like them to be) is not easy.
• Re-building them is even less easy.
• Everyone, and I mean everyone sucks at documenting or making accessible/usable things.
• Life is so full of problems that actually creating them yourself is foolish, just tackle any of the existing ones and you'll be occupied for several hours.

A question to the good linux guys members of the board (and particularly those that are part of the "Reboot Team" and solve problems):





Will any of you create a compiled dd.exe that:

• needs NO external .dll's
• has NO "convert" options
• can access Physical devices both read and write
• has an internal /dev/zero
• it is smaller than the mentioned ones (ideally below the 40 Kb mark)

?


Wonko

[Sha0]

...

I have been through experiences just like this, but I am fascinated that you actually DOCUMENTED the entirety of the pain. It gave me a good laugh because of how familiar it was, not because of your misfortune. It's really quite amazing how someone might have one tiny little item that is wanted and wind up chasing for hours (or days). I have screamed the beginning chunk of the title of this post several times.

I'm not sure that I agree with DOS / Windows NT people being doomed to failure with POSIX tools/environments. I think you just have to have 100 or so of these miserable experiences.

A question to the good linux guys members of the board (and particularly those that are part of the "Reboot Team" and solve problems):

Will any of you create a compiled dd.exe that:

• needs NO external .dll's
• has NO "convert" options
• can access Physical devices both read and write
• has an internal /dev/zero
• it is smaller than the mentioned ones (ideally below the 40 Kb mark)

?

I would suggest to you that if possible, you get coLinux with their offered, text-only Ubuntu distribution. Then you can install:

$ apt-get install mingw32

And you will have just enough that you can compile [some] Windows tools under Linux without filling up your Windows with Cygwins, MingWs, MSYSes, Services for UNIX, DDKs, and who knows what else... (Like I have on my system. )

Just out of curiosity, why are you concerned about the executable size for DD?

[Wonko the Sane]

I have been through experiences just like this, but I am fascinated that you actually DOCUMENTED the entirety of the pain. It gave me a good laugh because of how familiar it was, not because of your misfortune. It's really quite amazing how someone might have one tiny little item that is wanted and wind up chasing for hours (or days). I have screamed the beginning chunk of the title of this post several times.

Yep, should you need it, there is still some space available Outside of the Asylum, you would be welcome.

Documenting such experiences is in my view important as it may give to someone some ideas how to change things to make other people's life less complex.
We know from experience that his has never worked with the good MS guys, but maybe it could work with the good Linux guys .
Or at least a man can dream ....

I'm not sure that I agree with DOS / Windows NT people being doomed to failure with POSIX tools/environments. I think you just have to have 100 or so of these miserable experiences.

Naaah, after such experiences I would become "honoris causa" a Linux guy too.

I would suggest to you that if possible, you get coLinux with their offered, text-only Ubuntu distribution. Then you can install:

$ apt-get install mingw32

And you will have just enough that you can compile [some] Windows tools under Linux without filling up your Windows with Cygwins, MingWs, MSYSes, Services for UNIX, DDKs, and who knows what else... (Like I have on my system. )

You see, you are reasoning like a "Linux guy" "install this distro" to put together a bunch of bytes (maybe).
We didn't even touch the issue of setting up a compiling environment and the the zillion changes that one will need to make to the Sources (when found) in order to actually compile them.
As seen most of the sources - exception made for the non-compilable one and for the PLDD one that actually has some info on how to compile - end up in a bloated result with dependencies to HUGE .dll's.

Just out of curiosity, why are you concerned about the executable size for DD?

I am not "concerned", if it is possible to have it in 40 Kb size (including the "convert" features) why cannot it be made in (say) 32 Kb (still with direct device access) or maybe in less?
Do you think that having something that is in itself 70,158 bytes and needs more than 3 Mb of dependency .dll's can be called in ANY way "smart"?

You can stamp your feet as long and as hard as you can , but you will never convince me that anything that:

• is smaller in size
• is simpler and needs no .dll's

won't run "better" and be more usable, portable and what not than any of the bloated alternatives.





So the answer to the question is:

Because it can be done.

BTW and JFYI the PLDD I found covers my current "needs" with the bonus of the /dev/zero/-like feature in an all-in-all "compact enough" thingy.


BUT this is a challenge:
http://reboot.pro/15208/


Wonko

[Sha0]

You see, you are reasoning like a "Linux guy" "install this distro" to put together a bunch of bytes (maybe).
We didn't even touch the issue of setting up a compiling environment and the the zillion changes that one will need to make to the Sources (when found) in order to actually compile them.

Well, I suggested that because I think it could benefit you. That is the environment I use these days to make tiny Windows utilities that depend only on Windows DLLs. That is the environment in which I have started coding a DD for you. I really think you ought to consider it.

As seen most of the sources - exception made for the non-compilable one and for the PLDD one that actually has some info on how to compile - end up in a bloated result with dependencies to HUGE .dll's.

MinGW32 is one of the simplest environments I've found. I've mentioned dependency concerns above. The executables can even be stripped of symbols. Please think about it.

[Wonko the Sane]

Well, I suggested that because I think it could benefit you. That is the environment I use these days to make tiny Windows utilities that depend only on Windows DLLs. That is the environment in which I have started coding a DD for you. I really think you ought to consider it.


MinGW32 is one of the simplest environments I've found. I've mentioned dependency concerns above. The executables can even be stripped of symbols. Please think about it.

Thank you very much for your advice, I do consider it , and I would normally appreciate your interest in coding the dd , but I have to ask you this time to please NOT do it.

We both know the capabilities you have, no need to use your time on this trifling thing, if you have enough of WinVblock, etc. at the moment, you should tackle something at your level
like a win32 port of mkfifo (or actually a a command line app capable of creating, managing etc. named pipes in Windows), that is something that AFAIK is missing alltogether.


Wonko

[L A M A]

I'm sure i used dd.exe (single win file on insanelymac some years back), there was an early distribution file iatkos or something (in the usual place), which had command line file dd.exe included (and i used it to write entire image file on hdd from xp). It also worked, however destroyed all my partitions (an expected result i was prepared of).

I'm quite sure there were no other dll files attached... maybe this is what you're looking for? but since you're the master of search engine, i'll back off from this... (even if i found that torr... not sure of linking it here)


edit: nope, can't find it now... its too old somewhere near iatkos v1 (someone just made hdd compressed image out of it, which had to be restored with dd.exe)
the instruction was like: Unzip the 20GB image, type dd xxxxx command (which restored entire hdd image). XP crashed few min after completion, next restart started osx leo.


OK: I think its most probably THIS: http://www.chrysocom...oads/dd-0.5.zip (still searching...)
http://www.insanelym...showtopic=68480 (searching more...)


edit2: nope, wonko's search is pretty accurate

[Wonko the Sane]

I'm sure i used dd.exe (single win file on insanelymac some years back), there was an early distribution file iatkos or something (in the usual place), which had command line file dd.exe included (and i used it to write entire image file on hdd from xp). It also worked, however destroyed all my partitions (an expected result i was prepared of).

I'm quite sure there were no other dll files attached... maybe this is what you're looking for? but since you're the master of search engine, i'll back off from this... (even if i found that torr... not sure of linking it here)

How big in size (bytes) was it?
Please re-read (this time attentively) my post, in a nutshell:
I did find three independent dd's:

• unxutils "old" 32,768 bytes <- presumably with no device access
• unxutils "current" 40,760 bytes <- presumably with device access
• pldd 63,488 bytes <- the one I am currently testing because of the added /dev/zero-like feature

besides the obvious "dd for windows" 355,328 bytes in size.



Should the one you remember you used (please compare with Rule #2):

2. Use some common sense. Don't post your question under multiple topics. Think before posting and choose the right topic for you question or answer. When posting, have something to say, not just "My First post!" or "Cool site!!!" We sort of have the idea by now .
It is much appreciated, when you post a reference to something, to provide a link to it.
Replies like "I remember there was an app that did that, cannot remember it's name" are pretty much useless.

be any of the above or bigger than 63,488 bytes (or being so buggy as to destroy a hard disk ) I don't want to know about it but thanks anyway for trying .



Wonko

[Scott Dare]

Having recently to built an entire linux VM simply to use dd, this well-structured rant was a balm. Methodical documentation of problem solving (even our tribulations therein) is a gift to the community. I hope to follow your example!

[Wonko the Sane]

Now, let's see if anyone can guess (without unpacking the installer or actually installing the thingy , nor checking the source ) WHICH version of dd.exe the good Fedora Linux guys use for their "live USB creator"?
https://fedorahosted...iveusb-creator/

and which one the good Mandriva guys in their "usbdumper"?


Just to keep track of it, a dd-like tool (oriented to recovery and thus with added features) fits in 81,920 bytes
bbcopy.exe
http://alter.org.ua/...win/bb_recover/


Nothing (yet) "competitive" with the DSFOK toolkit, though:
http://members.ozema...eezip/freeware/
http://members.ozema...eware/dsfok.zip

• dsfi.exe 5,061 bytes
• dsfo.exe 6,637 bytes
• fsz 6,144 bytes <- same use as /dev/zero can create files filled with 00's

Wonko

[Max Shonichev]

Well, coreutils from GnuWin32 are actually available as a separate download, only 6megs size for full setup with no additional dependencies.
http://gnuwin32.sour...s/coreutils.htm

Typically if you want to repackage single application from such releases, you run it through depends.exe (Dependancy Walker, http://www.dependencywalker.com/ ) and there you have a list of all the .DLLs.

And just to clarify things, dd at linux also requries a pack of shared libraries (.so):

user@ubuntu# ldd /bin/dd
linux-gate.so.1 => (0x00652000)
librt.so.1 => /lib/i386-linux-gnu/librt.so.1 (0x00d5d000)
libc.so.6 => /lib/i386-linux-gnu/libc.so.6 (0x00110000)
libpthread.so.0 => /lib/i386-linux-gnu/libpthread.so.0 (0x00413000)
/lib/ld-linux.so.2 (0x00ca2000)

[Wonko the Sane]

Really?

Then you start looking for the stupid dd, and finally you find it in GetGnuWin32packagescoreutils-5.3.0-bin.zipbin yes you could have got just coreutils-5.3.0-bin.zip, but noone told you that dd was actually in coreutils, that is anyway good, since dd won't run, asking for a libintl3.dll, 101,888 bytes in size, that you will find in GetGnuWin32packagescoreutils-5.3.0-dep.zipbin together with another .dll, libiconv2.dll 898,048 bytes in size.

and anyway gnuwin32, for the intended scope and for the reasons expressed is a mass of bloat (in relative terms, of course ).


Wonko

[Brito]

Interesting link

WHY is it called "dd" and not "cc"?
http://www.djmnet.or...e/dd-origin.txt

I've went on to read the story and found a familiar name, Pat Place:

Subject: The Unix "dd" command
Why not collect responses and post them.
My understanding of the naming (and syntax for use)
is that this was done as a joke. Notably it is poking
fun at OS/360 - a once reasonably well known operating
system where commands like //DD SYSIN= ... abounded and
were used for setting up I/O devices for various programs
that were about to be run.

Pat Place prp@sei.cmu.edu

I recorded one of his lectures, very interesting person:
http://www.youtube.com/watch?v=Ad2_QBoqSqY

[sambul61]

It was quite interesting to read this one of Wonko DePonte's many opuses. I wonder why he went on such a challenging journey after seemingly using dd for the last (1)50 years?

[greatdevourer]

WinDD - Disk Dump for Windows! Windows XP version of Unix 'dd' command. Safe, effort-free backup for FAT, FAT32, NTFS, ext2, ext3 partitions.

http://sourceforge.net/projects/windd/

And it works in WinPE environments.

[Wonko the Sane]

WinDD - Disk Dump for Windows! Windows XP version of Unix 'dd' command. Safe, effort-free backup for FAT, FAT32, NTFS, ext2, ext3 partitions.

http://sourceforge.net/projects/windd/

And it works in WinPE environments.

But it DOES NOT do what dd does:
http://sourceforge.net/projects/windd/
Besides it is a Visual BAsic 6 app (thus has quite a few of dependencies).

This snippet from the help file:

WinDD is a rough-and-ready Windows XP version of the Unix disk dump (dd) command.
Unix dd performs a byte-wise dump of one disk partition to another, making for a powerful, zero-cost disk backup tool.
WinDD does the same job on Windows.

Actually means:

WinDD is a rough-and-ready Windows XP version of the Unix disk dump (dd) command.
Among the many actions the Unix dd can do, it can performs a byte-wise dump of one disk partition to another, making for a powerful, zero-cost disk backup tool.
WinDD does the same job on Windows, i.e. ONLY a byte (actually sector) -wise dump of one disk partition to another disk partition (AND NOTHING ELSE) .

Wonko

[Max Shonichev]

Really?

Use installer, Luke.

[greatdevourer]

Thanks for the correction Wonko. I'll keep looking.

[Wonko the Sane]

Use installer, Luke.

Use the Source, Luke.
http://linuxgazette....sue49/lamb.html

Missing reference:

Obi-Wan: Use the Force, Luke.

http://www.imdb.com/title/tt0076759/quotes?qt=qt0440717



Wonko

[Max Shonichev]

Well, as you've failed to use the Source, I adviced to use Setup =)

http://gnuwin32.sour...s/coreutils.htm

If you download the Setup program of the package, any requirements for running applications, such as dynamic link libraries (DLL's) from the dependencies as listed below under Requirements, are already included.

e.g. not the f#ing 'Binaries' ZIP, which is of course DLL-less until you also get the 'Dependencies' ZIP.

[Wonko the Sane]

Well, as you've failed to use the Source, I adviced to use Setup =)

I have NOT failed to use the Source, I would like to use the Source of another project, UNXUTILS, and NOT gnuwin32, which I find a bloated approach.

However the specific issue is/was that when you actually go here:
http://gnuwin32.sourceforge.net/
You try to get what you want:
http://gnuwin32.sour...t/packages.html
BUT dd is NOT listed (the list is about packages) thus, unless you ALREADY KNOW that it is in coreutils, which is described as "GNU file, shell and text utilities" you have NO WAY to know that you should get coreutils.

So you need to get the "generic" downloader through here:
http://getgnuwin32.sourceforge.net/
http://sourceforge.n...gnuwin32/files/
Where you are told:

Looking for the latest version? Download GetGnuWin32-0.6.3.exe (3.4 MB)

So you get it and run it, and you discover that you need to run download.bat, you comply and you later find out that it wil have downloaded some 232 Mb of stuff.

Is this "easy"?
Is this "friendly"?

If you are convinced that this is the "right" way to have things organized, and easily accessible, you are very welcome , but allow me to disagree.

Mind you it is not a "personal" problem I have, I usually can find what I need/want, including dependencies, and what not, after all I am "The Finder" , I simply doubt that a number of people would be able to do so, given the way things are set up.


Wonko

[Vortex]

Hi Wonko,

You are completely right. Unfortunately, sometimes we need to download large sized packages to extract get some useful tools. Humanity is always inclined to build more and more complicated things.

A designer knows he has achieved perfection not when there is nothing left to add,but when there is nothing left to take away.

Antoine de Saint Exupery

[Wonko the Sane]

Yep

Art is the elimination of the unnecessary.

Pablo Picasso

but, bettter :

Any intelligent fool can make things bigger, more complex, and more violent. It takes a touch of genius -- and a lot of courage -- to move in the opposite direction.


Albert Einstein

Wonko

[Vortex]

Hi Wonko,

We probably met at Solar OS homepage :

http://oby.ro

You can read at the main page the quotations of Saint Exupery and Einstein.