Comments on: Quickly creating large files

By: ddouthitt

ddouthitt — Mon, 14 Apr 2008 15:00:15 +0000

@Upayavira: I do believe you are right. Thanks for pointing out the error!

By: Upayavira

Upayavira — Sun, 06 Apr 2008 18:46:49 +0000

I think you mean /dev/zero not /dev/null. At least, the former worked for me and the latter didn’t.

By: ddouthitt

ddouthitt — Wed, 12 Mar 2008 22:00:47 +0000

@natophonic:

As you mentioned, /dev/urandom is nonblocking. The article you pointed to was the article on /dev/random. The article makes for very interesting reading.

Under Linux, /dev/random is designed to be free of cryptographic export controls (by not using ciphers to generate randomness), and may block at times to receive enough entropy from the system.

In contrast, /dev/urandom has a feedback loop where it feeds generated entropy back into itself, and it will not block.

Under FreeBSD, two things are notable: 1) the Linux-style randomness generator was replaced by something called the Yarrow algorithm; and 2) /dev/urandom is linked to /dev/random - put another way, neither /dev/random nor /dev/urandom will block.

Both /dev/random and /dev/urandom are available all over: including Solaris, MacOS X, FreeBSD, HP-UX, Tru64, and AIX to name a few.

In summary, if you require the best random number generator possible - use /dev/random. If you require nonblocking I/O or faster number generation, use /dev/urandom. Thirdly, if you’re using FreeBSD it doesn’t matter which you use.

The article is very interesting: if you are interested in mathematics, you simply must read it.

By: natophonic

natophonic — Wed, 12 Mar 2008 15:28:21 +0000

Generating the file was creeping along verrry slowly, but I found that switching to the non-blocking /dev/urandom

dd if=/dev/urandom of=myfile.dat bs=$(( 1024 * 1024 )) count=100

got me a perhaps slightly less random file in seconds.

http://en.wikipedia.org/wiki/Urandom

By: ddouthitt

ddouthitt — Fri, 29 Feb 2008 00:25:06 +0000

Thanks for the comment - and the fix. Done.

By: thetechnologyteacher

thetechnologyteacher — Thu, 28 Feb 2008 20:22:33 +0000

Thanks for this write up.

One small change: “(since bs=104856 and count=100).” should read 1048576

Cheers

By: Lou

Lou — Tue, 22 Jan 2008 16:37:43 +0000

Annoyingly
cat file.txt >> file.txt
Doesn’t work on my cygwin setup - it’s clever enough to realise that input and ouput are the same.

The:

dd if=/dev/random of=myfile.dat bs=$(( 1024 * 1024 )) count=100

Method worked great though, I created a 600Gb file in (relatively speaking) no time!

Thanks!

By: esofhub

esofhub — Tue, 25 Dec 2007 11:31:54 +0000

It’s part of the Solaris OS but you do make a good point regarding its portability.

Keep posting.

By: ddouthitt

ddouthitt — Mon, 24 Dec 2007 18:14:17 +0000

The mkfile command is a Linux-specific command; it is not on HP-UX 11i (I checked) and it certain isn’t on OpenVMS 8.3…. it’s also not on my FreeBSD 6.2 system either.

These methods will work anywhere.

Even so, it is a good point - just not portable.

By: esofthub

esofthub — Sun, 23 Dec 2007 15:05:18 +0000

Nice tips.

I’m fairly certain you already know this but I would also suggest using the mkfile command

# mkfile 1m 1MEGABYTE