Defragging on NTFS volumes

sjj1805 · Post by **sjj1805** » Fri Aug 11, 2006 12:46 am

Please view this article:
optimising the computer for video Editing.

In addition to the suggestions in the article concerning
Swap Files
Defragmentation
You may also wish to consider the size of the clusters. Normally small clusters are used because even if a cluster contains only 1 byte of information the entire cluster is marked as used and so no further data can be written to that cluster. Video Files are large and so you could use larger cluster sizes and get less fragmentation.

nyco_ork · Post by **nyco_ork** » Fri Aug 11, 2006 10:39 pm

Thanks, Steve.

I've read that article. And maybe changing the cluster size for the video drives is a good idea.

But, since I said I would report on developments, and I've had some more time to experiment, here are my findings:

Someone said that I probably have a hardware problem. If I do, it's because of the architecture of the laptop, which I can't change; and I don't have room for a full-size PC. Besides, I spent a lot of money on configuring and outfitting two identical laptops. What I needed to do was to get this hardware to work with my software.

I tried out Diskeeper 10 Professional Premium, and it's an excellent utility. Finally I'm able to get a list of which files need defragging, and can exclude the ones I don't want it to operate on. It also has a feature for "handling large files efficiently," which means it doesn't try to completely defragment a large file if the utiility reports there is no performance hit.

However, that feature didn't work on my system. When I tried burning to DVD using an mpeg that was in 2 fragments, which Diskeeper reported should not have a performance hit, there was a huge glitch in the last minute of a 90-minute program. This was the smoking gun that proved to me that fragmentation was the problem.

What I do now, which is working fine, is keep the source mpeg and DWZ file for a burn on a disk that has very few other files. I check the fragmentation status before I do the burn. If the files necessary for the burn are fragmented, I defragment them only (or use an alternative method which I'll get to). I also close MF4, then reopen it and the project, so the program can "clear its throat," because I noticed that a lot its housekeeping files on the C drive were also heavily fragmented after I'd been editing a project. After that, I proceed with the burn.

My allternative method for defragging a single file is to copy it to my super-fast outboard drive, which runs on Firewire 800. A 4 gig file takes three minutes to copy to the drive. I then delete the file on the internal drive and copy it back from the outboard drive, which takes 5 minutes; total, 8 minutes, which is twice as fast as a defrag of the same file.

In either case, the results have been excellent. I guess on some systems, at least, defragging is even more critical than it is on others.

R

snoops · Post by **snoops** » Sat Aug 12, 2006 12:18 pm

Hi R,
I wouldn't accept the workaround of defragging every file all the time. This would drive me nuts.

the architecture of the laptop, which I can't change

Did you ever check for IBM BIOS and firmware updates?? If the controller/drive properties (like HDD with UDMA5, and DVD maybe UDMA2) are really correct, then some component may need a fixed version.

I seriously suggest visiting the IBM support area and checking for needed updates including (most important) BIOS and disk or dvd drive firmware.
Their System Update v2.0 (2006/07/14) simplifies this task if you don't want to hunt for everything yourself. BTW the current disk firmware package is FWHD43 from 2006/06/19.

nosnoop · Post by **nosnoop** » Sat Aug 12, 2006 5:17 pm

snoops wrote:I wouldn't accept the workaround of defragging every file all the time. This would drive me nuts.

I agree. This is a lot of work and a lot of time wasted just to burn a DVD.

Have you tried using another DVD burning software? You can try creating an ISO image file with MF4, then use software like Nero Burning ROM to burn the disc. Nero allows you to increase the buffer size. Since you have enough RAM, you can choose a size larger than we normally need, like 80MB or 100MB.

Apart from the safety afforded by the larger buffer, Nero also has a real time buffer meter, and you can see how your system is performing. The device buffer (built-in buffer in your DVD drive) may bounce around; that's fine, and don't worry about it. But you PC buffer should stay more or less at 100% all the way until the end. You will only see a drop if you do any disk operation like launching a new progam. But if you see that the buffer is depleting and playing catch up a lot even when there is no other program running, you know something is not right.

Even though defragging from time to time is good practice, you really should not need to do so much work each and every time.

sjj1805 · Post by **sjj1805** » Sat Aug 12, 2006 5:36 pm

Computers are designed to be left turned on 24/7 and so what I do is create a schedule with the Windows XP scheduler to run a defrag during the middle of the night.

I have 3 internal hard drives and so it is a simple matter of creating a routine so that my drives are defragged as follows:
Monday Drive 1
Tuesday Drive 2
Wednesday Drive 3
Thursday Drive 1
Friday Drive 2
Saturday Drive 3
Sunday - a day of rest!

You can use the same procedure if you prefer to use a third party defrag program such as Diskeeper Pro, Norton Speed disk etc.

nyco_ork · Post by **nyco_ork** » Sat Aug 12, 2006 7:22 pm

Regarding my BIOS, etc; they are all up to date. I have the latest versions.

Regarding scheduling defrags: I don't leave my laptops on 24/7 because they overheat. I did try scheduling with Diskeeper, but unfortunately there was a conflict between Diskeeper's scheudling module and my Oxford English Dictionary. It suddenly stopped being able to find its data. I had to go back to manual defrag. But really, when I defrag the system drive once a day, just before I'm stepping away from the computer for a break, it only takes a few minutes. And on the video data volumes, the workaround of keeiping them free of clutter (no more than two projects on a volume at a time) seems to keep the files from fragmenting in the first place, so for the last week or so I haven't had to defrag any big files. I move the project and mpeg files to the outboard backup drive as soon as I've checked the DVDs, rather than leaving them on until the disk fills up.

Steve, I am interested in your cluster size recommendation. What cluster size would you suggest for mpeg files that generally run between 3-4 gig?

R

sjj1805 · Post by **sjj1805** » Sat Aug 12, 2006 7:28 pm

Sorry I overlooked that you were using a laptop and so it is impracticle to use the scheduling system in your situation.

Regarding cluster sizes please view:
http://phpbb.ulead.com.tw/EN/viewtopic. ... 1186#71186

nyco_ork · Post by **nyco_ork** » Sat Aug 12, 2006 7:39 pm

Thanks, Steve. I'll give it a look.

R

snoops · Post by **snoops** » Sat Aug 12, 2006 7:51 pm

nosnoop

Just for the record, that is not me with another name.

But I somehow have a funny feeling I know who it is...

maddrummer3301 · Post by **maddrummer3301** » Wed Aug 16, 2006 3:01 am

ruggy1 · Post by **ruggy1** » Wed Aug 16, 2006 1:20 pm

I cannot understand your problem and would agree with previous learned comments that you have a problem somewhere in the laptop. Laptops should be no different to desktop PCs, except they are generally much slower, but should still do the same things in the end if you have enough memory. My PC has 4 x 250GB drives, all very heavily fragmented with perhaps only 100GB of 1TB available overall. Yet MF capture and Nero burns pose no problem. It would take me ages to defragment them but as everything works fine I will not do so at the moment, as most of the files will be deleted in the near future.

labbie · Post by **labbie** » Wed Aug 16, 2006 4:53 pm

nyco_ork wrote:Thanks, Steve. I'll give it a look.

R

One other thing to look for is to check the Power settings and disable turning off the hard drives to save power. You want to keep the drives spinning while burning.

I've had problems like you are having because of drives powering down unexpectedly. It takes a good 3-4 seconds to fully spin up a hard drive. I don't think Windows really uses the timer counts listed in the Power Properties correctly. I think it checks every X number of minutes to see if the drive is in use, not that the drive hasn't been used for X number of minutes. Does that make sense?

Good Luck.

Rick

nyco_ork · Post by **nyco_ork** » Wed Aug 16, 2006 6:54 pm

Rick, I have all power shutoffs disabled. The screensaver, screen power off, HDD power off, all of them. I don't even like waiting for the HDD to spin up to speed when I'm doing normal work, never mind a DVD burn.

I followed Steve's suggestion for increasing the cluster size (I set it for 64kb) and I think that has really helped matters. For one thing, the files don't become fragmented when I import them, which they used to do. The larger cluster size forces the system to find a big enough space on the drive to hold the complete file.

I only made this change to my two video data volumes, not to the system volume.

R

heinz-oz · Post by **heinz-oz** » Wed Aug 16, 2006 11:23 pm

Neither should you change the cluster size for your system volume. Every little file you store will use a complete cluster, not very ecconomical with disk space. Video files on the other hand are huge files and a larger cluster size does not significantly affect the used space but aids in preventing/limiting excessive fragmentation.