Editing on a Shoe String.

[sjj1805]

.......When I format a drive for video or audio I usually go with formatting larger cluster sizes other than the default 4k for NTFS. A 4k cluster size is good for small files but large audio / video data uses much more space. A large cluster size is efficient for large files.
Windows default is 4k cluster sizes when formatting NTFS.........

John, Thanks for that - never thought of it before but it makes obvious sense. I would expect less fragmentation from the larger clusters. As you say video files are huge.

To save everyone looking up what John means by larger clusters here's a very quick explanation.

Your hard drive is split up into small bits called clusters. When the computer wants to find something on the hard drive it looks inside a directory to find the start of that something. The directory points it to a cluster - which is a small bit of hard drive (like a drawer). When the cluster is full the last bit of info in that cluster is to direct you to the next cluster used by that item. (Termed a chain)

Normally you keep these clusters small because as soon as something has been written in that cluster, the rest of that cluster cannot be used for anything else. So if the thing you wrote inside the cluster was only 1 byte - it has used the whole cluster up. This means that a large amount of hard drive space remains unused and is termed slack space.

For anyone interested in reading more about this there is an interesting article here:
http://www.pcguide.com/ref/hdd/perf/per ... ter-c.html

Although that link refers to the FAT filing system but most of us use NTFS - the article describes it very well. There is also a link in that article relating to NTFS.

[maddrummer3301]

Steve,
I'm rusty on the lingo but much has to do also with interrupt requests.
Every Disk I/O requires an Interrept request. Large clusters reduce the amount of interrupt requests for reads & writes.

I don't know how that plays into windows disk write caching along with DMA but should improve overall performance if disk write cache is disabled for direct streaming to the harddisk as many audio apps have that as a selection within the program such as Cakewalk Pro Audio & Sonar. In those apps one can enable/disable write caching within the software program.

I usually don't perform "Quick Formats". My perference is to format the standard method but harddisks today are so reliable I guess that's a matter of choice (or chance?)

When setting up the paging file under windows. You can assign more than one drive. Do you know how windows is accessing the 2nd paging file if there are 2? When the first is exhausted? or maybe similar to raid0?

MD

[sjj1805]

............When setting up the paging file under windows. You can assign more than one drive. Do you know how windows is accessing the 2nd paging file if there are 2? When the first is exhausted? or maybe similar to raid0?

MD

John, Its back to the all you wanted to know about paging files but were afraid to ask link. The specific part of that very long article dealing with multiple hard drives/paging files is here:
http://www.rojakpot.com/showarticle.asp ... 43&pgno=33

[maddrummer3301]

Steve,
Thanks, that article re-enforces Heinz's suggestions for a tweaked setup.
Need alot of harddisks though to keep an audio/video drive separate.
Nice article, thanks.

MD

[PeterMilliken]

I see that the discussion has wandered into the realms of tweaking overnight.

Not wanting to upset anyone here

All of this is great information but possibly slightly irrelevant to the original post - I would also suggest that unless you have done performance monitoring then you might also be misleading yourself. For instance, while coming up with all of these different disk configurations did any of you actually monitor where the bottle neck was in your system? i.e. any system when performing a task is in one of two states - 1) CPU bound or 2) IO bound (you could be bound by lack of memory which could lead to misleading symptoms, but I won't cover that here - but again you can monitor it and take appropriate action).

CPU bound means that the CPU is running just as fast as it can i.e. 100% and disk throughput is less than 100% utilisation - the time that the application can complete in is "bounded" by the CPU speed. If you can get a faster CPU in there (this encompasses memory transfer rate as well as raw CPU speed) then the application will complete sooner.

IO bound means that the IO requests to the hard drive(s) are at 100% and the CPU is running at less than 100% - so the time that the application takes to complete is "bounded" by the speed of the IO transfers. So in this case if you can throw more disks, different controllers etc at the problem to improve the IO throughput then you can speed up the time for the application to complete.

You can see this information in Task Manager - there is probably even better tools out there as well.

But I would put it too you that unless your performance monitoring indicated that your VS session was IO bound (before you added all of these extra disks) then all the extra hard drives are making minimal difference (especially from the perspective of the original poster i.e. shoestring budget ).

So to Roy, I would suggest that you purchase the fastest processor, fastest FSB (front side bus) memory and the most memory (this discussion above slightly ignores amount of memory available ) that you can afford and then look at what results. If performance monitoring indicates you are IO bound then by all means purchase extra disks - but if it is CPU bound then forget the extra expense because it is my opinion you would be wasting your money.

As I said in my earlier post - I have a single hard drive partitioned into 2 partitions (that are significent to VS usage that is) and when I perform rendering my system is CPU bound - so adding extra drives would not achieve any significent benefit - but it might give me a warm feeling

Note my system specs as well.

Lots of luck mate,
Peter

[GuyL]

I see that the discussion has wandered into the realms of tweaking overnight.

Not wanting to upset anyone here

All of this is great information but possibly slightly irrelevant to the original post - I would also suggest that unless you have done performance monitoring then you might also be misleading yourself. For instance, while coming up with all of these different disk configurations did any of you actually monitor where the bottle neck was in your system? i.e. any system when performing a task is in one of two states - 1) CPU bound or 2) IO bound (you could be bound by lack of memory which could lead to misleading symptoms, but I won't cover that here - but again you can monitor it and take appropriate action).

CPU bound means that the CPU is running just as fast as it can i.e. 100% and disk throughput is less than 100% utilisation - the time that the application can complete in is "bounded" by the CPU speed. If you can get a faster CPU in there (this encompasses memory transfer rate as well as raw CPU speed) then the application will complete sooner.

IO bound means that the IO requests to the hard drive(s) are at 100% and the CPU is running at less than 100% - so the time that the application takes to complete is "bounded" by the speed of the IO transfers. So in this case if you can throw more disks, different controllers etc at the problem to improve the IO throughput then you can speed up the time for the application to complete.

You can see this information in Task Manager - there is probably even better tools out there as well.

But I would put it too you that unless your performance monitoring indicated that your VS session was IO bound (before you added all of these extra disks) then all the extra hard drives are making minimal difference (especially from the perspective of the original poster i.e. shoestring budget ).

So to Roy, I would suggest that you purchase the fastest processor, fastest FSB (front side bus) memory and the most memory (this discussion above slightly ignores amount of memory available ) that you can afford and then look at what results. If performance monitoring indicates you are IO bound then by all means purchase extra disks - but if it is CPU bound then forget the extra expense because it is my opinion you would be wasting your money.

As I said in my earlier post - I have a single hard drive partitioned into 2 partitions (that are significent to VS usage that is) and when I perform rendering my system is CPU bound - so adding extra drives would not achieve any significent benefit - but it might give me a warm feeling

Note my system specs as well.

Lots of luck mate,
Peter

Peter,

No offense here and a great post, mate. I only took part in this discussion because the use of a separate drive for the swap file always intrigued me. There has been so much contradicting information over the years on virtual memory that...ah well you know how it is. I may just try it one day to see what happens.

My "problem" is that my system does not seem to be bound either way. It seems I have a lot of headroom running VS from both a processor/memory and disk I/O perspective. I'm not complaining except I think VS could be taking more advantage of the resources available to it based on the machine it is running on. A great example of this is dual core performance.

Don't get me wrong, I do not miss the days of old when there wasn't enough horsepower in any affordable PC to do any real editing. I may have just not accepted the fact that hardware has surpassed consumer video editing software demands. I realize that high end stuff would require much more system....but I'm not doing that sort of work.

[PeterMilliken]

Hi Guy,

You caught me out I have a similar system to you (dual-core etc) and it is not always 100% (on the one core it does want to run on) - but it generally runs very close to it i.e. about 90 - 95%.

I guess my point was that it is also not operating at 100% disk usage either - so this indicates to me that providing extra disks would not benefit me (significantly - I keep throwing that in because to be pedantic, you could always argue that an extra disk must have some effect - even if it is to make the render time 0.01% faster ).

These later posts were all great information - but more relevant to the "tweaker" than to Roy's original request.

Perhaps Steve can post a summary of the information contained here in somewhere for people to access when they need it? I know Steve is doing an excellent job - I just wished we had a better forum for this type of permanent information than the Web Board i.e. a web-site would (to my mind) be a better container than individuals posts. But that's another topic again I tried to start one with my wiki site (see the downunder perspective entry in the top post) - but that never took off, I am the only poster - John Hunter posted there originally as well but then shifted the information into the sticky post when "stickiness" became available.

Have a great evening there,
Peter

[maddrummer3301]

guilty me, sorry to let things go off topic