created video files are huge, and other newbie questions

[byoung]

Just bough VS ProX5. I'm an experienced stills photographer but new to video. Shooting with Canon 5DmkII DSLR, which provides 1920x1080 .MOV files at 29.97 fps. Data rate on the files is about 45Mbps, with a total bit rate marginally higher.

I have been asked to output .MOV at 29.97, 1920x1080 progressive with PhotoJPEG compression at 90%, square pixels.

In Proj. Properties, I seem to need Microsoft AVI (MPEG doesn't allow 1920x1080 option). No compression, and have obviously kept 29.97 fps. I have no idea whether I want to perform non-square pixel rendering at this point, and haven't yet found any info on whether the camera shoots square or non-square pixels. I also have no idea if I want frame-based or upper or lower fields first. I haven't found any explanation of what this means in the product documentation.

I've created an output template to give 1920x1080 at 29.97 fps, 24-bit colour. Again don't know whether to pick frame-based or upper or lower field first. Using PhotoJPEG compression (100% seems the only setting available, but that's OK).

In the final Create dialogue box Options, there is another chance to render non-square pixels (I'm assuming "no" here since I want square), as well as a checkbox for Smart Render, but again no explanation as to why I would need it.

So I've been practicing with 10sec clips to see what happens. With the above settings, the creation takes a couple of minutes and I have been able to generate a huge 1.6GB output file. Play is somewhat jerky since I don't think my computer can keep up. The input files are e.g. 233MB for 44sec, or about 5.3 MB/sec. Am I wrong to expect the output to be about the same size? Curiously, I have been able to generate 50MB files for two of the videos, and the rendering only took a couple of seconds. Unfortunately I have not been able to repeat this, but rather get these 1.6-1.7GB files as the routine. I have tried different input files with the same result.

All help on settings to use, and where the big files are coming from, is appreciated.
thanks, Bill

[Ken Berry]

I'm not sure that I can actually help produce what you have been told to produce, but I can only make a couple of preliminary comments. First, Photo JPEG or Motion JPEG is these days a rather outdated format. It produces good quality, but the downside is that its files are large -- and that is I suspect what you are seeing. Your original MOV files from the camera, while using the Quicktime .mov extension, are in effect AVCHD mpeg-4 which is highly compressed, and certainly much more compressed (and thus smaller) than your eventual MJPEG files.

As for the jerkiness, your computer is certainly more than powerful enough to play any current consumer level video format smoothly, so I don't think it is that. But I *am* wondering about the Field Order, as you are. Can you right click on one of the original .mov files, either in the VS timeline or the VS library window (not in Windows Explorer) and either take a screen grab of the Properties box which appears, or else copy down manually ALL the Properties in the box and post them here. That will tell us whether your original video is using Upper Field First (as I suspect it might be) or frame-based/progessive.

I ask because you tell us your original video is 29.97 fps, but that usually denotes interleaved video (i.e. 59.94 half frames per second, which comes down to 29.97 interleaved frames per second). But the people who have set the parameters have asked you for 29.97 fps progessive, which means 29.97 full frames per second, which is actually better quality than 29.97 interleaved frames. And the way X5 goes about producing those full frames would probably account for the jerky playback.

I am also puzzled as to why they want square pixels. Your original video will be wide screen and very probably using non-square pixels. Converting this to square pixels would IMHO squash the resulting video so that people or things would look taller and skinnier than reality.

I also wonder idly whether Quick Time Pro would give you a better chance of producing the sort of result they are after...

[byoung]

Hi Ken
thanks very much for the response. If I drag a new video into the library and look at the project properties, here's what I see
edit file format MPEG files
NTSC drop frame 29.97
MPEG files
24 bits, 720x480, 29.97
upper field first
(DVD-NTSC), 16:9

This seems to either be a default, or it's pulling the info from the files. All files seem to indicate 1920x1080 however.
If I go to Edit this, the general tab tells you the above.
Compression tab is
media type NTSC DVD
quality 70%
video format MPEG-2 (greyed out so can't change)
video data rate: variable 8000 kbps

If you perform non-square pixels rendering, or force pixels to square, do you change the 16:9 aspect, even though the pixel count is still 1920x1080? I ask because of your comment on squashing objects. I would have assumed that pixel aspect is disconnected from the frame aspect, but I've been wrong many times before when assuming too much.

Is there a good book on introductory video that answers all these questions (and the ones I have yet to ask)?
thanks again

[BrianCee]

You need to get the actual properties of the video itself - this is not necessarily your project properties - a difference between video properties and project properties is often the cause of problems with the final output.

to get the actual properties of your video - put it into the actual timeline and the RIGHT click over it and choose 'properties' from the fly out - this will tell us what the real properties of your original video are.

[byoung]

OK, just learned yet one more thing, thanks Brian. Here's what the Properties box has:
Quick time Movie Files
1920x1080
44.044 sec
1video track, 1 audio track

compression is blank
24 bits, 1920x1080
1,320 frames
29.970 fps
44.044 sec
start at 0.000

sound compression 16-bit Little Endian
48.000 KHz, 16bits, Stereo
2,114,112 samples
44.044 sec
0.000 sec

thanks.

[byoung]

As a follow up, I didn't see anything in the file properties related to what you call Field Order. Where else might I find this? The info from Canon is woefully inadequate. However, the video files from the camera are H.264.

Given all of this, what settings in VS ProX5 would you recommend to generate the best quality with a reasonable file size? You said that PhotoJPEG is an outdated compression format. What should I use?

Are any of these compression techniques lossless? i.e. do they compress the file size somehow without throwing away information? E.g. with stills, JPEG is a lossy file format, TIFF is not (and you can compress TIFFs losslessly as well).

I also have Adobe Lightroom which can output either DPX, whatever that is, or H.264 (similar to the input). Generated files show up as .mp4. I have no idea what compression is being applied; no options are given and no explanation. With a Max quality setting for H.264, is that a good way to go?

Unrelated, is there a way to save a template for Project Properties? Having to reset from the default every time is annoying.
thanks again
Bill

[Ken Berry]

When the Properties box does not indicate a Field Order, then the video is either frame based (if standard def) or progressive (if high def). So yours is obviously already 29.97 full (or progressive) frames per second. So that is number one on your path to an acceptable output.

And yes, I said MJPEG/PhotoJPEG are outdated formats, but that is not to say they are not still used -- as evidently the people who will be getting your video want it with that first format. It produces good quality but large files when compared to more modern formats such as AVCHD which is mpeg-4 produced with the H.264 codec.

And virtually all video formats are lossy apart from 'true' or uncompressed .AVI which is huge at around 65 GB per hour of video. The next best near lossless format is DV/AVI which is still large at 13 GB per hour. That means it has been compressed by a factor of about 5, but the algorithms used in the compression mean that further conversions in the same format produce almost no visible degradation in quality.

After that, though, most of the other common formats using some form of mpeg (mpeg-2, mpeg-4) use more compression and are more lossy, with repeated conversions producing noticeable loss in quality (though how many conversions it takes before the loss is apparent to the human eye is still debated). In high definition, the quality is so good that any loss on reconversion is probably going to be far less noticeable if using the same properties during the conversion.

Which brings us back to you. With your edited video in the timeline, have you tried to go to Share > Create Video File > Same As First Video Clip (if that option is not greyed out)? I am not sure it would be available since I don't know what happens when the first clip is MOV like your original format. But if it *is* available, then you should be able to at least produce a video in your original .mov format which is at least pretty close to what they are after.

Otherwise, you could go Share > Create Video File > Custom. Then in the dialogue box which appears, you would need to choose "Quick Time Movie Files (*.mov, *.qt)" in the 'Save As Type' box. Then you would need to go into the next box, and under the Compression tab, select at the top, under Compression, Photo JPEG from the drop down menu. Then, back on the General tab, you would need to make sure Frame Type is set to Frame Based and 29.97 fps, and under that, set 1920 x 1080 by ticking the User Defined button under Frame Size.

[byoung]

Ken, thanks again. This education is very useful.

You are correct when you surmised that the Same as First Video Clip would be greyed out.
To your last suggestion, that is essentially the template that I created initially which resulted in the huge files. One examples is a 20s clip at 1.73GB, or 86MB/s.

I also see that I have no choice on the output field, it all seems to be limited to frame-based. What is forcing this? Does it matter, given the input and project are upper frame first?
I've done some tests with the same 20s clip.
Saving a MOV with MPEG4 Visual compression at 100% yields 306MB/20s = 15MB/s. Same size is created reducing quality to 90%.
Saving as AVCHD via custom menu (yields .m2t file), with 100% quality yields 29MB/20s = 1.5MB/s. Quite a difference in the file size.
Saving as AVCHD preset 1920x1080p also yields 29MB.
Saving using the MPEG4-HD preset, the output is 33MB/20s = 1.7 MB/s.

I can't tell much of a difference visually between these, but then again, I'm new to video and don't necessarily know how these different settings would manifest themselves to my eyes.

As another reference, the Photoshop Lightroom export with H.264 at max quality yielded 56MB/20s = 2.8 MB/s, and shows a .mp4 file extension. I don't know the compression applied, nor how this file size relates to quality versus the other formats. Like the set above, it still looks good to me despite the file size being about half the original.

Any additional thoughts you have are much appreciated.