Unacceptable transport stream encoding errors

[Rendo]

I just bought DVD MF6 because I was excited to see that it finally supported handling M2TS transport stream files. Well, things have headed south fast, and I don't really know why. Every several seconds or so during playback of the final SD-DVD, there are very noticeable and annoying encoding glitches visible in the form of clustered pixelation and picture breakup. It looks similar to when your HD cable feed signal is fading in and out and the picture breaks up into blocks.

Prior to this, I have used VideoReDo TV Suite and SVCD2DVD, both of which can also handle transport streams relatively well, and neither of those applications ever showed this kind of frequent encoding error and loss of picture. Unfortunately, neither of those programs does a very good job of creating DVD menus. I also don't care for their encoder's compression quality or somewhat reddish color alteration. (Yes, I know, the encoder theoretically isn't supposed to affect color...)

I was actually hoping that DVD MF6 would have much better encoding quality. Instead, I'm really disappointed with my first coaster. I'm using a P4 3.0 Ghz PC, so I don't think a slow PC is the problem. For when the picture is viewable, which at least is the majority of the time, I still would like to see much better encoding quality. The current situation with so many frequent picture glitches is not acceptable for use.

Is there some better method for using DVD MF6 with .TS files? I'm just adding video files to the project timeline like any other type of video file. Is there a way to avoid such a messy encode?

I do have Adobe Premiere Pro 2.0, but I am not impressed with its encoder, either, and I know I'm not the only one, especially at its price, and it doesn't handle .TS files without a $1200+ plugin.

Any help or advice would be appreciated, and no, I don't have $50K saved up for a professional hardware encoder. Still, isn't there a better, consumer level way to do this, or am I too far ahead of the curve?

Thanks!

[etech6355]

Did you purchase MF6+ (Plus Version)?
Have you installed patch# 2 or the HD-Add-On pack.

With the media file loaded on the timeline Right_Click on it and select "Media Properties", please post the Video and Audio Properties that are displayed by MF.

How or where did you get these source files in TS format? Do you know if they are true PAL / NTSC framerate or something like 24P/30P.

You posted m2ts file extension which usually is avc/h264 or mpeg2 video.

What are your "Project Settings" video and audio parameters set to, which is the MF encoder.

[Rendo]

Thank you for the reply and for trying to help. I would really like to see this work.

Did you purchase MF6+ (Plus Version)?

Yes, I bought the full package of DVD MF6 Plus and the HD Pack for $100 direct from Corel.

Have you installed patch# 2 or the HD-Add-On pack.

Yes, I installed Patch 1, then Patch 2, then the HD Add-On Pack.

With the media file loaded on the timeline Right_Click on it and select "Media Properties", please post the Video and Audio Properties that are displayed by MF.

File format: MPEG-2 (Transport stream)
File size: 515,118 KB
Duration: 272.973 seconds

Video type: MPEG-2 Video, Upper Field First
Total frames: 8,181 frame(s)
Attributes: 24 bits, 1920 x 1080, 16:9
Frame rate: 29.970 frames/sec
Data rate: 38810 kbps

Audio type: Dolby Digital Audio
Total samples: 13,102,703 Samples
Attributes: 48000 Hz
Layer: None
Bit rate: 384 kbps

How or where did you get these source files in TS format? Do you know if they are true PAL / NTSC framerate or something like 24P/30P.

You posted m2ts file extension which usually is avc/h264 or mpeg2 video.

The source files in .TS format are from my Motorola 3416 DVR (NTSC), captured using the standard method with CapDVHS. The files play back perfectly in VLC Media Player. I used VideoReDo TVSuite to edit them and save them again as .TS files, utilizing the Quick Stream Fix tool to ensure clean time codes and audio sync. When I encode the files to MPEG-2 DVD compliant using either VideoReDo TVSuite or SVCD2DVD, there is no such messy encode, although I personally think the result is noisier than I would like, and I'm not happy with how the resolution reduction and encode makes the output rather reddish on my TV, which is calibrated fairly well and has very good color settings for store bought DVD's. I don't discount that I might be missing tweaking some settings in any of these apps, but I'm not sure what to change. Would incorrect field order affect the color quality due to noisier interlacing?

What are your "Project Settings" video and audio parameters set to, which is the MF encoder.

When I select the default "HQ 16:9 (Dolby Digital)", this is what appears in the Save Options info window:

MPEG files
24 bits, 720 x 480, 29.97 fps
Frame-based
(DVD-NTSC), 16:9
Video data rate: Variable (Max. 7000 kbps)
Audio data rate: 256 kbps
Dolby Digital Audio, 48 KHz, 2/0(L,R)

I also clicked on Options and chose the Two-pass encode to hopefully improve the output quality.

Note that I have also tried to export just a single clip from the timeline using the default settings for HQ 16:9 (Dolby Digital), but the encode quality was just as sloppy. I thought that mixing clips of different properties might have confused the encoder, but it evidently wasn't a factor, as the encode is bad even with only one clip on the timeline.

If you can think of anything that I'm doing wrong here, or if there are settings that could be improved, I would be grateful for any assistance. Honestly, the manual doesn't go into all that much detail on some of these settings.

Thanks again for your help!

[etech6355]

Rendo,
Under MF6 Preferences make sure quality encoding/re-sampling is set to "BEST"

Your source video is fielded UFF (Upper Field First), so make sure to change the project settings to UFF. Click on the GEAR Icon "Change Mpeg Settings -> Customize". Same with exporting.

Try these settings:
24 bits, 720 x 480, 29.97 fps
Upper Field First
16:9

(DVD-NTSC),
Quality/Compression = 100%
Video data rate: Variable (Max. 8000 kbps)
Audio data rate: 256 kbps
Dolby Digital Audio, 48 KHz, 2/0(L,R)

[Rendo]

Thank you very much for the quick reply!

Under MF6 Preferences, quality encoding/re-sampling was already set to "BEST".

I have followed the other suggestions and set UFF, Quality to 100%, and Video data rate to 8000 kbps in the preferences and also saved a custom template for timeline export.

Unfortunately, the result of a sample export from the timeline still shows globs of sporadic pixelation and image breakup every several seconds or so. Basically, there is no improvement for the main problem of a messy looking encode.

Is there anything else I can try?

[etech6355]

All I can think of is that if they are DVB-T video files they need to be converted first. I'm not familiar with DVB-T yet.
I've read where many have to use software conversion tools first before they can edit these videos.
So, I'm lost on this, sorry.

[DVDDoug]

I have used VideoReDo TV Suite and SVCD2DVD, both of which can also handle transport streams relatively well... Unfortunately, neither of those programs does a very good job of creating DVD menus.

You should be able to use one of those programs to make a DVD compliant MPEG-2 file. Then, you can use Movie Factory to author & burn the DVD. If you check Do Not Convert Compliant MPEGs, Movie Factory won't re-code the MPEG (assuming it agrees that it's compliant). If SVCD2DVD can't make the MPEG-2 files without making a full DVD, SVCD2DVDMPG (same vendor and free) probably can.

...and I'm not happy with how the resolution reduction...

The maximum resolution for a regular NTSC SD-DVD is 720x480, and the maximum bitrate is around 10,000kbps combined audio & video. So, the 1920 x 1080 resolution will have to be reduced (unless you go with Blu-Ray or HD-DVD).

...and encode makes the output rather reddish on my TV...

I don't have any ideas on that. If you apply a color-correction filter, the MPEG will have to be re-coded which might degrade the image in other ways...



P.S.
I'm glad to see VideoReDo improving and expanding their program. I don't own it, but when I tried it, it impressed me as an easy to use, rock-solid program, that worked exactly as promised!

[Rendo]

All I can think of is that if they are DVB-T video files they need to be converted first. I'm not familiar with DVB-T yet.
I've read where many have to use software conversion tools first before they can edit these videos.
So, I'm lost on this, sorry.

Thank you for trying to help. I have no reason to believe these are DVB-T files because that is a European DTT standard, and I live in the U.S., which uses ATSC.

I just ran another sanity check using SVCD2DVD with the exact same source file, and there were no visible encoding errors like with DVD MF6+HD. So, it would appear that there is a very serious problem with the Ulead product for processing certain types of transport stream files. It is highly worth mentioning that the errors seem to occur at the same places in multiple encode attempts. This would suggest that this is not a random bug, but rather a specific inability for DVD MF6+HD to handle certain parts of the .TS file format, depending on how the MPEG TS source is encoded.

This is extremely disappointing, as I was hoping to simplify the workflow for making standard SD DVD-Video DVD's, not make it even more convoluted. At this time, MF6+ is of limited use to me beyond making DVD menus, although I would hope that it would be of much greater value when I attempt to make BD or AVCHD discs later this year, provided I don't have to use MF6+ to re-encode the files.

Should I submit a trouble ticket to Corel, or do they keep track of this forum? I know many tech companies do not closely monitor their user forums, so I was wondering what your experience has been on how I can get the developers to fix this crippling showstopper for re-encoding HD .TS files.

[Rendo]

You should be able to use one of those programs to make a DVD compliant MPEG-2 file. Then, you can use Movie Factory to author & burn the DVD. If you check Do Not Convert Compliant MPEGs, Movie Factory won't re-code the MPEG (assuming it agrees that it's compliant). If SVCD2DVD can't make the MPEG-2 files without making a full DVD, SVCD2DVDMPG (same vendor and free) probably can.

Yes and no. VideoReDo TVS does not appear to have the ability to save an HD file as an SD file. Sure, you can load a TS file and save it as an MPEG2 file, but you cannot change the resolution, which is a glaring oversight. If you want to create an SD DVD-Video compliant MPEG2 file, you have to go all the way to the point of creating the VOB files through the DVD creation step. There is no middle point, as it were, for just re-encoding an HD TS file to an SD MPEG2 file.

You're right that SVCD2DVD makes the MPEG2 file only when it also spends time creating the full DVD folders, which would eventually get manually deleted in this scenario. Oddly, the program gives you the option to not create the MPEG2 stand-alone files, not the other way around, which is what we need. Still, at least the MPEG2 files created by SVCD2DVD show no such sloppy encode mess like DVD MF6+HD does, but as I've already stated, I'm not terribly impressed with the compression quality. The matrix parameters to alter this are not well documented, so tweaking the compression settings is much less straightforward than with other apps. I think that using the other free utilities from that developer such as SVCD2DVDMPG usually means sacrificing some quality with older encoding engines and no two-pass option.

The maximum resolution for a regular NTSC SD-DVD is 720x480, and the maximum bitrate is around 10,000kbps combined audio & video. So, the 1920 x 1080 resolution will have to be reduced (unless you go with Blu-Ray or HD-DVD).

While I understand that the resolution obviously has to be reduced to create a standard SD DVD-Video, what I don't fully understand is if it is possible to avoid a re-encode of the HD TS files before copying ("authoring") them to BD. Since DVD MF6+HD evidently cannot properly handle this type of HD TS file format, if a re-encode would be required, it would make the product unusable for the purpose. I suppose I could attempt to create an AVCHD disc, but it would be rather academic at this point since I don't yet own a BD player. (I'm holding out for a final spec player that also plays BD-R's for under $400.)

If you apply a color-correction filter, the MPEG will have to be re-coded which might degrade the image in other ways...

Agreed. I want to avoid another re-code at all costs, of course, which is why I was looking for other ways to improve the picture quality during the one-time encode. I would also be concerned that applying a color correction filter would cause other types of image degradation.

I'm glad to see VideoReDo improving and expanding their program. I don't own it, but when I tried it, it impressed me as an easy to use, rock-solid program, that worked exactly as promised!

I think VideoReDo TVS is a fantastically simple yet powerful tool for editing HD TS files and preserving their image quality when resaving them, and it easily justifies the purchase. Unfortunately, using the application to create DVD menus is another story entirely, and I consider the application basically unusable for that purpose. I also am not overly pleased with the down-rezzing compression quality when creating an SD DVD, hence my search for a higher quality encoder. Since VideoReDo does not appear to be able to save SD MPEG2 files as an interim step, I find myself having to use SVCD2DVD just to convert the HD TS files to SD MPEG2 DVD compliant ones, and then throw away the DVD-Video folders that SVCD2DVD creates.

So, you can see why it would be just peachy wonderful if DVD Movie Factory 6 Plus with the HD Plug-in actually worked with HD transport stream files captured from the Motorola HD DVR, a very common hardware choice with U.S. cable TV companies.

[etech6355]

Rendo,
I can't imagine that the source videos are 24FPS inside of a 60i TS. That would require pull-down.
I'm curious what module are you in? The standard DVD module, avchd, blu-ray etc. You can custom export from any of the above modules.

I have come across a small bug/issue with MF6+ and is related when going from HD to SD. That is in the avchd or blu-ray modules and converting from one codec to another (mpeg2 to avc/h264).
To fix that I use SD source mpeg2 720x480 instead of the HD file. Not sure if it was pixelation but something in the frame rate because when you step the video every so many frames one frame would backup and skip back to the previous frame (or field).

Four things to try:
1) If VideoRedo can export the TS file in PS format then being in the PS format it will re-align buffers & timestamps making the video easier to convert or edit.

2) Make sure your in the DVD creation module, set your project settings to the SD encodes you want to convert to and export the video by using the "Export -> Fast Export DVD compliant" (The problem could be in the HD module)

3) Under Preferences (F6 Hotkey), on one of the tabs mentions to "Create Index File to increase performance" (or similar wording). Maybe check that on.

4) Start a new Blu-Ray project, under Gear check on both "X-Disc" & "Do not convert compliant etc"
Load the video, uncheck create menus, go to the burning module & create Blu-Ray disc folders. If you see converting video title of title X then
MF see's the video as non-HD compliant. The blu-ray module will convert the audio to your project settings (this is normal) if it is Mpeg audio, MF also will tell you it's only converting the audio.
After the blu-ray disk folders are created test the video in the STREAMS folder. The video will actually be in a different container format. Maybe MF6 can read this container format better. Yes or No it's still good to know if MF6 is seeing the video as HD compliant or not.

If you could host the file I could download it and see what I can come up with.
I'm curious if your having the same problem I came across when down converting video, but that also included changing the encoding codec. I don't believe it was going from hd-mpeg2 to sd-mpeg2.

[Rendo]

etech6355,

Thank you for some very interesting suggestions.

I'm curious what module are you in? The standard DVD module, avchd, blu-ray etc. You can custom export from any of the above modules.

I've only tried using the standard DVD module up to this point.

Four things to try:
1) If VideoRedo can export the TS file in PS format then being in the PS format it will re-align buffers & timestamps making the video easier to convert or edit.

Yes, I have just now tried that, and MF6+ thankfully works without a sloppy encode, but haven't I now "touched" the file twice by encoding it once in VideoReDo as an MPEG PS file and then again in MF6+ as an SD MPEG2 file? The original HD TS file has a data rate of 38810 kbps, whereas the the MPEG PS file has a data rate of "Variable bit rate (Max. 16200 kbps)", so I'm definitely confused if this is a valid victory or not. Yes, the sloppy encode issue appears to be resolved by using a program stream instead of a transport stream, but the data rate was cut by nearly 60% to get there. Ouch. Yet, how relevant is this given that the SD DVD-Video end result is only 8000 kbps anyway? I'm not completely sure I know the answer, but as a rule of thumb I always try to preserve image quality as losslessly as possible up until the point of one and only encode, by whatever application.

2) Make sure your in the DVD creation module, set your project settings to the SD encodes you want to convert to and export the video by using the "Export -> Fast Export DVD compliant" (The problem could be in the HD module)

No change in problem - sloppy encode still occurs using source HD TS file.

3) Under Preferences (F6 Hotkey), on one of the tabs mentions to "Create Index File to increase performance" (or similar wording). Maybe check that on.

No change in problem - sloppy encode still occurs using source HD TS file.

4) Start a new Blu-Ray project, under Gear check on both "X-Disc" & "Do not convert compliant etc"
Load the video, uncheck create menus, go to the burning module & create Blu-Ray disc folders. If you see converting video title of title X then
MF see's the video as non-HD compliant. The blu-ray module will convert the audio to your project settings (this is normal) if it is Mpeg audio, MF also will tell you it's only converting the audio.
After the blu-ray disk folders are created test the video in the STREAMS folder. The video will actually be in a different container format. Maybe MF6 can read this container format better. Yes or No it's still good to know if MF6 is seeing the video as HD compliant or not.

I selected BDMV for the BD project type, and when asked if I wanted to modify the project properties to match the video file properties for SmartRender, I clicked Yes. I made sure "X-Disc" and "Do not convert compliant MPEG files" were on in the Project Settings. When I created the folders, it whizzed through the process in a matter of seconds for an 18-second excerpt TS file I had saved earlier in VideoRedo. The resulting .m2ts file in the BDMV\STREAM folder showed NO sloppy encode problems. I then tried the entire clip of about 4 1/2 minutes, and it took only about a minute to do the Video/Audio Muxing part of the process, and there was no sloppy encode evident in the .m2ts file since I assume there was no video recoding. I think this test proves that the HD MPEG TS source file is indeed HD compliant??? I don't think I saw much going on in a way of converting, as that would have taken a much longer time. On a positive note, it looks like there is a strong hope that I can actually use these TS files with DMF6+ to create BD discs when I finally get a PC BD burner and STB BD player. In the meantime, though, I'm stuck like most people with only being able to practically create standard SD DVD's.

If you could host the file I could download it and see what I can come up with.
I'm curious if your having the same problem I came across when down converting video, but that also included changing the encoding codec. I don't believe it was going from hd-mpeg2 to sd-mpeg2.

That's very generous of you! Thank you! I think I'll save a short excerpt of around 30 seconds before and after MF6+ encode so that you see what I see and then perhaps you can replicate the problem. I'll work on getting these two files uploaded later today or tomorrow. Thanks so much!!!

[DVDDoug]

Rendo,

I know of a couple of free utilities that are supposed to convert transport streams to program streams (without re-coding, I think):

PVAStrumento
ProjectX

However, I'm not sure if that will help since your problems are showing-up during the re-coding to DVD-compatible MPEG-2.

Doug.

[etech6355]

PVAStumento! I couldn't remember the name of that program, thanks!

The original HD TS file has a data rate of 38810 kbps, whereas the the MPEG PS file has a data rate of "Variable bit rate (Max. 16200 kbps)", so I'm definitely confused if this is a valid victory or not.

Making some progress right?

Bit-Rates are confusing the 38810 is a max bit rate. Video encoders use 3 settings MAX/Average/Minimum. I have many videos that are 25,000kbs yet average about 18,000kbs.
VideoRedo tells you the true average bit-rate of that video and the max it will peak at using Variable bit rate.
kbs means Kilo_Bits_Per_Second
The video is 4.9 Minutes, if the bit-rate was 38810kbs then the file should be - ((38810k * 60) / 8 ) = 291 MegaBytes Per Minute, so 291x4.9=@1.426 GigaBytes if 38810kbs was the true bit-rate.
The video is really 515.118Megs and plays for 4.9 minutes, so for a minute thats (515.118 / 4.9) = 105 MegaBytes per Minute.
Convert that to MegaBytes per second (105/60)=1.75MegaBytes per Second.
1.75 MegaBytes converted to BITS is (1.75 * 8 ) = 14MBS or 14,000kbs (Kilo_bits_per_second).
So your video averages 14,000kbs (thats the combination of the video & audio bit rates added together).

I don't think VideoRedo is recoding your high definition video. If it is, this doesn't matter, it should still have great high definition and more than adequate resolution to down convert to Standard Definition. I've recoded many HDV's before converting them to SD. They still look great in SD.

I was hoping you could insert the xxxxx.m2ts file from the BDMV directory structure (STREAMS folder) to downconvert to SD. Unless I missed you did that down conversion yet. That would prove MF doesn't like the container format.
Converting from TS to PS works, that's good.
I've use Avidemux2 to re-muxing TS back to PS. It's seamless & very fast, you can also replace the audio track with another.
It's a freeware program.
The program "SUPER" will also do this. It has a VOB mode, no re-encodes at all, simply re-multiplexes the stream. That would be another solution, maybe, VideoRedo may also be fixing things inside the video..

[Rendo]

DVDDoug,

Thank you for the suggestions.

I know of a couple of free utilities that are supposed to convert transport streams to program streams (without re-coding, I think):

PVAStrumento

I downloaded and tried out PVAStrumento. It splits the TS file into separate MPEG program stream and AC3 audio files. When it comes across something it doesn't like, the default settings discard data so that during playback, there are chunks of frames suddenly missing. I suppose I could investigate further and tweak the settings, but the bottom line is that when I take just the video MPEG file into the DMF6+ timeline and look at the properties, it shows "Variable bit rate (Max. 17239 kbps)", so I'm not sure this tool does any better of a job than VideoRedo to save TS files as PS files without recoding or affecting the bitrate.

(Das Hilfe Forum ist auf Deutsch, und ich kann Deutsch nuer ein bisschen.)

ProjectX

The downloadable sourcecode requires a Java compile... think I'll pass on this tool while it's in development, but thanks for the tip.

However, I'm not sure if that will help since your problems are showing-up during the re-coding to DVD-compatible MPEG-2.

The sloppy encodes are happening when I use an HD transport stream file on the DMF6+ timeline. If I use a program stream file that has the questionable ~60% data rate reduction, then the resolution reduction and encode seems to work without a sloppy encode. I'm not really sure this is the right solution, though. I think the best solution would be for the HD TS processing bugs to be fixed in DMF6+. I have two other apps that don't have this issue with TS files, but they have other peculiar limitations and frustrations, as I've already noted.

Do you have any idea if the HD transport stream encoding errors issue is already being addressed in the next DMF6+ patch, if there is one?

Thank you again for trying to help.

[Rendo]

etech6355,

If the average bit rate of the original TS video is around 14 mbps, then the VideoRedo PS file having a maximum of around 16mbps is still higher.

Here's the math on a small 18 second excerpt of the larger file, but saved from VideoReDo in Program Stream format at HD 1080i resolution, with a variable max data rate of 16.2 mbps:

The video is 34.914 MB and plays for 18.519 seconds, or ~1.89 MB/sec, which is a data rate of about 15 mbps. That means this 18 second clip actually has a slightly higher average data rate than the total 4 1/2 minute file and is rather close to the maximum variable bit rate of 16.2 mbps.

So, I guess this is supposed to give me a warm fuzzy to go ahead with using PS files saved from VideoReDo.

But wait, there's more...

I was hoping you could insert the xxxxx.m2ts file from the BDMV directory structure (STREAMS folder) to downconvert to SD. Unless I missed you did that down conversion yet. That would prove MF doesn't like the container format.

I'm sorry if I didn't pick up on the subtlety of that request, but now I understand, and the results are VERY interesting! I took the 18 second BDMV .m2ts clip from the STREAMS folder, put it on a timeline for a standard SD DVD-Video, exported it using the settings we agreed upon earlier, and there are NO sloppy encode errors. Wow! So, I think that proves that DMF6+ doesn't like the original TS container format? What exactly does that mean?

So, which method do you think will give better quality?
1) Using program stream files saved from VideoRedo with a variable max. data rate of 16.2 mbps. (Point and click simplicity for easier workflow.)
or
2) Using DMF6+ to create BDMV .m2ts files that keep the original 38.8 mbps data rate. (More convoluted in terms of workflow.)

Unless I'm missing something important, I'm not sure I see a significant difference in terms of source material to down convert to SD, given the data rate calculation comparisons we've done.

I've use Avidemux2 to re-muxing TS back to PS. It's seamless & very fast, you can also replace the audio track with another.
It's a freeware program.
The program "SUPER" will also do this. It has a VOB mode, no re-encodes at all, simply re-multiplexes the stream. That would be another solution, maybe, VideoRedo may also be fixing things inside the video..

I have just downloaded those two programs and will take a look at them tomorrow.

You also got me thinking about whether or not VideoReDo might be doing something to the TS container when the file is edited and resaved as a TS file. I'm currently getting inconsistent and inconclusive results, so I will have to continue testing tomorrow after I run a longer render overnight for an original capture file not yet touched by VideoReDo. In the meantime, I will PM you the link for the 18 second file sample, before and after a sloppy encode. If you have time to look at it, thank you. Perhaps you have a tool to look at the container and go "Oh, there's the problem!"?

Thanks again for your help. This has certainly become one heck of a learning experience.