Very Poor & Blocky h.264 Output

Moderator: Ken Berry

Aaron407
Posts: 5
Joined: Sat Jan 23, 2021 4:19 pm
operating_system: Windows 10
System_Drive: C
32bit or 64bit: 64 Bit
motherboard: Hewlett-Packard
processor: Core i7-4700MQ
ram: 16GB
Video Card: Nvidia GeForce GT 740M
Hard_Drive_Capacity: 1TB

Very Poor & Blocky h.264 Output

Post by Aaron407 »

I've been fighting with Videostudio's h.264 encoder for some time and through multiple versions. The output that I get with it is quite poor with heavy macroblocking, seemingly regardless of the output bitrate. Encoding to wmv or hevc works better in terms of this quality issue, but I can't tweak the settings as much as I need to with wmv (e.g., can't have 60 fps with wmv and can't set audio bitrate/sampling rate), and my system isn't properly compatible with hevc for playback. I can't understand why the output is so bad with mp4, and, in the 2022 version, I can literally set the bitrate at 650 Mbps (not a typo...) and it still puts out blocky video. I'm not using proxy files, which I know can cause issues as well. It also doesn't matter whether or not I use hardware acceleration, nor whether I use main or high profiles.

I can link to some sample outputs, but does anyone know why it can't perform well for h.264, even at high bitrates? The WMV output seems to be very high bitrate and is worlds ahead of the h.264 output. I know it's not an inherent issue with h.264 mp4 as I've used it successfully with many other programs, and I haven't managed to find a way to apply a different driver than the Ulead one.

Is there just a fundamental problem with the Ulead MPEG-4 vio Driver? I was hoping that the 2022 version had fixed this, but I'm regretting my decision to upgrade.
tletter
Posts: 1221
Joined: Thu Jul 16, 2015 12:23 am
operating_system: Windows 10
System_Drive: C
32bit or 64bit: 64 Bit
processor: i7-3632QM
ram: 16GB
Video Card: NVIDIA RTX 3080
Hard_Drive_Capacity: 1TB
Corel programs: X4,X5,X6,X7,X8,X9,X10,2018,2019,2021
Location: Canada

Re: Very Poor & Blocky h.264 Output

Post by tletter »

Aaron407 wrote: I've been fighting with Videostudio's h.264 encoder for some time and through multiple versions. The output that I get with it is quite poor with heavy macroblocking,
FWIW, I use VideoStudio's H.264 High Profile Video encoder frequently and don't have the issue being reported. However, if difficulties are encountered, sometimes turning off Smart Render, and Hardware encode acceleration helps, as does allowing VS to match the properties of the first video clip.
Aaron407 wrote:I can literally set the bitrate at 650 Mbp
100Mbps is tops in my experience, so how is this done, and what produces video clips at such a extraordinary frame rate?

tletter
https://www.youtube.com/user/tletter
User avatar
Ken Berry
Site Admin
Posts: 22104
Joined: Fri Dec 10, 2004 9:36 pm
operating_system: Windows 11
System_Drive: C
32bit or 64bit: 64 Bit
motherboard: Gigabyte B550M DS3H AC
processor: AMD Ryzen 9 5900X
ram: 32 GB DDR4
Video Card: AMD RX 6600 XT
Hard_Drive_Capacity: 1 TB SSD + 2 TB HDD
Monitor/Display Make & Model: Kogan 32" 4K 3840 x 2160
Corel programs: VS2022; PSP2023; DRAW2021; Painter 2022
Location: Levin, New Zealand

Re: Very Poor & Blocky h.264 Output

Post by Ken Berry »

It might also be useful to know what the original video format is and what it comes from. Is it mp4 from a phone? Many phones use mp4 as their standard format, and it is not h.264 mp4. So if from a phone you could look at its properties in VS (right click a clip in the timeline) to see what its says about the format. If mp4 but not h.264, you should be able to insert such a clip first in a project, then later after editing, when you get to Share, choose "Same as first clip". Or as tletter has said, when you insert a clip first in the project you should get a message asking if you want to Project Settings to match those of the clip if you have Preferences > General set to pose that question. Then later, in Share, you can choose "Same as Project Properties".

While getting the clip Properties, and especially if the video comes from a phone, check to see if the video is using a standard frame rate (e.g. 25, 29.97, 30, 50, 59.94 or 60 fps). If using a variable frame rate, which a lot of smartphones do these days, VS usually has difficulties. It might then be better to convert the video using an external converter to an mp4 which uses one of these standard frame rates but otherwise similar properties to the original. Then insert the new mp4 in VS for further editing.

Or is your video from a GoPro Hero? That films using either h.264 but can also be set to film in HEVC h.265. Both produce video which is also called mp4. But if the latter (H.265), you need to have the HEVC Extension from Microsoft installed. It costs next to nothing. We recently had someone who had his Hero set to film in HEVC but didn't realize it, and he didn't have that extension installed . And he had VS set to use h.264. He also poor output. But it got better when he either filmed in h.264 or used h.265 with the proper HEVC Extension installed.

But that's about all I can think of for the moment. I have been using both the standard mp4 in high def and also h.264 (and h.265) mp4 for a long time now in successive versions of VS and like tletter don't have the type of problem you describe.
Ken Berry
Aaron407
Posts: 5
Joined: Sat Jan 23, 2021 4:19 pm
operating_system: Windows 10
System_Drive: C
32bit or 64bit: 64 Bit
motherboard: Hewlett-Packard
processor: Core i7-4700MQ
ram: 16GB
Video Card: Nvidia GeForce GT 740M
Hard_Drive_Capacity: 1TB

Re: Very Poor & Blocky h.264 Output

Post by Aaron407 »

tletter wrote: Tue Sep 27, 2022 4:31 am
Aaron407 wrote: I've been fighting with Videostudio's h.264 encoder for some time and through multiple versions. The output that I get with it is quite poor with heavy macroblocking,
FWIW, I use VideoStudio's H.264 High Profile Video encoder frequently and don't have the issue being reported. However, if difficulties are encountered, sometimes turning off Smart Render, and Hardware encode acceleration helps, as does allowing VS to match the properties of the first video clip.
Aaron407 wrote:I can literally set the bitrate at 650 Mbp
100Mbps is tops in my experience, so how is this done, and what produces video clips at such a extraordinary frame rate?

tletter
https://www.youtube.com/user/tletter
In past versions, it would generally limit the bitrate to a much lower figure as a function of codec, framerate, and resolution. h.264 high profile for 1080p60 now allows up to 650,000 kbps for some reason, which you'd swear is an order of magnitude higher than it should be.

Here's an allowable configuration for example where it's set to 650,000 kbps for the video bitrate: https://drive.google.com/file/d/118h7TW ... XX7K6_yaYq

Here are some examples of the comparative output issue. If you take a look at detail in the shadow, as well as the sky, you'll see that there's heavy macroblocking in the h.264 example, but it looks good with wmv and hevc. Even with the absolute highest bitrate setting for h.264, the blocking is still there.

*Make sure you download the files rather than view through google's built-in viewer.

h.264: https://drive.google.com/file/d/115Yeei ... EMR8U0V25m
wmv: https://drive.google.com/file/d/114BSEq ... v9-RtGlz_L
hevc: https://drive.google.com/file/d/1141yYb ... DaRxWEpxOu

(Hopefully the google drive links work.)
Aaron407
Posts: 5
Joined: Sat Jan 23, 2021 4:19 pm
operating_system: Windows 10
System_Drive: C
32bit or 64bit: 64 Bit
motherboard: Hewlett-Packard
processor: Core i7-4700MQ
ram: 16GB
Video Card: Nvidia GeForce GT 740M
Hard_Drive_Capacity: 1TB

Re: Very Poor & Blocky h.264 Output

Post by Aaron407 »

Ken Berry wrote: Tue Sep 27, 2022 4:48 am It might also be useful to know what the original video format is and what it comes from. Is it mp4 from a phone? Many phones use mp4 as their standard format, and it is not h.264 mp4. So if from a phone you could look at its properties in VS (right click a clip in the timeline) to see what its says about the format. If mp4 but not h.264, you should be able to insert such a clip first in a project, then later after editing, when you get to Share, choose "Same as first clip". Or as tletter has said, when you insert a clip first in the project you should get a message asking if you want to Project Settings to match those of the clip if you have Preferences > General set to pose that question. Then later, in Share, you can choose "Same as Project Properties".

While getting the clip Properties, and especially if the video comes from a phone, check to see if the video is using a standard frame rate (e.g. 25, 29.97, 30, 50, 59.94 or 60 fps). If using a variable frame rate, which a lot of smartphones do these days, VS usually has difficulties. It might then be better to convert the video using an external converter to an mp4 which uses one of these standard frame rates but otherwise similar properties to the original. Then insert the new mp4 in VS for further editing.

Or is your video from a GoPro Hero? That films using either h.264 but can also be set to film in HEVC h.265. Both produce video which is also called mp4. But if the latter (H.265), you need to have the HEVC Extension from Microsoft installed. It costs next to nothing. We recently had someone who had his Hero set to film in HEVC but didn't realize it, and he didn't have that extension installed . And he had VS set to use h.264. He also poor output. But it got better when he either filmed in h.264 or used h.265 with the proper HEVC Extension installed.

But that's about all I can think of for the moment. I have been using both the standard mp4 in high def and also h.264 (and h.265) mp4 for a long time now in successive versions of VS and like tletter don't have the type of problem you describe.
The original video format is also h.264 high profile, but a higher resolution (4k). In the example I posted above, it came from a phone, and I was trying to create with a mix of resolutions (which means some re-encoding). However, I have had the same issue with matched resolution h.264 mp4s sourced from a DJI drone, an older GoPro Hero 4 Session, as well as AVCHD from a handheld camcorder. It doesn't seem to matter much what the input is, the h.264 output is subpar, and it gets worse when any color correction is applied. The example in the links was a file that shows 60 fps in its properties, but I've had the issue with 30 and 60 fps sources.

As for HEVC, I have the extension installed, but my computer is getting a bit long in the tooth and doesn't seem to want to play it back smoothly. For the few frames I can see, though, the blocking doesn't seem to be present.

Any additional thoughts? I'm all ears as I'd really love to get a solid, well-compatible output.