PSP Smart Carver & Photoshop Content Aware Scale

Corel Paint Shop Pro

Moderator: Kathy_9

Post Reply
JoeB
Posts: 2778
Joined: Fri Mar 28, 2008 10:04 pm
operating_system: Windows 8.1
System_Drive: C
32bit or 64bit: 64 Bit
motherboard: LENOVO 4524PE4 ThinkCentre M91p
processor: 3.10 gigahertz Intel Quad Core i5-2400
ram: 8 GB
Hard_Drive_Capacity: 4.6 TB
Corel programs: PSP 9, X7 to 2019, 32 & 64-bit
Location: Canada

PSP Smart Carver & Photoshop Content Aware Scale

Post by JoeB »

This post is my way of moving a topic that was becoming OT in the PSP 9 Wishlist thread into the main PSP thread as a separate subject. It was in the Wishlist thread I became aware of the Smart Carver tool in PSP, although I had used the Content Aware Scale tool in PS. I then wanted to test to see which tool might work best. The question is still open to some extent but I did one test leaving all settings in both programs on default except for setting them to resize the 800 x 534 image I started with, which I borrowed from @stormchaser in the thread he started at this link:

http://forum.corel.com/EN/viewtopic.php?f=56&t=57341

NOTE: For those who don't know, content aware scaling, or seam carving, is a way to resize an image INCLUDING changing its aspect ratio if desired, as well as removing some unwanted objects if desired, without distorting important or even most elements of the image by only removing pixels that an algorithm determines are only of minor importance to the image. Google those terms to learn more and view videos of various software in action if interested.

In any event, I opened the subject image in both PSP X8 and Photoshop CS6 and applied the tool to the image leaving settings at default EXCEPT to tell the programs to resize to 479 px while maintaining the aspect ratio (leaving an image of 479 x 320 px). I did NOT use masking tools available to remove or preserve any of the important elements, but that can be done in both programs for more specific effect output. I then resized the original input image (without rescaling) to the same pixel size just to make comparisons of all 3 images (original, PSP Smart Carve and PS Content Aware Scale) easier. I have attached all three.

In this example, and with this particular image, I believe PSP did the better job by far. If you compare both scaled images with the resized original, you'll see that in the original the birds are somewhat centered vertically over the water between the river banks. In the PSP scaled image there is still water above and below the birds - i.e., between the river banks - while in the PS scaled image the birds are crowding the river bank at the bottom and top. As well, in the PSP scaled image there is some water space above the flying birds between them and the 3 birds that are swimming in the water so that the swimming birds are all still untouched parts of the image and remain intact.

Also, in the PS scaled image, the third swimming bird (the one at right) has been crowded out by one of the flying birds and has, in fact, been partially erased. (NOTE that using masks or selections to preserve that bird would have prevented it from being distorted - an option also available in PSP's tool), but I was going with defaults).

And last, because the river in the PSP scaled image hasn't been squeezed as much as it has in the PS scaled image, the overhanging trees in the upper river bank are more in proportion to the original in the PSP scaled image but take up a bigger percentage of the image in the PS result.

In BOTH scaled images, the two flying birds at the back of the flight are moved closer to the others because of the removal of the less vital water and trees elements that separates them in the original input image.

In my very limited opinion, and based only on this one single test with default settings, it appears that PSP made the most intelligent decisions overall. That is, the PS program seemed to decide (correctly) that the water of the river had the least "energy" (i.e., was less important overall) and concentrated on removing pixels in that element to resize vertically, replacing them with the more detailed overhanging trees, but this resulted in distorting the third swimming bird, crowding the flying birds between the river banks, and putting the overhanging trees out of proportion to fill the space.

On the other hand, PSP's invocation of the algorithm seems to have concluded that some of the water (those portions near the top and lower banks and near the third swimming bird) were energetic enough (perhaps given its proximity to more energetic pixels) and preserved them (maybe just to attempt to preserve the third swimming bird and keeping the flying birds from impinging on the river banks). In other words, it seems to have concluded that the third swimming bird's pixels (and those of the river banks) were of sufficient energy (therefore importance) that it had to preserve some water space between them and the flying bird that, in the Photoshop image, impinged on the swimming bird and the banks. But of course that's just my speculation.

It should be noted that PS has an option to set a Reference Point Location for the algorithm to use, and it defaults to the center of the image, so I'm just assuming that this is generally considered the best place for it in most circumstances. I definitely don't have time at present to experiment with changing that option to see how it affects the result.

In any event, however, I do think that Smart Carver in PSP did a better job of this particular project, but willing - and look forward to - hearing other opinions or advice on this tool. It is definitely one that has its uses - albeit most useful and effective when your image has lots of open space, such as outdoor images with lots of trees, sand, water, sky, and etc. - as you might find by doing some Google searches and watching some videos if it interests you.

IMPORTANT NOTE!: You'll see one important use of seam carving as a resize tool compared to regular resizing methods by comparing the images. Regular resizing (as I did on the original input image), makes the birds all somewhat smaller overall because their size is resized smaller in proportion to everything else in the image, even though it is the birds that are the main subject of the image. In the seam carved scaled images, the birds are larger relative to the rest of the image even though the overall image is resized to 470 x 320 pixels just like the original, so maintain their place as the important parts and focal point of the image. In fact, their presence is enhanced overall. Of course, it depends on the effect you're trying to achieve, and perhaps the background environment with the birds being only part of it might be just as important to you, the photographer, as the flying birds. It's all personal preference - and I admit I like the resized original better overall with the birds only part of the overall effect, but only as modified in @stormchaser's original thread (link early in this post) with the last two birds closer to the others in the flight. :-)
Attachments
Original input file_resized to the others
Original input file_resized to the others
Paintshop Pro Smart Carver
Paintshop Pro Smart Carver
Photoshop Content Aware Scaling
Photoshop Content Aware Scaling
Regards,

JoeB
Using PSP 2019 64bit
photodrawken
Posts: 684
Joined: Wed Sep 26, 2012 8:40 am
operating_system: Windows 10
System_Drive: C
32bit or 64bit: 64 Bit
ram: 16Gb
Hard_Drive_Capacity: 324Gb
Location: USA

Re: PSP Smart Carver & Photoshop Content Aware Scale

Post by photodrawken »

JoeB wrote: In this example, and with this particular image, I believe PSP did the better job by far.
I disagree.

Neither result looks good -- the flying birds are grossly exaggerated in size, effectively "moving" them (visually) a few dozen yards closer to the camera, so that they look to be only a few feet away from the camera.

Therefore, the PSP result showing water beneath the flying birds does not make any sense with respect to the depth of the scene. Retaining the shadows of the flying birds on the water adds to the confusion and further degrades the logic of the depth of the scene.

At least in the PS result, those birds can be interpreted as flying over the riverbank near the camera (in other words, they appear visually to be somewhere between the camera and the edge of the water), which makes sense....
Ken
Yes, I think it can be eeeeeasily done...
Just take everything out on Highway 61.
User avatar
hartpaul
Advisor
Posts: 2893
Joined: Tue Mar 20, 2012 3:38 pm
operating_system: Windows 10
System_Drive: C
32bit or 64bit: 64 Bit
motherboard: ASUSTeK P7P55D STRIX B240F GAMING
processor: IntelCore i7 7700 3.60 Ghz
ram: 8 Gb
Video Card: Nvidia GeForce GTX 1050
sound_card: Nvidia High Definition Audio
Hard_Drive_Capacity: 1000 Gb
Monitor/Display Make & Model: AOC
Corel programs: PSP8,X2 to X9,2018,2019,2020
Location: Australia

Re: PSP Smart Carver & Photoshop Content Aware Scale

Post by hartpaul »

I think the second bird in the photoshop one has a shadow on the water as well. Both are unreal and show that relying on automatic operations for all scenes does not work. You would have to use this on only general background scenes which can work no matter how they are reconstituted by the software. When there are complicated backgrounds both will fail.
Systems available Win7, Win 8.1,Win 10 Version 1607 Build 14393.2007 & version 20H2 Build 19042.867
JoeB
Posts: 2778
Joined: Fri Mar 28, 2008 10:04 pm
operating_system: Windows 8.1
System_Drive: C
32bit or 64bit: 64 Bit
motherboard: LENOVO 4524PE4 ThinkCentre M91p
processor: 3.10 gigahertz Intel Quad Core i5-2400
ram: 8 GB
Hard_Drive_Capacity: 4.6 TB
Corel programs: PSP 9, X7 to 2019, 32 & 64-bit
Location: Canada

Re: PSP Smart Carver & Photoshop Content Aware Scale

Post by JoeB »

hartpaul wrote:I think the second bird in the photoshop one has a shadow on the water as well. Both are unreal and show that relying on automatic operations for all scenes does not work. You would have to use this on only general background scenes which can work no matter how they are reconstituted by the software. When there are complicated backgrounds both will fail.
First, I agree with @photodrawken and you that neither result looks good, which is why I said in my post that my preference was for the original image as modified in @stormchaser's original thread. This test, however, was not to determine which tool provided the best esthetically pleasing image. It was to determine which tool "appears" to make the most intelligent decisions regarding what is important in the image and what is less important (i.e., which pixels have the least energy and can be safely discarded and which have the most energy and should be protected, even when using default settings). That's because the whole purpose of the algorithm is to analyze an image and make a determination as to Important (high energy) elements and least important (low energy) pixels. In a relatively complicated scene such as this one that determination is not as easy as in general background scenes like seascapes, etc. So my opinion as to which did the best job was based solely on the fact that the PSP render kept important scene elements more proportionally positioned when compared to the original than did the PS render. That is, the flying birds are still centered over the river with approximately equal amounts of water above and below, there is still some space between the third swimming bird and the wing of the flying bird, the third swimming bird has not been distorted, and the overhanging trees are more proportional to the overall scene as compared to the original than is the PS render.

And yes, of course the birds are out of proportion because that is a natural result of the purpose of the algorithm. That is, its intent is to maintain important elements as close as possible to how they appear in the original while discarding less important elements of the scene to facilitate resizing and/or changing the aspect ratio of the overall image. The fact that it does that with both PSP and PS in this image while using default settings with no masking speaks to the effectiveness of the algorithm given its intent IMHO.

I definitely agree that this is not actually the type of image one would normally use these tools on but I wanted to make my single test a tough one :-) . I also definitely agree that in real world workflow a person would likely not be using strictly default settings even when working with an image more suited to the abilities of these tools.

I have attached the image but this time using Smart Carver for what might be considered the more traditional use of the tool. In @stormchaser's original thread he wanted the two geese in the rear moved closer to the others in the flight. So this time I used Smart Carver to move them closer by eliminating much of the space between the rear geese and the others. This resulted in my 800px wide image being output as 706px but still the original 534px high, which changed the aspect ratio. So I used Smart Carver again on this 706px image and had it resize vertically to make the output 706 x 471px thus maintaining the aspect ratio. I used the Preserve masking brush in both operations, but UNFORTUNATELY was a bit sloppy with it and failed to properly mask the right wing of the last flying bird near the bottom bank. In any event, here is the result along with the original. I decided not to go back and remask that bird's wing :-).

In any event, given this attached result, I would take back what I might have said or implied that this image might be a bit too complicated for what the algorithm/tools are meant to do. While JMHO and based on very limited testing, it would seem that good results can be obtained to remove objects (in this case unwanted water space) and resize even with an image like this as long as the available masking options are used properly. And one thing this option did compared to using various Move options to achieve @stormchaser's original goal was to maintain the proportions - that is, where moving the back birds with Move tools created more water space between the back birds and the right edge of the image thus changing the original image compositon, Smart Carver's result keeps that water space approximately proportionately the same as in the original so doesn't affect the original image composition in a way that might be unwanted.
Attachments
Geese - original image
Geese - original image
Moved geese and kept aspect ratio
Moved geese and kept aspect ratio
Regards,

JoeB
Using PSP 2019 64bit
Post Reply