Any tools or methods to recover a corrupted PSPimage (2020)

[dangelic0]

I have a 50+ layer PSPimage file that was somehow corrupted. I look at it with a binary editor and see much of it looks intact (lots of readable PSP text and image data inside it). Have Corel or others on this forum ever found a way to recover a corrupted file?

I don't know how it was corrupted, but PSP did not crash, so there was no "Autosave" file. Here is the workflow using PSP 2020

1) Import a bunch of (50+) images into layers in one master image. Save as PSPimage file. Close
2) Open that PSPimage and manipulate the layers (move them around, change blend modes, etc.). Save. Close
3) Open again (all fine). Do some more manipulation. Save.
4) Run script that walks the layers and saves out individual JPGs. This script works fine and I have all my outputs.
5) Exit PSP. It asks me if I want to save the PSPimage. I say no - don't need to due to #3 above
6) Come back later. Try opening the PSPimage
"Unable to open file. Please verify that the file is valid."

I can post the PSP image file if it would help (~33MB). Nothing fancy. Around 50 layers, but no masks. All Raster

Thanks
Dean

[Jean-Luc]

In my experience, PSP is unreliable when working with a lot of layers you handle. You have to save regularly with a different name for each saving.
If your file cannot be read by your PSP, it will not be read by ours, IMHO.

[LeviFiction]

So, the answer is no there is currently no recovery tool available.

It is, however, a desire of mine to see if I can at least diagnose a file corruption. I have a base understanding of the PSP Image format. If you're willing to post a link to it I can try taking a look when I get home tonight. But there is no guarantee I can find a solution.

If nothing else a basic script of all actions taken should be recoverable if you have history saving turned on.

[dangelic0]

https://www.dropbox.com/s/k3phw7swyso71 ... image?dl=0

I see reference to my 59 named layers in here, but only "Layer Properties" for the first two.
Layer names are:

Black
20200213_D50_1489_Plastic_Fire
20200213_D50_1444_Plastic_Fire
20200213_D50_1445_Plastic_Fire
20200213_D50_1449_Plastic_Fire
20200213_D50_1450_Plastic_Fire
20200213_D50_1455_Plastic_Fire
20200213_D50_1456_Plastic_Fire
20200213_D50_1460_Plastic_Fire
20200213_D50_1470_Plastic_Fire
20200213_D50_1465_Plastic_Fire
20200213_D50_1471_Plastic_Fire
20200213_D50_1474_Plastic_Fire
20200213_D50_1475_Plastic_Fire
20200213_D50_1477_Plastic_Fire
20200213_D50_1481_Plastic_Fire
20200213_D50_1482_Plastic_Fire
20200213_D50_1485_Plastic_Fire
20200213_D50_1489_Plastic_Fire
20200213_D50_1490_Plastic_Fire
20200213_D50_1493_Plastic_Fire
20200213_D50_1494_Plastic_Fire
20200213_D50_1497_Plastic_Fire
20200213_D50_1504_Plastic_Fire
20200213_D50_1505_Plastic_Fire
20200213_D50_1498_Plastic_Fire
20200213_D50_1508_Plastic_Fire
20200213_D50_1509_Plastic_Fire
20200213_D50_1511_Plastic_Fire
20200213_D50_1522_Plastic_Fire
20200213_D50_1529_Plastic_Fire
20200213_D50_1537_Plastic_Fire
20200213_D50_1536_Plastic_Fire
20200213_D50_1543_Plastic_Fire
20200213_D50_1548_Plastic_Fire
20200213_D50_1549_Plastic_Fire
20200213_D50_1554_Plastic_Fire
20200213_D50_1555_Plastic_Fire
20200213_D50_1561_Plastic_Fire
20200213_D50_1566_Plastic_Fire
20200213_D50_1567_Plastic_Fire
20200213_D50_1573_Plastic_Fire
20200213_D50_1579_Plastic_Fire
20200213_D50_1585_Plastic_Fire
20200213_D50_1596_Plastic_Fire
20200213_D50_1591_Plastic_Fire
20200213_D50_1597_Plastic_Fire
20200213_D50_1603_Plastic_Fire
20200213_D50_1611_Plastic_Fire
20200213_D50_1612_Plastic_Fire
20200213_D50_1616_Plastic_Fire
20200213_D50_1617_Plastic_Fire
20200213_D50_1624_Plastic_Fire
20200213_D50_1630_Plastic_Fire
20200213_D50_1636_Plastic_Fire
20200213_D50_1637_Plastic_Fire
20200213_D50_1640_Plastic_Fire
20200213_D50_1643_Plastic_Fire
20200213_D50_1489_Plastic_Fire

[dangelic0]

What I really need from this corrupted PSPimage are the positions of the layers. I see two pieces of data therefore that could help me. Otherwise I have ~50+ JPGs to patch

1) If the Move commands can somehow be extracted. I see a lot of them in here.

Question for you Professor Levi: are the HISTORY commands saved for all editing sessions? Or only since the final save? I would need all moves across several editing sessions

2) If I could somehow determine the layer's position based on the transparency data. That is, if the raster layer is defined simply as an array of RGB values, could I somehow recognize the "no data" transparency channel from the upper left corner and determine the position that way.

Thinking out loud here.
Thanks
Dean

[LeviFiction]

1) It should be all of the history each session separated by a date code.

2) Each layer has an image rectangle giving the position and size of the layer. ((x,y),width,height) So, as long as all of the layers are in the image that information should be intact.

If they aren't then they should be available in the history.

Sorry I haven't had a chance to look at it yet. But I hope to investigate today.

[LeviFiction]

So, still looking everything over. The majority of the format looks complete and fine. My PSP Reader script isn't complete yet. So I'm going through it slowly. The output shows all of the layers and their X,Y positions. I can also make it print the transparency, layer types, links, blend ranges, and blend modes if you would like. I will also get working on making a human readable version of the edit history.

Code: Select all

PSP X8 - 2020 compatible Format
PSP_IMAGE_BLOCK
Chunk Size: 46
Width: 5568
Height: 3132
Resolution: 300.0
Metric: PSP_METRIC_INCH
Compression Type: PSP_COMP_LZ77
Bit Depth: 24
Plane Count: 1
Color Count: 16777216
Greyscale?: 0
Total Image Size: 3034381824
Active Layer: 9
Layer Count: 58
Graphic Contents: keGCMergedCache
Graphic Contents: keGCRasterLayers
Graphic Contents: keGCCompositeTransparency
Graphic Contents: keGCComposite
Graphic Contents: keGCThumbnail
Expansionn field: False
PSP_EXTENDED_DATA_BLOCK
PSP_CREATOR_BLOCK
PSP_COMPOSITE_IMAGE_BANK_BLOCK
PSP_LAYER_START_BLOCK
	Black : (0, 0)
	20200213_D50_1444_Plastic_Fire : (0, 0)
	20200213_D50_1445_Plastic_Fire : (-150, 0)
	20200213_D50_1449_Plastic_Fire : (0, 0)
	20200213_D50_1450_Plastic_Fire : (50, 50)
	20200213_D50_1455_Plastic_Fire : (-50, 600)
	20200213_D50_1456_Plastic_Fire : (0, 100)
	20200213_D50_1460_Plastic_Fire : (96, 50)
	20200213_D50_1470_Plastic_Fire : (-199, -349)
	20200213_D50_1465_Plastic_Fire : (-62, 0)
	20200213_D50_1471_Plastic_Fire : (0, 0)
	20200213_D50_1474_Plastic_Fire : (100, 50)
	20200213_D50_1475_Plastic_Fire : (-33, -2331)
	20200213_D50_1477_Plastic_Fire : (-156, 860)
	20200213_D50_1481_Plastic_Fire : (0, 0)
	20200213_D50_1482_Plastic_Fire : (0, 0)
	20200213_D50_1485_Plastic_Fire : (0, 0)
	20200213_D50_1489_Plastic_Fire : (-96, -119)
	20200213_D50_1490_Plastic_Fire : (0, 0)
	20200213_D50_1493_Plastic_Fire : (165, 39)
	20200213_D50_1494_Plastic_Fire : (0, 0)
	20200213_D50_1497_Plastic_Fire : (157, 1415)
	20200213_D50_1504_Plastic_Fire : (0, 0)
	20200213_D50_1505_Plastic_Fire : (16, -66)
	20200213_D50_1498_Plastic_Fire : (-66, 1060)
	20200213_D50_1508_Plastic_Fire : (103, 494)
	20200213_D50_1509_Plastic_Fire : (0, 0)
	20200213_D50_1511_Plastic_Fire : (0, 0)
	20200213_D50_1522_Plastic_Fire : (234, -728)
	20200213_D50_1529_Plastic_Fire : (-217, 321)
	20200213_D50_1537_Plastic_Fire : (0, 0)
	20200213_D50_1536_Plastic_Fire : (204, -89)
	20200213_D50_1543_Plastic_Fire : (-212, 527)
	20200213_D50_1548_Plastic_Fire : (-121, -125)
	20200213_D50_1549_Plastic_Fire : (207, -581)
	20200213_D50_1554_Plastic_Fire : (-164, -57)
	20200213_D50_1555_Plastic_Fire : (-209, 142)
	20200213_D50_1561_Plastic_Fire : (270, -124)
	20200213_D50_1566_Plastic_Fire : (-288, -53)
	20200213_D50_1567_Plastic_Fire : (279, 121)
	20200213_D50_1573_Plastic_Fire : (0, 0)
	20200213_D50_1579_Plastic_Fire : (-224, 1161)
	20200213_D50_1585_Plastic_Fire : (303, -613)
	20200213_D50_1596_Plastic_Fire : (278, 555)
	20200213_D50_1591_Plastic_Fire : (-330, -554)
	20200213_D50_1597_Plastic_Fire : (-294, -1592)
	20200213_D50_1603_Plastic_Fire : (159, 1141)
	20200213_D50_1611_Plastic_Fire : (-355, 520)
	20200213_D50_1612_Plastic_Fire : (-35, 689)
	20200213_D50_1616_Plastic_Fire : (276, 179)
	20200213_D50_1617_Plastic_Fire : (-176, 176)
	20200213_D50_1624_Plastic_Fire : (336, -206)
	20200213_D50_1630_Plastic_Fire : (-301, 153)
	20200213_D50_1636_Plastic_Fire : (-376, 698)
	20200213_D50_1637_Plastic_Fire : (366, -2292)
	20200213_D50_1640_Plastic_Fire : (233, 0)
	20200213_D50_1643_Plastic_Fire : (0, 390)
	20200213_D50_1489_Plastic_Fire : (-1113, -579)

[LeviFiction]

You know I've been meaning to write a program that will convert PSP History information into a re-useable script. I should do that someday. It looks pretty straight forward. Anyway, here is the history for your image.

https://drive.google.com/open?id=1G-4vy ... GMuLhr1w0f

[Jean-Luc]

The majority of the format looks complete and fine.

Just curious: what make PSP unable to recognize the file? In other words, where is the corruption?

[dangelic0]

That is fantastic. Thank you thank you thank you. I can easily now reproduce my broken file. I will also use the History extractor script. Excellent.

I can't answer Jean-Luc's question (perhaps Levi knows now), but I did notice this: the PSP error message "Unable to open file. Please verify that the file is valid." is the same one PSP throws when it can't open a file due to another process having it open. It may be a generic message for all file-open failures, but that caused me to run CHKDSK, tried opening a copy of the file, etc. So I don't know what went wrong with it.

[LeviFiction]

That I don't know. I was expecting it to be a failure to write all of the data to the file before closing. And it might be, but my reader hasn't errored out yet while reading it. It usually errors out I also don't have it programmed to interpret the actual image data yet. That's slower in coming. But as far as formatting, where blocks start, what data is available to be read from them, how long they're supposed to be, all of that is checking out just fine. I'm only getting started, but I'm also extremely slow at things like this.

According to the PSP file format, PSP has two main compression methods LZ77 and RLE. LZ77 is the same type used by PNG minus the fitlers and requires a contiguous file stream. I have a pure python library for PNGs I can read through to see how they do the compression and see if it works on PSP images. Just haven't gotten around to it yet.

[Jean-Luc]

According to the PSP file format, PSP has two main compression methods LZ77 and RLE. LZ77 is the same type used by PNG minus the fitlers and requires a contiguous file stream.

dangelic0, do you use LZ77 compression when saving your pspimage file?

[dangelic0]

Yes I use LZ77 compression most of the time, with the X8 - 2020 compatibility option set.

[Jean-Luc]

Yes I use LZ77 compression most of the time, with the X8 - 2020 compatibility option set.

I'm not sure what Levifiction means with "LZ77 ... requires a contiguous file stream" but if it is not respected during saving, perhaps it could be the origin of the corruption.

[LeviFiction]

I was beginning to think it as the layer styles block But that was a dead end. It's still kind of crazy though. PSP Blocks all have the same header. 4 Bytes for the signature, 2 Bytes for the type, and 4 Bytes for the total length of the block minus the header. I also saved my own PSPImage to test this on. Both your image and my image give a Layer Styles block length of 263. But if you actually compare the start of the block with the start of the next block there are 7 extra bytes in between. This should never happen. Which means there is something special/different about the Layer Styles Blocks. It was giving me the biggest headache trying to work out what was wrong. Especially because the Layer Styles block was added 3 or 4 versions after the File Format document that I'm using.

My document only shows 31 different block types in the PSP Image format. But Layer Styles is block 33. So not only is it not documented forcing me to reverse engineer what the values mean, but there is also another type of block that I don't even know about.

This is why analysis of corruption is so slow. I'm making up for all of the changes and improvements that have been made since PSP9 with no reference document to guide me. Just a lot of trial and error. Speaking of which, I found out that the zlib that comes with Python will do LZ77 compression. So I get to test the individual layer images tomorrow.