The Micro Four Thirds Lens Correction Project

[andrewj]

Hi All,

At the moment Bibble and AfterShotPro can't apply the same corrections to Micro Four Thirds (MFT) shots that are applied in camera, with the result that most require significant effort to correct geometic errors. The correction parameter data is stored with the RAW file, but although there's no obvious reason why not, Panasonic and Olympus have not published the specification for this data.

I have done some work to build parsing of this data into CAQuest (initially for Bibble, although I will forward fit to ASP shortly). However although getting the raw data is fairly straightforward I have discovered that the algorithm is much more complex than we thought, and seems to vary from lens to lens.

I am therefore hosting a "crowd-sourcing" exercise to try and get several eyes on the problem. As we uncover algorithms which work well for one lens or another I'll publish them, and also build them into CAQuest. Over time we may come to completely understand the complete MFT algorithm, and our work will then be done. Of course, if one of the MFT partners wants to help by publishing the algorithm, that would also be perfectly acceptable .

The project pages are at the following URL:

http://www.andrewj.com/mft

I'd like to use this thread to host a discussion where we can publish interim findings, thoughts and problems. I'd be grateful if we can keep it on topic, and move any off-topic discussions elsewhere.

Please visit the project page, and let me know how you get on!

Andrew

[klausw]

Thanks for posting and collecting this!

The correction parameter data is stored with the RAW file, but although there's no obvious reason why not, Panasonic and Olympus have not published the specification for this data.

I suspect that one reason for not publishing it might be that it's not some neat formula, but instead may involve a collection of hacks and special cases for specific lenses.

If I find time I'll try to hack on it a bit. To start off, here's a raw dump of the 16 values for my lenses at various focal lengths, matching the uploaded sample images I had contributed earlier. This is all on a GH2. Note the all-zero dumps for the Panasonic 100-300mm and the Olympus 45mm f1.8, is there a separate block of data we should be looking at?

Code: Select all

## FocalL     v0     v1     v2     v3     v4     v5     v6     v7     v8     v9    v10    v11    v12    v13    v14    v15
# Oly-45mm-f1.8
045.0 mm  -15035      0      0      0      0      0      0      0      0      0    500      0   2860      0 -11852  21823 
# Pana-100-300mm
100.0 mm  -15035      0      0      0      0      0      0      0      0      0    500      0   2860      0 -11852  21823 
150.0 mm  -15035      0      0      0      0      0      0      0      0      0    500      0   2860      0 -11852  21823 
197.0 mm  -15035      0      0      0      0      0      0      0      0      0    500      0   2860      0 -11852  21823 
246.0 mm  -15035      0      0      0      0      0      0      0      0      0    500      0   2860      0 -11852  21823 
300.0 mm  -15035      0      0      0      0      0      0      0      0      0    500      0   2860      0 -11852  21823 
# Pana-12-35mm
012.0 mm    -308 -10825    638      0   -348      0    192      1   4284    137    500   -815   2860    637  10761  13755 
014.0 mm  -16447   5379    529      0   -621      0    484      1   3657      5    500   -451   2860    505 -10553 -22546 
018.0 mm  -21227 -16441     34      0   -480      0     16      1   1640    557    500    -68   2860   1057 -15554  10073 
025.0 mm   32048 -26328    175      0    -83      0    544      1    144    651    500     -3   2860   1151  -7665  -4658 
035.0 mm  -15035      0      0      0      0      0      0      0      0      0    500      0   2860      0 -11852  21823 
# Pana-14-140mm
014.0 mm   -3868  -3982    421      0    790      0    395      1   3031    505    500   -559   2860   1005  24650   3828 
018.0 mm    6811   2466    611      0   -109      0    510      1   2650      0    500      0   2860    500   7046 -12178 
025.0 mm   32558  29863    104      0   -244      0    330      1   1160    169    500    118   2860    669  20452 -13595 
032.0 mm   13612   1161     36      0    -85      0    542      1    567     99    500     69   2860    599 -27245  30646 
048.0 mm  -15673   -486    325      0     17      0     22      1    267     54    500     38   2860    554   9811   3394 
070.0 mm   20487   7863    211      0     49      0     65      1    173     34    500     24   2860    534  23275  17633 
099.0 mm    6193 -24038    203      0     66      0     87      1    167     28    500     20   2860    528  14350  12148 
140.0 mm   13004   2208    178      0     75      0     99      1    146     25    500     18   2860    525  17421 -14257 
# Pana-20mm-f1.7
020.0 mm  -12288 -21098    278      0    512      0     25      1   1840    136    500   -360   2860    636  -5487 -25396 

[andrewj]

Hi,
I think the all zero dumps mean "no geometric correction required", i.e. the lens is good enough without in camera correction (the makers don't seem to bother with anything less than about a 1% error). It's noticeable (and expected) for the primes, but for my 45-175 there are correction values at the wide end (where correction is required), which go to zeros at the long end (where the uncorrected raw looks pretty good).

Andrew

[greerd]

I don't know if this will help or not but I queried Panasonic Canada about the lack of lens data available for their m4/3' s lenses. This is their response.

"Thank you for your email to Panasonic Canada.

Could you please confirm which editor you are using? So we can confirm if
they have our data.

Typically the factory does provide camera RAW data to 3rd party software, but
it is up to that company to make their own plugin and distribute it. For
companies like Adobe, that could mean a delay between the time a camera is
released to the market and when the RAW plugin becomes available.

If you require any additional assistance, please feel free to email us back.

Regards,

E.C.
Customer Care
Panasonic Canada Inc."

I never responded, but will asap.

Regards,
Doug

[andrewj]

Dear Doug,
Thanks. You could try putting them in contact with Corel, but as it's not clear whether ASP is being developed this might not work.
Alternatively I would be happy to take on the role of "plugin developer" in this context (as I have already started), so you could direct them to myself if that would make sense.
Thanks
Andrew

[greerd]

Dear Doug,
Thanks. You could try putting them in contact with Corel, but as it's not clear whether ASP is being developed this might not work.
Alternatively I would be happy to take on the role of "plugin developer" in this context (as I have already started), so you could direct them to myself if that would make sense.
Thanks
Andrew

Yes Andrew,
Makes sense to me. If I get anything worthwhile back from them, I'll direct them to you.

Regards,
Doug

[klausw]

I haven't had a chance yet to try the plugin since I only have ASP and not Bibble installed on my laptop, but I've experimented with a Javascript file that converts the dumped parameters to a/b/c values as used in the calibration DB. I couldn't get any of the algorithms to match the values for my 12-35mm f/2.8 lens though.

Here's the script: http://pocketworkstation.org/m43/conv-m43.html . The Javascript embedded in that page is self-contained and doesn't need a server, you can download the file, edit the script, and load it in a local browser if you want to experiment with different algorithms.

See also this thread where I posted new values for the 12-35mm lens from Hugin's auto-optimizer: http://forum.corel.com/EN/viewtopic.php ... 57#p255257

[andrewj]

Thanks. The 12-35 manual values will be useful.
I'm working on the ASP version of the plugin, but it means forward-fitting quite a lot of new code, so I can't promise this will be available very quickly.
Andrew

[klausw]

One question - it looks like there are several variants of the lens distortion formula which aren't quite equivalent, and just modifying a,b,c within one of them may not work right if the camera uses a different variant.

For example, compare these two:

Code: Select all

  r' = a * r^4 + b * r^3 + c * r^2 + d * r
  r' = a * r^4 + b * r^3 + c * r^2 + (1 - a - b - c) * r

The first one has the linear "d" parameter as an independent parameter, the second one derives it from a,b,c automatically. The Hugin docs at http://wiki.panotools.org/Lens_correction_model describe "d" as an overall size correction, but that's simplifying things - it does ensure that r=1 is mapped to r'=1 when using d = (1 - a - b - c), but since the formula is nonlinear its effect on other r values is more complicated.

When using the 12-35mm lens at 12mm, I've noticed that the camera JPGs seem to be more strongly cropped than the results I get with the hand-tuned or Hugin optimized a,b,c parameters, so I suspect that it's using a different formula. It could be consistent with treating "d" as an independent parameter and not calculating it from a,b,c, or it could be something more complicated.

[andrewj]

Good find. I hadn't seen this version.

I've just posted a blog post on my thoughts on the formula and the "shape" of the data: http://www.andrewj.com/blog/mft

Andrew

[klausw]

Some random thoughts inspired by the blog post, this reminds me of linear algebra lessons.

In theory the parameters from the raw file can be considered as inputs for an arbitrary function F(r, v0, v1, ..., v15), where plugging in fixed values for v0, ..., v15 reduces it to a function F'(r) that defines the radial transformation. This assumes that it's purely radial, but I haven't seen evidence that there are x/y dependent transformations going on. If we don't know anything about function F, it'll be tricky to deduce it just from the behavior.

Things would be simpler if it's a function that depends linearly on each of the parameters v0, ..., v15, where each is a weight giving a multiplier for a set of basis functions f0(r), ..., f15(r), so that the result is r' = v0 * f0(r) + v1 * f1(r) + ... + v15 * f15(r). In linear algebra terms, this is a 16-dimensional vector space where the basis is a set of functions. Some functions can just be constantly zero, for example f15(r) if that's confirmed to just be part of the checksum and not contributing to corrections.

So far we've been looking at the special case where we assume that the basis functions are simple powers of r, for example for "Model 2" (ignoring scaling factors):

Code: Select all

   f8(r) = r^4 
   f2(r) = r^3
   f11(r) = r^2
   others are constant = 0.

It's possible that the basis functions are something more complicated than just a simple power of r, which would explain why in some cases a sum of parameters results in a reasonable fit for the polynomial model. For example, Chebyshev polynomials are popular for numerical approximation of smooth functions, see http://en.wikipedia.org/wiki/Chebyshev_ ... s#Examples :

Code: Select all

  T0(r) = 1
  T1(r) = r
  T2(r) = 2*r^2 - 1
  T3(r) = 4*r^3 - 3*r
  T4(r) = 8*r^4 - 8*r^2 + 1
  ...

This would be equivalent to a plain polynomial, but the multipliers for the T functions would contribute in various weight combinations to the a,b,c terms, for example the factor for T3(4) would contribute to both a and c with weights 4 and -3.

Unfortunately there's an infinite number of possibilities including Fourier series, even when assuming linear combination of basis functions. But hopefully they went with somehing well-known.

[andrewj]

Thanks Klaus for the update.

I also got the following via email from Raphael:

About the formula used, I think it is the standard polynomial (at least a basic variation) that is used, because based on what I saw while reverse engineering, I'm 100% sure SilkyPix validates the paramaters this way (as I said on my page) :

double table[100]; for(i=0; i<100; i++) table = f(i/25.0);

f being Ru = scale*(Rd + a*Rd^3 + b*Rd^5 + c*Rd^7)

If the parameters are invalid (i.e. Ru > 2 or if the derivative is negative, which would make no sense for distortion correction) then it considers the data as in valid.

I agree that it looks like for at least a significant subset of cases, the formula is a straight polynomial. However, I'm still puzzled by the "shape" of the data if that's the case. We have too much data for "scale + 3 values" or even "scale + 3 values + 2 CA values", and not enough for "three sets of four values" although I need to have a look at the data in a second tag which is supposed to hold the CA values.

I'll look a bit further and report back.
Andrew

[andrewj]

Hi All,
Here is the first release of the ASP plugin. (An updated version of the Bibble plugin will be on my website - I can't upload it here.) At the moment these will help explore the data, but you'll need to do some manual work behind the scenes if you want to combine the values to get the geometric and CA correction values.

Please note that you must have exiftool installed on your path, and you must update the extended exif info database for this to work. If you get a red "image not found" warning at the top of the plugin then the MFT functions won't work.

I'm working on a modified approach where the formulae can be defined in an XML file and updated independently of the plugin. Watch this space...

Let me know if you have any problems with these.

Andrew

[andrewj]

Dear All,
I've updated the plugins so that they use an XML file CAQuestMFT.xml (in the top Bibble/ASP user directory) to define the algorithms. This will make it easier for everyone to experiment with different algorithms and see how they work. Hopefully the XML model is self-explanatory, but if any further detail is required please post here and I'll provide a bit more explanation.

The ASP package is below. The Bibble and ASP packages will both be updated on the B5 forum and my website.

Let me know how you're getting on.

Andrew