Using PixInsight PixelMath to Correct a Flat Frame Issue

In my last post I talked about a problem I had with flat frames collected at f/2. To correct the issue I used the processes AutomaticBackgroundExtraction and GradientCorrection. This worked very well for two of my images, but I recognized it might not with the huge nebula Sh2-264, (AKA the Lambda Orionis / Angelfish Nebula).

I tried the ABE+GC method on this and it failed spectacularly by obliterating the eastern half of the nebula. So my approach will instead be to use a method that employs PixelMath and an ad hoc model of the donut hole.

Here is the hole as seen in the green master frame from one the other images collected the same night:

Green Master Frame with the donut at center in all its glory

This donut appears in the master green frame of Sh2-264, if not as prominently. So I'll use the above photo as the basis for the correction's structure. It's worth noting that the master frames have already been flatted, so either my system is dust-free, or the flatting took care of any dust donuts. What I'm about to do is apply a simple correction that's limited to the central area of the image.

At the very center the brightness doesn't seem diminished and no correction is needed. As distance from the center increases the distortion progressively darkens to form a ring, then quickly brightens again. It's this ring that must be brightened. Because I'm correcting the master linear frames I'll use simple multiplication of the existing pixel values.

Eyeballing this led me to a polynomial model of the needed correction. Here it is in an Excel graph.

Sixth-order polynomial fit to estimated donut darkness

The horizontal axis is scaled distance from the donut's center, with the value 1 corresponding to radius of the ring's darkest values at about 700 pixels from center. The vertical axis is a dimensionless value for the darkness of the donut relative the image's true background.

This is a complex shape that is fit well by a sixth-order polynomial 

So, how to make use of that polynomial? Let's look at the PixelMath script!

Script for hole correction. Click for larger version

This is a simple script. Basically it looks at the distance a pixel is from the center of the hole, calculates the factor by which to increase the pixel's brightness, and multiplies the pixel's existing brightness by that factor. Pixels outside the donut (R > 1.38)  are left unchanged. 

The constants are essentially input parameters that can be played with to refine the correction. The main adjustment is Amplitude; too large a value and the donut becomes a light ring, too small and there's not enough improvement.

The multiplier is 

        1.0 + (polynomial value times Amplitude). 

I found the best result used Amplitude = 0.04, so at most the correction is a 4% increase in linear value.

So what did this do? Here is the Sh2-264 Green Master before and after the script is applied:

Uncorrected

Corrected

Note that these are given the "Boosted" PixInsight stretch to emphasize the hole; it won't be this evident in the final image.  The script did a fair job, I think. 

 

My post-WBPP workflow for this image uses ABE to subtract first order light pollution from each master light frame; the script is then applied to each channel. After that I return to my conventional workflow: CombineChannels, BlurXTerminator to reduce chromatic aberration prior to color calibration, and so on. 

The result:

Finished Image, corrected (1/4 scale)

North is up; The Angelfish swims nicely westward! I think the best way to validate any image is to compare it with clearly superior one by an accomplished imager, so I used Adam Block's superb image for this one. I think my processing does a very decent job of reproducing his in the central area of the nebula, so I'm happy!

This completes the rehabbing of the Iowa Star Party images.  Next I'll be revisiting some of my images from 2022. Also, Masters of PixInsight are doing a workshop on mosaics in December, so maybe I'll finally process that Veil Nebula mosaic data I've been sitting on.

Iowa Star Party Images: Repaired!

To recap: I took one night's data at the October Iowa Star Party.  As was my usual (rather lazy) practice I deferred shooting flat frames until my return home. When I processed the data I got something of a shock. The stacked calibrated frames had very obvious dark "donut holes" in the middle. Here's what I mean:

Master Green after ABE

This was the result of not matching the exact focus used while gathering light frames. The remedy would be to obtain correct focus and retake the flats, so the next clear night when the temperature was close to what it had been in Iowa I set up in my back yard, got focused, shot flats, and hoped for the best. Something was off, though; the new result after recalibrating the light frames was an even larger donut hole than what you see above.

I've read that flatting at f/2 is a fiddly thing, and based on this I have to agree.

The only real solution is to collect future data and flats at a slower speed so that focus is less of an issue. And to take flats in the field at the time of data acquisition. But that's too late for the Iowa data, which I really don't want to throw away. So what can I do to salvage it?

How about creating a sort of secondary multiplicative layer of my own? One way to do this might be by using using the GAME script for PixInsight, which basically would create a round mask that I could use to stretch the hole area a bit, or even to create a synthetic secondary flat to use in recalibrating the already calibrated light frames. Nice thought, but I tried both methods and didn't get satisfactory results. 

More labor-intensive would be to create applications in a high level programming language that could directly adjust pixel values in the master frames. It's been maybe 12 years since I did any serious coding and had none of the needed compilers installed on my computer, so no -- that would be too involved. Much simpler would be to use PixInsight's PixelMath to do the same thing. I used Microsoft Excel to model the hole as a 6th-order polynomial and used a simple PixelMath script to transform the master images. This actually worked to some extent, but not well enough. I could have fiddled with that and eventually found the right polynomial, scale size, and amplitude to make a good correction, but I really didn't have the patience for that. There had to be a better way.

And in this case, there was. I wouldn't recommend this as a cure for any other flatting problem, but it seemed to work well enough for the Iowa data. The nice thing is that it used two standard PixInsight processes.

Because the data for two of the targets was low in the eastern sky over a rather conspicuous light dome, it suffered from a considerable light pollution gradient. I used PI's ABE to make a first pass at getting rid of that. This was a fairly typical application of ABE, using subtraction and a function degree of 4:

ABE settings

This essentially revealed the hole and whatever else the bad flat didn't correct. GradientCorrection was then applied with some non-default settings designed to work better on the hole's small-scale structure.  

GradientCorrection settings

Here is the result. The hole is almost completely gone, as are the edge issues left behind by ABE.

I applied this to all three channel masters and processed normally. Without the hole I could process a little more aggressively to bring out Barnard's Loop.

My image based on Iowa data

Wikipedia image (an RGBHa image by Hunter Wilson),
cropped and scaled to match my image above.

Below my image is the picture of Barnard's loop on its Wikipedia page for comparison. I've rescaled and cropped the image to match mine. Wilson's acquisition data is RGBHa, which probably accounts for the very red nebulosity.

Am I totally happy with my image? Not entirely. Theres still some weak signal suppression at the the very center of the image, and I wish the Running Man was bluer. Given that this is based on only half an hour of total exposure (ten minutes per color channel) I am pleased with is how it reveals a lot of the blue reflection nebulae in western Orion. It's a shame it didn't quite extend far enough to catch the Witch Head. It would be wonderful to devote hours to this area, but that's not going to happen.

Here's the other de-holed image from Iowa:

L to R, Soul Nebula, Heart Nebula,
and the Double Cluster (1/4 scale)

As with the Orion image, the hole is essentially eradicated. Too many stars, though. 

Reprocessing the third image, the huge Lambda Nebula, will be problematic as the hole is very entangled with the nebula. I may need  to use my PixelMath script method for that.

Next spring I'll try to get dithering working more reliably so that I can drizzle process and get better stars. With some luck I may get my long-sought wide field image of the Polaris-area IFN after all.

But now it's time for winter hibernation and reprocessing of old data. With what I'm learning from the Masters of PixInsight folks, I may even tackle that big old Veil Nebula mosaic data I've been sitting on for over a year!

Next time: the Lambda Orionis Nebula (Sh2-264) after de-holing!