Judging a Fit

A fit ends with a χ² and a structure. This chapter decides whether you can report them. The fitting procedure judges a fit by several checks on the curve and by the spread of the result over repeated runs. It judges the fits that Chapter 17: Periodic or Polynomial? ended with: CoC4 in Polynomial mode of order 1, CoC5 with one period and the period free, and the Chapter 3 demo at order 1 and order 3. Each was run five times (Chapter 16: Several runs, and their spread).

χ² is one check among several

The author judged fits by χ². The author’s words when the procedure was written down were “The KPI is ChiSQR.” The procedure keeps χ² and adds checks beside it: “Chi2 alone is not the judgment. It is one check among the following.” (§10.1)

The reason is in the demo. Its order-1 fit gave the same χ² of 6.40 in all five runs. Its drift was still wrong by 1.6 Å (Chapter 17: Raising the order). On CoC4 the order-1 fit has χ² = 0.959, well under the laboratory’s limit. Two of its Bragg orders still come out at a quarter of their measured height. χ² sums the misfit over thousands of points. A feature that holds a few of them can be wrong without moving the sum much.

The author also gave a rule for a very large χ²: “small ChiSQR is the indicator itself, ideally below 10. If it’s above 100, something is wrong, usually the low angles are not cut properly.” The laboratory’s preset now sets the limit at 1.3, for its own weighting and range. It keeps the second part of the rule as an alarm. Above 100, check the start of the fitting range and the scale before you change anything in the model (§10.4; Chapter 14: Where the fitting range starts and ends).

Table 18-1 lists the checks and one laboratory’s limits for them.

Table 18-1: The checks of a fit and one laboratory’s limits, from the preset of the fitting procedure. The limits were set on two Co/C curves, CoC4 and CoC5, fitted over θ = 0.22–3.2° with four orders present, at Δθ = 0.012°, with the θ weighting off and the point weighting on, and with the scale solved in the fit.

The fits of this book are all outside it in some way. CoC4 is fitted at Δθ = 0.014° over eight orders, out to 2θ = 13.4°. CoC5 is fitted out to 2θ = 13.2°. The demo is a different material, calculated without noise. χ² values also don’t compare between conditionings. In the paper on the procedure, the expert’s fit and a test session’s fit of the same curve changed places when each was scored under the other’s range, scale and Δθ.

Reading the checks in X-Ray Calc

Result > Fit report … works out every check of this chapter for one model:

1. Click the fitted model in the project tree, so that the triangle moves to it.
2. Choose Calc > Run (F5). The report needs the model’s calculated curve.
3. Choose Result > Fit report ….

The Fit report window has five groups (Figure 18-1): Bragg orders, Critical edge, Fringes between orders 1 and 2, Residual by band and Parameters within 5 % of a bound. The first line of the header names the model and its data. The second gives the scale. The report multiplies the measured curve by the scale that the last calculation solved, as χ² and the chart do (Chapter 14: Letting the fit solve the scale). So its ratios match what you see on the chart. The angles are in 2θ when the chart shows 2θ.

The window shows numbers only. It marks no pass or fail, because the limits belong to your laboratory. Copy as text copies every table, with tabs between the columns, so you can keep it with the fit. The report needs a model with linked data, calculated by angle.

The order table

The order table writes down what the author checks first on the chart: whether the Bragg peaks are where they should be, and as high. For every visible order, it gives the measured peak, the calculated peak and their ratio, calculated divided by measured. An order is visible when its measured maximum stands more than 3 times above the background (Chapter 11: Numbering the orders). The peak is the highest point in a window around the order’s predicted position, on each curve separately.

The limit is 0.75–1.25. The first version of the procedure accepted 0.5–2, and the second tightened it. The expert’s own fits of the qualification curves lay at 0.69–1.76, so the author was asked whether the band should be widened. The author kept it: “I expected that after all you can do it better than me, so I would keep 25%”.

Table 18-2: The order table of CoC4, Polynomial mode of order 1, the run with the lowest χ² (0.959). Peaks in counts: the value on the chart divided by one count. The calculated peak is at the chart’s scale. Fitted by X-Ray Calc 3.9.4.1250.

Orders 1 to 6 pass. Order 5 passes near the lower edge of the band, at 0.77. Orders 7 and 8 fail, in this run and in every other run and mode of Chapter 17, at 0.26–0.42. Figure 18-2 shows the ratios of all five runs, next to CoC5 and the demo.

CoC5 fails order 4 in every run, at 0.46–0.59. The demo shows the other side. Order 1 fails order 5 at 0.50. Order 3 passes every order in its best run, at 0.98–1.04. But one order-3 run, with χ² = 0.389, puts order 5 at 1.30. So a verdict belongs to one run. The mode or the order alone doesn’t decide it.

Look at the orders that fail. Figure 18-3 shows them in counts. Order 7 of CoC4 is one point of 5 counts, and order 8 one point of 4. The tail has many points of 2–4 counts. These two count as orders only because they sit where the line through the strong orders puts them (Chapter 11: Numbering the orders). The model has almost no peak there, 1.3 and 1.1 counts. Order 4 of CoC5 is a real peak. The model follows its shape, and most of its points lie at 8–13 counts. The measured maximum is one point of 16 counts. The ratio compares the model’s peak with that one point.

Both kinds of fail count. The limits were set on curves with four orders present. Orders 7 and 8 of CoC4 lie beyond them, so the verdict says that the 25 % band is applied outside its calibration. CoC5 has four orders. The procedure’s own qualification sessions failed that same order in two of three rounds. When an order fails says what to do next.

The edge

The edge check compares the curve on the critical edge, the steep fall after the plateau. It takes three angles between θ_max and the first minimum of the calculated curve, at a quarter, a half and three quarters of the way. At each, the calculated value divided by the measured one must lie within a factor of 1.5 (§10.3–10.4).

The edge is where the scale was set (Chapter 14: Setting the scale). A fail means the scale no longer fits the model. The model has moved away from the one the scale was set with. Then normalize again with the fitted structure as the model, and fit again. The author’s words: “sometimes you need to renormalize again if the model moves far away” (§4.2).

On CoC4 the order-1 fit gives 1.06, 1.01 and 1.00, at 2θ = 0.543°, 0.669° and 0.798°. On CoC5 the fit with one period gives 1.15, 0.95 and 0.96, at 2θ = 0.507°, 0.606° and 0.708°. Both pass. The plateau of a model can have a very shallow minimum of its own, where the edges of two materials meet. The report takes the first minimum only after the curve has risen 5 % above it again, so such a ripple doesn’t end the edge. Look at the three angles in Critical edge before you read the ratios. They must lie on the fall.

The fringes

The fringes between orders 1 and 2 show the blur of the resolution most clearly. Their contrast is each maximum divided by the minimum next to it, averaged over the band (Chapter 16: The fringe contrast as a check). On CoC4 the measured mean contrast is 2.17 and the order-1 fit gives 1.75. On CoC5 they are 2.29 and 1.76. In both, the model’s fringes are shallower.

The fringe check is reported. It has no limit. The procedure’s pass criteria are χ², the orders and the edge (§10.4). When the fringes and the peaks disagree, the peaks win (§10.13). The Bragg peaks carry the period structure. Fringes that don’t match are the first place to suspect a missing surface layer. The author ruled this way on a CoC4 fit whose peaks passed and whose fringes didn’t: “In this case I select peaks. Non matching fringes can mean something on the surface (light hydrocarbons) we omit in our model.”

The residual by band

The last curve check divides the fitting range into eight equal bands of θ. In each band it takes the mean of log₁₀(calculated / measured) over every point. A mean of 0.1 is a factor of 1.26: the model sits 26 % above the data over the whole band, or the same amount below it at −0.1. That is a warning (§10.4). Look at the ends of the stack first, the contamination layer and the substrate (§10.4; Chapter 13: Two rules for reading a curve).

A band catches a misfit that no single peak shows. Figure 18-4 compares the two demo fits. Order 1 gives three warnings: +0.10, −0.13 and +0.12, in bands 3, 4 and 6. A straight-line drift lets the model drift away from the data between the orders, first on one side and then on the other. Order 3 keeps every band within 0.01.

On CoC4 the order-1 fit stays within 0.043, and on CoC5 the fit with one period stays within 0.045. So neither warns. The last three bands of CoC4 sit at +0.026 to +0.043: the model lies above the data in the tail. That is the floor (Chapter 14: The floor). R min was set at one count before the fit. The chart draws the model divided by the solved scale, 0.877, so the floor stands at 1.14 counts, above the one-count points of the tail (Figure 18-3). The effect is small, well under the warning.

Residual by band in the Fit report lists the eight bands with their mean and rms. The rms is taken around zero. It includes the fringes and the orders, so read the mean.

The chart shows the same residual point by point, in a strip under the curves. Result > Residual strip > Show turns it on or off. It is on by default. The strip draws log₁₀(calculated / measured) at every measured point. A line above zero is a model above the data. A line below zero is a model below the data. The numbers are the Fit report’s. The strip uses the same eight bands and the measured curve at the scale solved in the last calculation. Each visible model gets its own line, in its own color, so two fits of one curve can be compared on one strip.

Three more items of the same menu are also on by default:

• ±0.1 lines mark the warning level.
• Band means draws the mean of each band as a step over the band. These are the means of Residual by band.
• Floored points draws the points on the floor as gray dots. A point is on the floor when the model is within 1 % of R min, or when the measured value, at the solved scale, is at or below R min. These points are not a misfit. But the band means still count them, as the report does. So a band mean that rests on gray dots shows the floor.

The vertical range of the strip is fixed at −1 to +1. The strip needs linked data and a calculation by angle. It is part of the chart, so a zoom or a pan moves it with the curves, and a copy or an exported image includes it.

Near a limit

A parameter that ends on a limit or within 5 % of the range from it is reported with the limit. The limit is extended and the fit repeated before any value is read (Chapter 15: The near-bound rule). The Fitting Limits dialog marks such cells in yellow. Chapter 17 widened the limits of CoC4 and CoC5 this way before the fits of this chapter. Two kinds of limit stay: a σ at 0, and a density at the ceiling of the bulk value. The contamination layer of CoC4 ends with σ at 0 in the best run.

What the runs don’t agree on

Chapter 16: Several runs, and their spread reports each parameter with its standard deviation over the runs. The procedure turns that spread into a rule. When the spread of a parameter is larger than the precision you would claim for it, report the parameter as undetermined, without a number (§10.7). The runs disagree by more than the number would claim. One laboratory’s precision limits: 0.5 Å for a thickness and for σ, 0.2 g/cm³ for a density, and 0.1 Å for the period.

For a Polynomial fit, test the value in each period. Leave the coefficients out of the test. The specimen has thicknesses. It has no coefficients. The Thickness page draws the per-period values of each run (Chapter 17: Reading a drift in X-Ray Calc). The ends of the stack, the top and the bottom period, spread the most, so test them.

Table 18-3: The run with the lowest χ² and the standard deviation over five runs. Undetermined: the standard deviation is larger than one laboratory’s limit (0.5 Å for H and σ, 0.2 g/cm³ for ρ, 0.1 Å for D). CoC4: Polynomial mode of order 1, with H of C and Co at the top and the bottom period. CoC5: one period, with the period free. Fitted by X-Ray Calc 3.9.4.1250.

On both curves the thicknesses and the period are determined, and the σ values are not. The per-period thicknesses of CoC4 pass with little room, at 0.32–0.44 Å. Report the drift of the period and leave out its split between C and Co, as Chapter 17: Decide on the drift says.

The demo shows both ways a spread can mislead. Its order-1 fit agreed with itself to 0.007 Å on the drift in all five runs. The drift was wrong. Its order-3 fit found the true profile in its best run. But the five runs spread by 0.35 Å at the top period and 0.63 Å at the bottom one, so the ends of that stack are undetermined. A small spread doesn’t prove a value right. A large one doesn’t make the best run wrong. The spread shows what the curve doesn’t fix. The rule reports exactly that (§10.6).

Two layers as a sum

Sometimes the runs agree on the total of two neighboring layers and disagree on how it divides. Then report the two as a sum, and the split as undetermined (§10.8). The procedure’s example: “Two independent fits of one curve agreed on the total metal in a period to within a few tenths of an angstrom while placing the thick interlayer on opposite sides of the metal layer.”

Use the rule when at least one of the two layers is undetermined by itself and the spread of their sum is smaller than the spread of either. The layers of a two-layer period always add up to the period, so the sum is always better held than its parts. In a Co/C stack the split is still determined. In one laboratory’s tests on calculated Co/C curves, the Co thickness came out within 0.05 Å rms.

A W/B4C stack with WC interlayers is different. On calculated curves, the W thickness came out +0.88 ± 0.62 Å off, while the W and WC per period together were off by only −0.16 ± 0.29 Å. Two fits of one measured W/B4C curve agreed on the period to 0.01 Å and on the metal per period to 0.25 Å. Their W thicknesses differed by 1.7 Å. Report such a stack as the metal per period, W plus WC, with the split undetermined.

Which interface is rougher?

The two σ values of a two-material period move the same features of the curve. The fit finds two minima, one with the larger σ in each layer (Chapter 9: Figure 9-2). The procedure: “Which of them a fit lands in is decided by the random seed” (§10.14). The starting model plays no part when the swarm starts spread over the limits.

The paper on the procedure records this on CoC4. A test fit gave σ = 2.64 Å for C and 4.33 Å for Co. The expert’s fit gave the pair the other way round, 5.58 and 2.46 Å. Both had the same peaks and the same period. They differed only in the fringes between the orders. A contamination layer on top lowered χ² from 7.94 to 6.09 and still didn’t settle which layer is rougher. The author passed the fit.

CoC5 does the same in this chapter. Of the five fits with one period, three gave C the larger σ, 5.53 Å against 4.50 Å for Co, at χ² = 0.6825. Two gave Co the larger one, 6.04–6.06 Å against 4.45–4.46 Å for C, at χ² = 0.705. That is why σ is undetermined in Table 18-3.

So the assignment is not a check. Judge the fit by the peaks, the edge and χ². Report the σ values as fitted. Don’t claim which interface is the rougher one.

Asymmetric interlayers: fit each assignment

An interlayer is a thin layer of mixed material at an interface (Chapter 7: Interlayers). The two interfaces of a period often grow differently. A heavy metal deposited onto a light amorphous layer mixes differently from a light material deposited onto a dense or crystalline metal. But a metal layer thin enough to stay amorphous can make the two interfaces alike (§10.9). So the side of the wider interlayer is not a property of the material pair. It is a question for each fit.

Without other evidence, fit every admissible assignment (§10.10):

1. The wider interlayer on the interface where the metal is deposited onto the light layer.
2. The wider interlayer on the other interface.
3. The two interlayers equal.

Run each of them five times. Report all three, each with its χ² and its spread. The split stays undetermined unless a check of this chapter rules out one of the assignments. A χ² difference smaller than the spread over the runs doesn’t choose between them.

In X-Ray Calc, set each assignment through the limits. Give the interlayer that must be wider a Hmin above the Hmax of the other one. X-Ray Calc can’t hold two layers equal during a fit. For equal interlayers, give both the same H and freeze it. Fit, then repeat with a few other values of H, and keep the one with the lowest χ² (Chapter 19: Testing the model on CoC4 and CoC5).

Evidence from another measurement can decide the assignment instead: transmission electron microscopy or photoelectron spectroscopy. Then fit only that assignment, and say in the report where it came from. Check it against the fit afterward. A declaration that holds only while the metal layer is crystalline doesn’t hold for a fit that returns that layer too thin to be crystalline (§10.11). An assumption can decide the assignment too. The report must then say it is an assumption.

When an order fails

When an order fails, the procedure first applies the two rules of Chapter 13 (§10.12). The level between the peaks is set by the ends of the stack: the contamination layer and the substrate. The fine structure of the high orders is set by the layer ratio, the interfaces and the drift. Change those before anything else. When the first fit passes every check, don’t fit again, except for the runs and the limits that the earlier chapters require.

This decides the question that Chapter 17 left open. A higher order of the Polynomial profile is kept only if it passes every check of this chapter (§9.8). On CoC4, order 3 fails orders 7 and 8 in every run, like order 1. It also fails order 5 in three runs. On CoC5 it fails order 4, like the fit with one period. So order 3 is not kept on either curve. The demo’s order 3 passes every check in its best run and is kept.

CoC4 and CoC5 still fail their weak orders in every mode. A fit that doesn’t reach the data after these steps goes to Chapter 19: When the Fit Will Not Converge. That chapter takes up the two causes, a bad measurement and a model that is too simple.

Writing the verdict

Write the verdict one check at a time, with the calibration of the limits (§10.5). Table 18-4 does it for the three fits of this chapter.

Table 18-4: The verdict, check by check, for the best run of each fit, with one laboratory’s limits of Table 18-1. Fitted by X-Ray Calc 3.9.4.1250.

A fail is a result. Report it with the fit. Don’t widen a limit of this chapter until the fit passes.

What this means when you fit

Read χ² last. Above 100, check the range and the scale first. Below the limit, it still doesn’t pass the fit.

Build the order table for every visible order. Look at a failing order in counts before you act on it.

Check that the edge angles lie on the edge, and renormalize with the fitted model when the edge fails.

Report the fringe contrast, and let the peaks decide when the two disagree.

Read the eight bands. A band beyond 0.1 points to the ends of the stack.

Report a parameter as undetermined when its spread over the runs is larger than your precision limit. For a profile, test the end periods.

Report a sum when a layer is undetermined and the sum is held better. Don’t claim which interface is rougher. Fit every assignment of an interlayer that nothing else decides.

Write the verdict check by check, with the calibration of each limit. A fit that fails goes to Chapter 19: Two causes, and only two. Chapter 23: A complete report for CoC4 puts the verdict into the report. A fitted density is not judged here. Chapter 20: The runs agree, the models don’t explains why.