No observations, I am sure I saw spots!

From time to time, particularly with radiation sensitive samples, you may see:

Status: error "no observations run 1: 1 to 1795"

From xia2. Essentially this means that there are no spots with an I/sigma > 2 or so in the data set and Scala complains. However, you may know that there are strong spots there because you have seen them. So, what's going on?

If your sample is mostly dead some way through the data collection, XDS will continue to process the images. When it gets to the end and scales the values will be pretty poor. To accommodate this
the sig(I) values are inflated, which has the side-effect of reducing the I/sigma for the entire data set. This is what has happened in this example.

So what do you do? As it has not scaled / merged the data properly it is tricky to work out what images are good. Well, here follows a recipe I would suggest.

First, move to the directory where the processing was performed, which could well be DEFAULT/NATIVE/SWEEP1/integrate (say). Then use GNUPLOT to look at the raw measured intensities in INTEGRATE.HKL as follows:

> plot 'INTEGRATE.HKL' using 8:($4/$5) with dots

This will give you a scatter plot:

Which pretty clearly shows that there is something funny going on. In particular, there are very few weak data after frame 1200 ish. So, I would then edit the automatic.xinfo file which the previous xia2 run generated and limit the number of frames. In this case I ended up just using the first 900 frames, before the first gap, though it would be sensible to also consider including the frames 950 - 1250 as a second sweep.

In other cases, particularly when the input beam centre is unreliable, this phenomenon can also result from misindexing the reflections. XDS will in this case inflate the sig(I) estimates to the point where non symmetry related reflections are compatible within errors, which is obviously wrong. I still need to work on an automated tool inside xia2 to spot this!