Tuesday, 15 March 2011

Much belated xia2 release

In the usual place, changes since
  • Support for new XDS build.
  • Small bug fix / improvement in XDS indexer implementation.
  • Capability to ptovide refined experimental geometry via XDS GXPARM.XDS - useful for polycrystal data reduction.
  • Now check what you tell the program, viz:
    gw56@ws050 nonsense]$ xia2 -3d -nonsense /data/gw56/dl/Cowan/Insulin/insulin/
    Traceback (most recent call last):
      File "/home/gw56/svn/xia2/Applications/xia2.py", line 45, in 
        from xia2setup import write_xinfo
      File "/home/gw56/svn/xia2/Applications/xia2setup.py", line 35, in 
        from Handlers.CommandLine import CommandLine
      File "/home/gw56/svn/xia2/Handlers/CommandLine.py", line 1513, in 
      File "/home/gw56/svn/xia2/Handlers/CommandLine.py", line 349, in setup
        raise RuntimeError, nonsense
    RuntimeError: Unknown command-line options: -nonsense
    and complain if you type in something xia2 does not understand.
  • Removed a lot of cruft from the xia2 code base in preparation for some more substantial refactoring.
  • Found bug which meant that the labelit beam centre computed in the setup phase was not used - mostly this is not important, unless you are using -3dii.
  • Added -ice command-line argument, which will exclude measurements from regions which are typically where ice rings land. Will need to add more subtle mechanism which will allow specific regions to be excluded.
  • Specifying resolution with 3d(r) pipeline now works correctly.
  • Removed XDS version check by default, which was annoying every new year. If you would like to check that the XDS version is explicitly supported, add -check_xds_version to the command-line.
  • Added table of scala runs to sweep name in the debug output, helpful for reviewing in polycrystal data reduction cases.
  • Mosaic spread used from Mosflm is now average of all images, not the first or last image.
  • Fixed header reading for new Pilatus instruments at DLS and elsewhere.
  • Added capacity to pass in known globally postrefined experimental setup via -xparm GXPARM.XDS, only useful with XDS processing pipelines.
  • Removed use of ccp4 printpeaks tool, which crashes on new pilatus images, replace with Mosflm for the moment. Move to replacement with labelit code at later stage planned.

Monday, 14 March 2011

Underscore weirdness

A weird error I had reported to me last week turned out to be due to the use of underscores_in_sweep_names and (perhaps) to numbering crystals. This is most likely a side-effect of the heuristics I use to understand what is in MTZ files and such. Some recommendations:

* Don't use underscores in your project, crystal, wavelength, sweep names
* Don't use numbers for your crystal names

The second one is a problem as the CCP4 parser can get confused between datasets which are numbered and the number name of data sets. I would prefix it with P/X/W/S if you want to number things...

Friday, 4 March 2011

No observations, I am sure I saw spots!

From time to time, particularly with radiation sensitive samples, you may see:

Status: error "no observations run 1: 1 to 1795"

From xia2. Essentially this means that there are no spots with an I/sigma > 2 or so in the data set and Scala complains. However, you may know that there are strong spots there because you have seen them. So, what's going on?

If your sample is mostly dead some way through the data collection, XDS will continue to process the images. When it gets to the end and scales the values will be pretty poor. To accommodate this
the sig(I) values are inflated, which has the side-effect of reducing the I/sigma for the entire data set. This is what has happened in this example.

So what do you do? As it has not scaled / merged the data properly it is tricky to work out what images are good. Well, here follows a recipe I would suggest.

First, move to the directory where the processing was performed, which could well be DEFAULT/NATIVE/SWEEP1/integrate (say). Then use GNUPLOT to look at the raw measured intensities in INTEGRATE.HKL as follows:

> plot 'INTEGRATE.HKL' using 8:($4/$5) with dots

This will give you a scatter plot:

Which pretty clearly shows that there is something funny going on. In particular, there are very few weak data after frame 1200 ish. So, I would then edit the automatic.xinfo file which the previous xia2 run generated and limit the number of frames. In this case I ended up just using the first 900 frames, before the first gap, though it would be sensible to also consider including the frames 950 - 1250 as a second sweep.

In other cases, particularly when the input beam centre is unreliable, this phenomenon can also result from misindexing the reflections. XDS will in this case inflate the sig(I) estimates to the point where non symmetry related reflections are compatible within errors, which is obviously wrong. I still need to work on an automated tool inside xia2 to spot this!