xia2 and very latest XDS/XSCALE

A problem has been found with the very latest version of XSCALE. WK changed the way that the input to XSCALE is written which I have only just found out about. I guess that this will be a key feature of 0.2.6.3.

In the meantime please use an earlier than a couple of weeks ago version of XSCALE! Ill publish a "fix" as soon as possible.

Release 0.2.6.2

To a large extent this fixes the bugs found in the previous version. However there are two new "features" which address some current issues. The fixed bugs relate to the problems people have been finding with detector names. This should never have happened! The cell refinement images are now selected in a more careful way based on the crystal orientation and trying to find the best images for refining individual cell axes. This appears to work well.

There is a test performed to check that the selected lattice symmetry is good. This uses the ratio of rmsd with and without the lattice constraints. If this ratio is > 1.5 by default then that indexing solution will be rejected. If you think this is wrong, add -rejection_threshold 2.0 (say) to the command line.

I am now by default using the latest version of Mosflm. If this proves to be a problem the old version can be specified by -old_mosflm.

Release can be found at the usual place: http://www.ccp4.ac.uk/xia

Help for Data Collection

So, there are lots of fun things in xia2 for doing data reduction - wouldn't it be good if there was something to help with the data collection? Well there is something on the way now, which will use Labelit (optionally) Mosflm and BEST to suggest strategies for data collection, in advance of a full "data collection expert system" which is still some way in the future but will take ideas from how xia2 is built to deliver data collection help.

Mosflm versions

Currently testing out the official 701 release of Mosflm with xia2. If this passes the tests then it will be included as the default version in the next release of xia2 - 0.2.6.2.

Pilatus support

Is on the way... presuming that you have the images which contain the "mini CBF" headers. A version with this support should be available in the next month or so...

0.2.6.1 Buglet

If you find that perfectly correct indexing solutions are being rejected by the cell refinement test (with messages like "incorrect lattice constraints" you may have stumbled across a slight problem with the stability of the Moslfm cell refinement.

Previously I have used 1 (if tetragonal or higher, else 2) then 2, then 3 wedges for cell refinement. I found by jumping straight to 3 that the results are more reliable. However, on one test system and with one example the behaviour was completely wrong - it rejected a perfectly correct tetragonal solution with *very* high RMSD's. These are an artefact of unstable refinement.

If you remove the code:


# change 10/AUG/07 perhaps I should always be using
# three wedges?

Debug.write(
'Cell refinement: overriding the number of wedges to 3')

num_wedges = 3

from

$XIA2_ROOT/Wrapper/CCCP4/Mosflm.py

around line 768 everything should be fine. On windows this can be reached at

%XIA2_ROOT%/Wrapper/CCCP4/Mosflm.py

Any editor, even notepad, will be fine for this.

Sorry!

xia2 0.2.6.1 released

With the following fixes:

* Fixed bug where if the distance was read incorrectly from the
image header (or was wrong therein) XDS would get the wrong value even
if you had put the correct value in the .xinfo file.

* Now "unroll" the unmerged reflections from XSCALE and then merge
them in their original sweeps in Scala. This should give a useful Rmerge
vs. time plot.

* Including updated version of Pointless.

* Includes new and more robust handling of pointgroups, lattices
and unstable refinement of parameters during integration.

* Include support for a reference reflection file, which will
ensure that the reflections are indexed in the same way and with the
same spacegroup - useful for mutants.

* Now always use three wedges of images for the cell refinement
with Mosflm as this makes the lattice elimination more reliable.

* If one of the images in a sweep is broken (e.g. corrupted) xia2
will tell you more helpfully rather than just crashing.

* Now correctly read distance from Mar 165 images. Thanks to
Francois Remacle for this fix to DiffractionImage.

* Fixed the use of xia2 -3d with RAXIS IV detectors.

* Use more images for Mosflm autoindexing (three instead of two)
as this gives uniformly better results.

Duff images and odd exceptions

Previously if you have run xia2 but one of the images in a sweep was corrupted, all you got was a "child aborted" exception. Now it will tell you that there was a problem reading a certain image - very useful if you have a single broken image in a set!

xia2 and Mar CCD 165

So, it has been noticed that xia2 (and particularly diffdump) does not work quite right with some Mar 165 images - the distance is not correctly found. This has been tracked down and will be fixed in the next version of xia2 - 0.2.6.1 - which will be available shortly.

xia2, XDS and RAXIS

So, it turns out that xia2 didn't work with Raxis detectors for a number of reasons. These are now fixed -

- Raxis detectors need to have the distance negative.
- Different polarization plane normal.

Also added guess am I at a synchrotron check for the polarization fraction itself...

xia2 Environment

Since xia2 uses CCP4, the correct working environment is needed. This is usually set up by ccp4.setup, but if you are running on a cluster and you have qsubbed copying the environment across (which is a good way to run xia2) the BINSORT_SCR directory may not exist. In the next release of xia2 this is checked for and created if necessary.

New Scala & Windows

Next version of xia2 will probably have an updated version of scala included - 3.2.34. This is much improved in a number of ways which are not clear to me but I believe. However, this seems to require that on windows the CCP4_OPEN environment variable is in UPPERCASE as UNKNOWN not lower case as unknown. So if you find some strange errors with files this is probably the problem. The setup script should overwrite this though...

Mosflm autoindexing

As I have found that autoindexing with Mosflm works more reliably for three images than for two, I have made this the default for xia2. If you have more than 90 degrees of data then images at 0, 45, 90 are used, else images at the start, end and in the middle.

Software Versions

Just updating to 7.0.1 Mosflm (30th July 2007) and pointless 1.2.0. This should make integration more reliable with the 2D pipeline.

Software verrsions

Reference reflection files

This is now implemented. If you include a REFERENCE_REFLECTION_FILE in a CRYSTAL block it will be used for determining the setting and assigning the spacegroup. This should contain a single data set with intensity values. Currently the free set is not copied from this - I will write a small "jiffy" application to copy the FreeR_flag column from your existing data to the newly reduced data, as it is unusual to include the Free column in intensity data.

0.2.6.1 is currently undergoing testing.