Wednesday, 12 December 2007

Ongoing developments

Right at the moment I am looking at radiation damage analysis, with the aim of being able to suggest decent subsets of the data to use to get optimal structure solution (don't worry - this won't happen for a while!)

Also looking at improving the way that the cell refinement tests in Mosflm are performed, figuring out how to make this parallel and also improving the integration of XDS into xia2 (no pun intended) to ensure that the correct indexing solution is used at all times...

Finally still chipping away at some "known bugs" for instance the old favourite, the BGSIG error.

Listening to: Tangerine Dream

Monday, 19 November 2007


I am reaching the stage in xia2 where it would be really useful to be able to use crystallographic code from CCTBX. This will however incur another dependency. How do you feel about this?

Please cast your vote on the right!

Friday, 9 November 2007

XIA2 released

Many fixes made:

Changes since

Now works for images with long image numbers, as typically recorded on Rigaku X-Ray sets.

Signed off for operation with Rigaku Saturn and RAXIS IV detectors.

Now works with no input, e.g.
xia2 -project TG6623 -crystal X77788 -atom se /my/images/are/here

However this relies on your image headers being accurate and the images having some kind of recognisable format...

Repaired operation on ppc and intel macs - added required libraries to the installation and reset the library paths appropriately.

Slightly improved error reporting from xinfo file errors.

A great many other small changes so this is a worthwhile update!

Friday, 26 October 2007

Slightly delayed release

During testing of the next forthcoming release I have found a few cases where XIA2 works rather badly for nasty cases - e.g. gets stuck in a loop. The release will come once I have fixed these, hopefully next week...

Tuesday, 16 October 2007

User interface

*finally* I have gotten around (with a great deal of help from Pete briggs) to putting together a CCP4i interface to xia2. This means no more messing around on the command line - you can just type everything into the GUI.

This will be included in CCP4 6.1 but may also be available before then...

Monday, 15 October 2007

xia2 for the *truly* lazy / busy

So, quite a few users have complained about the pain and suffering required to write the input file. Well, I have done away with it now then. You can still keep it - and I would advise this - but if you don't want to bother then you can try:

xia2 /path/to/images

and it will do it's best. If you are feeling slightly charitable you can give some clues:

xia2 -project SG901 -crystal X66252 -atom Se /path/to/images

If you have MAD data, it will pick this up. If you have multiple passes, it will pick this up. If you have SAD data you *have* to give it an atom else I don't know. You are allowed to lie.

If xia2 screws up just edit automatic.xinfo and you should be fine.

Usual commandline options like -debug -2d -3d -parallel etc still work.

Wednesday, 3 October 2007

XDS versions

The latest version of XDS (well, XSCALE really) seems to demand a slightlyt different format of XSCALE.INP file - I am working on this and will publish a patch here (and on the mailing list) as soon as I have one...

Wednesday, 19 September 2007

Error corrections

Currently working on a "proper" way to correct the chi-squared values (and hence the sd's) of intensities from Scala - first pass seems that it is much more effective and much quicker than the current run-scala-lots-remerging solution...

Monday, 10 September 2007

New Support Email

So, there are issues with emailing me direct at work - mostly to do with attachments. Thus I have set up a google mail account for providing xia2 support, which gives me around 2GB of storage.

Email at - of course the xia2bb is still there as well.

Tuesday, 28 August 2007

xia2 and very latest XDS/XSCALE

A problem has been found with the very latest version of XSCALE. WK changed the way that the input to XSCALE is written which I have only just found out about. I guess that this will be a key feature of

In the meantime please use an earlier than a couple of weeks ago version of XSCALE! Ill publish a "fix" as soon as possible.


To a large extent this fixes the bugs found in the previous version. However there are two new "features" which address some current issues. The fixed bugs relate to the problems people have been finding with detector names. This should never have happened! The cell refinement images are now selected in a more careful way based on the crystal orientation and trying to find the best images for refining individual cell axes. This appears to work well.

There is a test performed to check that the selected lattice symmetry is good. This uses the ratio of rmsd with and without the lattice constraints. If this ratio is > 1.5 by default then that indexing solution will be rejected. If you think this is wrong, add -rejection_threshold 2.0 (say) to the command line.

I am now by default using the latest version of Mosflm. If this proves to be a problem the old version can be specified by -old_mosflm.

Release can be found at the usual place:

Thursday, 23 August 2007

Help for Data Collection

So, there are lots of fun things in xia2 for doing data reduction - wouldn't it be good if there was something to help with the data collection? Well there is something on the way now, which will use Labelit (optionally) Mosflm and BEST to suggest strategies for data collection, in advance of a full "data collection expert system" which is still some way in the future but will take ideas from how xia2 is built to deliver data collection help.

Mosflm versions

Currently testing out the official 701 release of Mosflm with xia2. If this passes the tests then it will be included as the default version in the next release of xia2 -

Friday, 17 August 2007

Pilatus support

Is on the way... presuming that you have the images which contain the "mini CBF" headers. A version with this support should be available in the next month or so...

Friday, 10 August 2007 Buglet

If you find that perfectly correct indexing solutions are being rejected by the cell refinement test (with messages like "incorrect lattice constraints" you may have stumbled across a slight problem with the stability of the Moslfm cell refinement.

Previously I have used 1 (if tetragonal or higher, else 2) then 2, then 3 wedges for cell refinement. I found by jumping straight to 3 that the results are more reliable. However, on one test system and with one example the behaviour was completely wrong - it rejected a perfectly correct tetragonal solution with *very* high RMSD's. These are an artefact of unstable refinement.

If you remove the code:

# change 10/AUG/07 perhaps I should always be using
# three wedges?

'Cell refinement: overriding the number of wedges to 3')

num_wedges = 3



around line 768 everything should be fine. On windows this can be reached at


Any editor, even notepad, will be fine for this.


xia2 released

With the following fixes:

* Fixed bug where if the distance was read incorrectly from the
image header (or was wrong therein) XDS would get the wrong value even
if you had put the correct value in the .xinfo file.

* Now "unroll" the unmerged reflections from XSCALE and then merge
them in their original sweeps in Scala. This should give a useful Rmerge
vs. time plot.

* Including updated version of Pointless.

* Includes new and more robust handling of pointgroups, lattices
and unstable refinement of parameters during integration.

* Include support for a reference reflection file, which will
ensure that the reflections are indexed in the same way and with the
same spacegroup - useful for mutants.

* Now always use three wedges of images for the cell refinement
with Mosflm as this makes the lattice elimination more reliable.

* If one of the images in a sweep is broken (e.g. corrupted) xia2
will tell you more helpfully rather than just crashing.

* Now correctly read distance from Mar 165 images. Thanks to
Francois Remacle for this fix to DiffractionImage.

* Fixed the use of xia2 -3d with RAXIS IV detectors.

* Use more images for Mosflm autoindexing (three instead of two)
as this gives uniformly better results.

Duff images and odd exceptions

Previously if you have run xia2 but one of the images in a sweep was corrupted, all you got was a "child aborted" exception. Now it will tell you that there was a problem reading a certain image - very useful if you have a single broken image in a set!

Thursday, 9 August 2007

xia2 and Mar CCD 165

So, it has been noticed that xia2 (and particularly diffdump) does not work quite right with some Mar 165 images - the distance is not correctly found. This has been tracked down and will be fixed in the next version of xia2 - - which will be available shortly.

Wednesday, 8 August 2007

xia2, XDS and RAXIS

So, it turns out that xia2 didn't work with Raxis detectors for a number of reasons. These are now fixed -

- Raxis detectors need to have the distance negative.
- Different polarization plane normal.

Also added guess am I at a synchrotron check for the polarization fraction itself...

xia2 Environment

Since xia2 uses CCP4, the correct working environment is needed. This is usually set up by ccp4.setup, but if you are running on a cluster and you have qsubbed copying the environment across (which is a good way to run xia2) the BINSORT_SCR directory may not exist. In the next release of xia2 this is checked for and created if necessary.

Tuesday, 7 August 2007

New Scala & Windows

Next version of xia2 will probably have an updated version of scala included - 3.2.34. This is much improved in a number of ways which are not clear to me but I believe. However, this seems to require that on windows the CCP4_OPEN environment variable is in UPPERCASE as UNKNOWN not lower case as unknown. So if you find some strange errors with files this is probably the problem. The setup script should overwrite this though...

Mosflm autoindexing

As I have found that autoindexing with Mosflm works more reliably for three images than for two, I have made this the default for xia2. If you have more than 90 degrees of data then images at 0, 45, 90 are used, else images at the start, end and in the middle.

Friday, 3 August 2007

Software Versions

Just updating to 7.0.1 Mosflm (30th July 2007) and pointless 1.2.0. This should make integration more reliable with the 2D pipeline.

Software verrsions

Reference reflection files

This is now implemented. If you include a REFERENCE_REFLECTION_FILE in a CRYSTAL block it will be used for determining the setting and assigning the spacegroup. This should contain a single data set with intensity values. Currently the free set is not copied from this - I will write a small "jiffy" application to copy the FreeR_flag column from your existing data to the newly reduced data, as it is unusual to include the Free column in intensity data. is currently undergoing testing.

Monday, 30 July 2007

More developments towards

So, I have very much simplified the way that lattices and so on are handled, and the way that pointless is used. Now that I have a clear idea of what I am doing (which will hopefully be published in the fullness of time) the procedures are much more straightforward, which should greatly improve the reliability of the system.

Thursday, 26 July 2007

Developments towards

The following features are pencilled in for xia2 -

- Better XDS support
- More careful elimination of autoindex solutions
- Inclusion of a reference reflection file (must be MTZ, containing I's)

The latter is to use for setting standard indexing, spacegroup (if pointgroup is compatible) and FreeR flags. Useful when you want to refine an existing data set against newly processed data.

More information on progress and details will be posted here.