Friday, 26 September 2008

xia2 released

After a fairly long delay I have put together xia2 - this has had a pretty significant amount of testing so should be more solid than previous versions - this is a worthwhile update. Anyhow, here are the changes:

Changes since

- All command line options now echoed to the standard output if -debug selected.
- Command line options now copied to .xinfo file when operating in automatic mode.
- XDS: low resolution limit determined from spot list from IDXREF.
- Now include explicit python version check in xia2 main program.
- High resolution limits for integration now assessed from Wilson plot rather than integration program log output.
- If no images are given, a more helpful error message is produced than before, viz:

| No images assigned for crystal DEFAULT |

- If images are not readable (i.e. a permissions problem) then warnings will be sent to the debug channel (switch on -debug = for more information.)
- If in automatic mode an image is given in place of a directory, xia2 will now complain in a more helpful way.
- Resolution limits will now be based on analysis of the reflection files rather than the program output, since this will generally give a more helpful answer...
- Now correctly set the reindex matrix for running correct with multiple sweeps of data for non primitive matrix (subtle bug, thanks to Kay Diederichs for helping to fix this one.)
- If images are missing from a sweep, xia2 will now tell you this before starting processing rather than giving a strange error - if the images are unreadable you will also be told.

The biggest fix is the sorting out of resolution limits - you should no longer find xia2 wondering off and then falling over, with ever decreasing resolution limits, in -3d mode. All available from the usual place...

Monday, 15 September 2008

Unhelpful error messages

So, this is an old bug which I still need to fix, but since I have had a user report (again) that it is a problem, I thought a description of what the problem is would be helpful.

If you see:

"no Integrater implementations assigned for scaling"

This means you have not given xia2 any data for processing. The classic case is that you run xia2 in completely automatic mode and include the image name, viz:

xia2 -3d /data/graeme/s15a/xtal8/xtal8_1_001.img

Since the last argument should be a directory xia2 gets confused. This is simple to fix, I know, and will be for the next release.

Another perhaps more interesting cause of this is when the images are not readable - either they belong to someone else or someone has messed with the permissions. Again I need to check this, but it is harder...

So there you go, if you see this message check that you are giving a directory and that the images are readable.

Thursday, 24 July 2008

xia2 released

xia2 has been released, and can be found in the usual place. The following changes have been made:

Changes since

Now supports CCP4 6.1 (forthcoming release) though you may need to add XIA2_CCP4_61=1 to your environment. This release should include xia2.
Added -resolution keyword, also support for this on a per-wavelength basis in the xinfo file.
Added support for reverse-phi data collection - either add reversephi to the xinfo file or -reversephi to the command line.

The last of these has been tested on only one data set from the Australian Synchrotron, so please shout if you get any problems. If you are a Labelit user, you may need to update to get this functionality. Also fixed the "KeyError aP" bug, which was due to an ancient version of iotbx.lattice_symmetry being picked up.

As always, if you have any problems please drop me a line...

Tuesday, 22 July 2008

Reverse Phi

Another feature in the next release ( will be to support data reduction from reverse-phi beamlines. This will need either "-reversephi" on the command line or REVERSEPHI within the sweep block in the .xinfo file...

Monday, 21 July 2008

CCP4 6.1

Next release of xia2 will include support for CCP4 6.1, since it is going to be included in there. Only thing you need to do is to export XIA2_CCP4_61=1 to tell it...

User defined resolution limits

So, this is something people have been asking for a while now and I have been resisting - well I have finally given in! In the next release of xia2 there will be support for an overall resolution range set from the command line or per wavelength / sweep resolution limits from the .xinfo file. This allows you to get complete control over what the system is doing and also gets around the currently a little bit broken resolution limit setting.

You don't need to do this though - it works as before otherwise.

You may find it is a bit faster (or much faster!) with this switched on though, since xia2 doesn't end up fussing around for the right resolution limit.

Friday, 11 July 2008

-freer_file but column not FreeR_flag

Had a bug report that adding the FREER_FILE option to a .xinfo file or -freer_flag to the command line caused a very nasty error if the file in question did not have a column named FreeR_flag. This is ok, except that the default in Phenix is now different. To fix this please download:

and run

tar xvfz patch-3156.tar.gz -C $XIA2_ROOT

this will update the wrapper for CAD and add a new module called FindFreeFlag. This runs MTZDUMP and looks at the column names to guess which is right. You free column should be called (something)free(something) and be of type I...

Tuesday, 1 July 2008

XIA2 release

Hello, World,

Have just updated the xia2 web page with the latest version of xia2 - - which includes:

- Now handles complex image names with e.g. + in them correctly.

- New option for processing -3dii added - this will run the 3D pipeline (XDS, XSCALE) using all images for spot picking and autoindexing - useful for properly tricky cases.

- Now correctly propogate the profile information to Mosflm when not using Mosflm for autoindexing - this is done by running a "fake" autoindexing task.

- Use the raster information for the cell refinement test, and changed the selection of images for this.

- Added support for the output of BioXHIT XML in the LogFiles directory - for project tracking.

- Now copy the XDS log files ("LP" files) into the LogFiles directory - as always these are the most up-to-date versions from

- Added support for latest release of XDS (June 2, 2008.)

As always, please get in touch if you have any problems...

Monday, 30 June 2008

xia2 on CCP4 wiki

Just as an update: I have received a couple of perfectly accurate comments that the documentation for xia2 is inadequate. I have started a xia2 wiki page on the ccp4 wiki:

Currently this is fairly weak, but the idea is to focus my documentation efforts on here. If you wish to mirror the changes on any other wiki please feel free...

xia2 & Latest XDS

The current version of xia2 ( will not work with the latest version of XDS. This was OK, except the previous versions of XDS expired today, so as of right now running with the -3d option won't work. I think I have fixed this so that it should work fine with the new version (June 2, 2008) and I am running final tests now. Should be able to bundle a release tomorrow.

Friday, 16 May 2008

xia2 in CCP4 6.1

Hi All,

xia2 should be included as a part of CCP4 6.1. There should be a beta release of this soon. If you get any problems with this please don't hesitate to get in touch as, although I have tested it, quite a few of the programs in CCP4 have changed and so may do slightly unexpected things - perhaps...

Tuesday, 13 May 2008

xia2 updates

For information, the reason I have been a little slow with the latest update for xia2 (0.3.0) is that I have been making changes to ensure that it works correctly with the forthcoming ccp4 6.1 release, in which it will be included. A lot of little things have changed which sadly resulted in a broken xia2 - however these are now fixed.

If you are waiting for any features in particular shout at me at the usual addresses or add a comment here and I will ensure they are included.

Pointless / child segmentation fault

If you have had a problem with pointless in xia2 runs giving "child segmentation fault" errors, you may have hit a problem I have just had reported. This is not the end of the world! This is for recent linux distro's for example Fedora 7.

It turns out that the version of pointless 1.2.16 I included in the xia2 bundle for linux is not the same as the one that Phil now has on his web page - so the fix is pretty straightforward. Download the latest pointless from the MRC in Cambridge:

and copy it to


You should then be sorted. It may be worth checking that you have execute permissions, and if not adding them with "chmod a+x pointless-1.2.16" since sometimes ftp removes these...

Thursday, 13 March 2008

XIA2 release

Have just updated the web page to include release - this is a bug fix for a couple of outstanding issues, namely spacegroup assignment and provision of an existing FreeR column from a file. This also fixes a number of other minor issues.

Changes since

Added possibility to give a reflection file with an existing FreeR_flag column, which will be copied to the output MTZ:
xia2 -freer_file free.mtz

FREER_FILE free.mtz

in the xinfo file (in the same way as the reference reflection file - indeed, this may be the same file...)
Added command line control of the spacegroup - note well that this means that the data WILL NOT be reindexed to a standard setting.
Added support for ADSC Quantum 270.
Intelligent selection of autoindex threshold for Mosflm.
Added support for latest XDS version.
Added support for pilatus 6M mini CBF images.
Fixed problem with cell refinement giving negative mosaic spreads sometimes for 2d pipeline.

Wednesday, 27 February 2008

Spacegroups & FreeR columns

Two feature requests that I am working on at the moment are the inclusion of spacegroup assignment from the command line or xinfo file, and copying of an existing FreeR set from a reflection file. These are both to aid use of xia2 in an automated context.

The spacegroup assigment can be either in a USER_SPACEGROUP record within the xinfo file or with -spacegroup on the command line. This spacegroup will be used to determine the lattice for integration and the pointgroup. If these are found to be wrong an exception will be raised.

If you assign the screw axes, these need to be in order of cell axis lengths, since otherwise I don't know what order you mean. This means that you may need to assign them as P 21 2 21, say.

The FreeR flag is more straightforward - either provide FREER_FILE in the xinfo file or -freer_file on the command line - the FreeR_flag column will simply be copied out. If the new data go to a higher resolution than the existing free column I don't do anything - you should know what you are doing (I assume) if you are using this option.

Friday, 15 February 2008

Resolution limit calculation

Ok, so it looks like there will be a better way of resolution limit calculation in the next (full) release of xia2. This has taken a lot longer than I had anticipated though as the calculations are relatively complicated.

As this is such a major change, the next release will have this as a command line option to switch on (at the moment it also only works with data from Scala as well...) but it will hopefully be worth it.

An interesting side effect of this is that the processing will most likely be much faster than it currently is, since I will ditch the (perhaps unnecessary) reprocessing of the data in Mosflm prior to obtaining a robust resolution limit. Should give an approximately 30% speedup on how things currently work.

Listening to: Massive Attack

Tuesday, 29 January 2008

Pilatus support

Since the latest Mosflm (702) supports pilatus detectors I am adding support in xia2. This means that the next version ( or 0.3.0) will be able to run "fine" with them.

As always - any problems to xia2bb or (at)

Tuesday, 15 January 2008

Where is 0.3.0?

So, the question is - where has the next release got to? Well, I am currently working on a "proper" way to determine the resolution limits for the data which will take account of things like Rmerge, I/sigma and so on in a hopefully holistic way. This is clearly not trivial.

If you feel that an intermediate "bugfix" release ( is needed, please give me a yell!

Listening to: Amorphous Androgynous

Thursday, 3 January 2008

CCP4 Study Weekend

Over the next couple of days I will be at the CCP4 study weekend in Leeds, and I have a couple of "lunchtime bytes" sessions, which due to a printing error are listed under "aix2". ;o) - Perhaps see you there

Wednesday, 2 January 2008

CCTBX into xia2? Result

So, the votes went 10 for, 0 against and 4 don't mind, so I will start to make use of CCTBX tools in future versions of xia2. CCTBX is free to download and pretty easy to install (binaries are available) and will also be included in CCP4 6.1 which is due sometime soon. Hopefully this won't be a problem...

Happy new year!