Bug found: error parsing Aimless XML file for MAD data

Found a bug in the parsing code for Aimless XML which only tries to extract the first data set (not enough testing, fail) only for MAD data sets... This will give an error something like:

Reindexing to first spacegroup setting: P 41 21 2 (h,k,l)
Status: error "('AUTOMATIC', 'DEFAULT', 'WAVE2')"

I have now fixed this in revision 4879, now testing prior to updated release... sorry for any inconvenience.

Warning: Deprecating Things

Trying to clean up the interface which will in the long run help the documentation, but in the mean time will make some warnings... if you see

Warning: -spacegroup option deprecated: please use space_group='P422' instead

Warning: -resolution option deprecated: please use d_min=1.5 instead

Don't panic - this is to be expected - but you may want to change the way you run xia2 or your scripts. More of a warning for beamline / automation people! The outcome of this should however be automated generation of command-line documentation and the ability to keep "recipes" for running xia2 in tidy files....

Corner case bug spotted (thanks Luca Jovine)

If you

• use xinfo files
• use -2d
• do not set START_END

you may find you get a weird complaint about detector_class.

This is easily resolved for the moment by setting START_END to correspond to the first and last image you want to process... I need to fix this ;o) it did not used to work like this...

Thanks to Luca for pointing this out and patience in debugging...

xia2 0.3.8.0 released

Major changes:

Changes since 0.3.7.0: Added DIALS (mostly)

• Added support for DIALS software. Thanks to Richard Gildea at Diamond for most of the work.
• Create AIMLESS xml output; useful for displaying results through alternative interfaces.
• Squashed nasty bug where input spacegroup set to e.g. I222 but pointgroup derived to be P222 (think this one was reported a while back but I could not find the source at the time... sorry!)
• Squashed some other bugs which have appeared in recent history.

Changes since 0.3.6.3: Bug fixes, add CC-half, removing Scala.

• -cc_half 0.5 (say) resolution limit criterion available - suggest set -misigma 0.5 -isigma 0.5 or similar with this.
• -freer_file where free flag != FreeR_flag fixed.
• Massively trimmed all of the command line options for the pipelines and removed the use of Scala in all pipelines, replaced with Aimless (not before time!) This means that -3dr and so forth will no longer work so please use -2d, -3d, -3di etc.
• -phil command line option now deprecated: phil parameters passed on the command-line will be respected, also phil files, without this clue
• Parallelisation options: can now cleanly process sweeps from one wavelength in parallel (to aid with multi-crystal data sets) use Phil parameters multiprocessing.njob=4 multiprocessing.nproc=12 multiprocessing.mode=parallel (say) to allow processing four sweeps at a time each using 12 cores. This multi-sweep parallelisation is probably most useful in conjunction with the Phil parameter multiprocessing.type=qsub, which tells xia2 to submit jobs to a cluster, with each job using nproc cores.

Running xia2 from the GUI

A question to xia2.support last week indicated that the documentation has no information on how to run xia2 from the GUI in ccp4i. While this is rather old and cumbersome, if you are on Windows it is close to your only option.

Here is a short guide.

Obviously you need to select xia2 GUI from the main CCP4i:

This will pop up something which looks like this:

Now confusion reasonably starts: where to put in the image information? The GUI works like the xinfo files specified in the manual (http://www.ccp4.ac.uk/newsletters/newsletter48/articles/Xia2/manual.pdf) so you first actually need to define what wavelengths you have (i.e. SAD, native etc.) then associate images with these wavelengths:

What I have done here is create a new data set Demo / Native and added a Native wavelength to it, which I have given images insulin_1_001.img and onwards in /data1/gw56/demo. Hitting "run" will now process the data with xia2 / Mosflm. To use XDS select "run processing with XDS" under key parameters.

Two comments: this GUI needs to be updated, and the manual needs to include the GUI! Sorry for this omission.

Big changes in the world of xia2

Heads up: a big commit just hit xia2 containing preliminary support for the DIALS project: this may have had side-effects we have not forseen, so if you have any problems over the next few days this may be related. One particular issue may be you need a relatively recent cctbx...

As always any issues to the usual address xia2.support@gmail.com

Installing xia2: with details

The installation instructions are really rather brief - some may say non-existent! For example

http://www.ccp4.ac.uk/newsletters/newsletter48/articles/Xia2/manual.html#x1-140009

For old UNIX hacks such as YT these are perfectly adequate, but are unhelpful for many. Here are some clearer instructions.

Useful Instructions

xia2 is written entirely in Python and relies on things like CCP4 CCTBX to perform more expensive crystallographic calculations, and as such it does not really need anything more than a properly configured crystallographic environment to run. xia2 does however need to know where it is, so it can look to find other bits of itself to run. Traditionally this has been done through the medium of setting an environment variable (XIA2_ROOT) and then sourcing a setup file (setup.sh or .csh depending on your platform). Today all that is really necessary is to ensure that the xia2/Applications directory is in the PATH.

The installation instructions today are therefore:

1. ensure that all of the dependencies namely CCP4, CCTBX, XDS are installed correctly and may be found in the PATH
2. unpack the xia2 distribution somewhere you are happy to leave it e.g. in /usr/local or /xtal or something similar
3. add something like "export PATH=/path/to/xia2/Applications:$PATH" to your .bash_rc file
4. start a new terminal and type "which xia2" - this should point to /path/to/xia2/Applications/xia2

You can find out more about PATH variables and .bashrc files here:

http://unix.stackexchange.com/questions/26047/how-to-correctly-add-a-path-to-path

So the full thing will look like:

1. mkdir ~/xtal/xia2
2. cd ~/xtal/xia2
3. tar xvfz ~/Downloads/xia2-0.3.6.3.tar.gz
4. cd xia2-0.3.6.3/Applications
5. pwd (to show the full path)
6. copy and paste this to the export PATH= line

If you type xia2 and see something like

Traceback (most recent call last):

  File "/tmp/xia2-0.3.6.3/Applications/xia2.py", line 38, in

    from xia2setup import write_xinfo

  File "/tmp/xia2-0.3.6.3/Applications/xia2setup.py", line 32, in

    from Handlers.CommandLine import CommandLine

  File "/tmp/xia2-0.3.6.3/Handlers/CommandLine.py", line 32, in

    from Schema.XProject import XProject

  File "/tmp/xia2-0.3.6.3/Schema/XProject.py", line 29, in

    from Schema.XCrystal import XCrystal

it is probably installed correctly.

xia2 major work happening

Just to be warned some major work is happening "under the hood" in xia2, which will be happening in branches but from time to time may result in broken-ness in nightly builds...

This is all worth it - the end result will be a leaner, meaner xia2 with much more capability :o)

Thanks for your patience...

Incoming feature: resolution limits from CC-half

Seems that CC-half is a very fashionable way of arriving at resolution limits these days, so am modifying xia2 to work using these: watch for this in the next release.

New year plans

One of the things I plan to start doing is "xia2 howtos" here on the blog - interesting test cases and how to resolve them. On my hit list are:

• working with weak data
• handling multi-crystal data sets

Any other suggestions? Please post in the comments section.

Cryptic error message from xia2 which means "you need new cctbx"

Happy new year!

Had this error reported a couple of times, which results in xia2 needing new features in cctbx - particularly a problem if you use e.g. Phenix 1.8.2.

Error ends:

  File "/Applications/PHENIX-1.8.2-1309/Contents/phenix-1.8.2-1309/cctbx_project/libtbx/phil/__init__.py", line 1906, in process_includes
    phil_path=phil_path).objects)
  File "/Applications/PHENIX-1.8.2-1309/Contents/phenix-1.8.2-1309/cctbx_project/libtbx/phil/__init__.py", line 1946, in process_include_scope
    where_str=object.where_str)
  File "/Applications/PHENIX-1.8.2-1309/Contents/phenix-1.8.2-1309/cctbx_project/libtbx/utils.py", line 873, in __init__
    error_prefix, module_path, where_str))
ImportError: include scope: no module Wrappers.XDS.XDSIntegrate (input line 4)

(not very helpful!) Best way to resolve this: install Phenix 1.8.4 or new cctbx nightly build. (I need to improve xia2 error reporting)

Sorry for any annoyance, Graeme