Aimless for merging data from XDS (N.B. not yet scaling!) now working

Bloody edge of svn has this working, which is useful if like me you have better things to be doing than recompiling Scala a lot :o)

Will be in forthcoming 0.3.4 release. Any time now.

Aimless

Have been working on including Aimless from Phil Evans (his "Pointless" replacement for Scala) and if you would like to try this out, the code is now in the trunk. Usual svn checkout, download aimless 0.1.4 or later from Phil's FTP site, then run -2da.

Early results show that it makes the overall xia2 pipeline quite a bit faster, while maintaining the quality of results. Worth looking at.

xia2 0.3.3.3 release

Hi Folks,

0.3.3.3 release of xia2 is now available - which includes a bunch of useful fixes but fundamentally no new functionality:

Changes since 0.3.3.2

Many bug fixes as a result of implementing a comprehensive testing scheme.
Support for full cbf format, making use of cbflib Python bindings pycbf now included in cctbx releases.
Merged functionality from XDSIntegraterR into XDSIntegrater, removed duplicate code. Will make for more maintainable suite in the future.
Improved 2d pipeline scaling, now tests 8 permutations.
Improved handling of twinned data for multiple sweeps.
Fixed occasional bugs with multiple sweeps, reindexing trigonal data sets with XDS.
Removed "old" scaling modes, 2d and 3d, as these were unhelpful.
Changes since 0.3.3.1

-xparallel -1 option added which will split your data into 30 degree chunks and process with forkintegrate Note well you are responsible for setting up forkintegrate correctly!

Extensive testing

So, finally I have set up a system for doing bulk tests, which is surprisingly illuminating. Results of this will follow, but if you have a data set where xia2 fails but manual processing works please get in touch - I will be happy to add it to the test system and work on making it work.

CCP4 6.2.0

If you try running xia2 with CCP4 6.2.0 and XDS it probably won't work, and will complain about something like "End of file".

If you get this, please get a new pointless from

http://www.ccp4.ac.uk/problems.php#6.2.0-programs

and everything should be fine.

Addendum:

My bad: this version is still buggy - better to get a version from ftp://ftp.mrc-lmb.cam.ac.uk/pub/pre and overwrite the ccp4 one with this.

Does anyone use xia2-summary.dat?

Just found this:

[gw56@ws050 3]$ cat xia2-summary.dat
Project: AUTOMATIC
Crystal: DEFAULT
Sequence length: 0
Wavelength: WAVE1 (0.97966)
Sweep: SWEEP1
Files /data2/gw56/jcsg/12287_2L/12287_1_E1_###.img
Images: 1 to 60
Beam 105.10 101.05 => 108.95 105.04
Distance 170.00 => 169.89
Date: Sun Sep 26 14:13:49 2004
Wavelength: WAVE2 (1.00000)
Sweep: SWEEP2
Files /data2/gw56/jcsg/12287_2L/12287_1_E2_###.img
Images: 1 to 60
Beam 105.10 101.05 => 108.93 105.04
Distance 170.00 => 170.09
Date: Sun Sep 26 14:17:56 2004
For AUTOMATIC/DEFAULT/WAVE1:
High resolution limit                           1.66    7.41    1.66
Low resolution limit                            36.8    36.8    1.7
Completeness                                    97.9    96.4    85.7
Multiplicity                                    4.3     3.5     2.9
I/sigma                                         16.5    33.9    2.2
Rmerge                                          0.043   0.023   0.372
Anomalous completeness                          94.3    92.4    78.2
Anomalous multiplicity                          2.4     2.3     1.6
For AUTOMATIC/DEFAULT/WAVE2:
High resolution limit                           1.65    7.39    1.65
Low resolution limit                            36.8    36.8    1.7
Completeness                                    96.3    96.4    74.5
Multiplicity                                    4.2     3.5     2.7
I/sigma                                         17.4    35.6    2.2
Rmerge                                          0.04    0.021   0.369
Anomalous completeness                          92.1    94.2    69.9
Anomalous multiplicity                          2.3     2.3     1.5
Cell:  51.570  51.570 157.600  90.000  90.000  90.000
Spacegroup: P 41 21 2

which I remember from a long time ago - just wondering, does anyone use this? Is it worth keeping (as I clear up...)

Issues with xia2 / 3d / pointless 1.6.1

Many thanks to bug reporters. If you see:

Status: error "End of File "

at the end of a xia2 job which looks like it should have worked, and you have recently updated your pointless version, you have hit a feature resulting from a small change in pointless. Essentially when copying reflections from XDS to MTZ format there is no real orientation matrix, so pointless makes one up. It used to be [1] and was changed to [0] as it is meaningless - however reindex relies on this and therefore explodes.

Phil has fixed this in 1.6.2 (thanks Phil!) and is quite reasonably of the opinion that this is a bug in reindex...

A subsequent question to this is why I still use reindex inside xia2 now pointless does it all. "Historical reasons" would be the fair answer i.e. I haven't got around to fixing this :o)