I thought I'd fixed the problem of target words appearing more than once in the output, but I hadn't done enough. While I had prevented a word from appearing twice in one phrase, it could still get into the output twice by being aligned to words in two separate but <linked> spans. Fixing this required a bit of restructuring, as generation of target phrases needed to be more aware of what was going on elsewhere.

The gist of the fix was to create a new class, TranslationPair.SpanSet, which groups connected spans together, and coordinate the generation of target phrases from there. Any target tokens which are used by one span are then unavailable for use by any other spans.

Whilst making this fix, I also removed all support for multiple alignment levels. Having discovered that the GIZA++ alignment scores are of no use to me, there is no requirement for this functionality and it just complicates the code.

There was no major BLEU score change with this bug fix.