Line-wrapping bug: space, then punctuation

Everything related to our flagship word processor.
Post Reply
foobar
Posts: 53
Joined: 2005-09-22 09:08:09
Location: bugmenot.com
Contact:

Line-wrapping bug: space, then punctuation

Post by foobar »

I'm seeing a repeated problem with lines wrapping/breaking too early when a space, then a punctuation mark, comes directly after the word that should end the line. That is, a line in the middle of a pgph., whose contents are "blah blah blah verylongword ." -- where the "." should wrap onto the next line -- instead wraps after "blah", beginning the next line with "verylongword . ", and leaving a big unsightly space on the page.

Can this be fixed? Fast, if possible?
y8 | Miniclip | y3 | friv | ben10 |

Kino
Posts: 400
Joined: 2008-05-17 04:02:32

Re: Line-wrapping bug: space, then punctuation

Post by Kino »

As TextEdit behaves in the same way, I think it is not a problem with NW Pro but with the Cocoa text system.

A workaround is to put an invisible character called "zero width non-joiner" before the full stop(s) in question. You can use the following macro to insert ZWNJ at the caret (insertion point).

Code: Select all

Type Text "\x{200C}"
You might find the macro below better: it will insert ZWNJ only before full stops in selection(s) which are situated at paragraph end AND preceded by a space.

Code: Select all

Replace All '(?<=\x20)(?=\.$)', '\x{200C}', 'Es'
If you want it to process the whole document (not only in selections)...

Code: Select all

Replace All '(?<=\x20)(?=\.$)', '\x{200C}', 'E'
To remove ZWNJ, use this.

Code: Select all

Replace All '\x{200C}', '', 'Es' # in Selection(s)
or

Code: Select all

Replace All '\x{200C}', '', 'E'

foobar
Posts: 53
Joined: 2005-09-22 09:08:09
Location: bugmenot.com
Contact:

Re: Line-wrapping bug: space, then punctuation

Post by foobar »

Thanks for the workaround; I guess I can always insert a ZWNJ before every punctuation character that's preceded by a space. This seems awfully kludgey, though. Even if Apple can't be prevailed upon to fix the bug quickly it would be nice if NWP stuck these ZWNJs where they needed to go to fix this behavior, automatically, behind the scenes, and without user intervention needed.
y8 | Miniclip | y3 | friv | ben10 |

Kino
Posts: 400
Joined: 2008-05-17 04:02:32

Re: Line-wrapping bug: space, then punctuation

Post by Kino »

Apple is unlikely to change the behaviour because it does not seem to be a bug. Today I read http://www.unicode.org/reports/tr14/#CL.

On CL: Closing Punctuation:
The closing character of any set of paired punctuation should be kept with the preceding character [...] This is desirable, even when there are intervening space characters, so as to prevent the appearance of a bare closing punctuation mark at the head of a line.
Full stop is not mentioned in the CL section because it is primarily defined as decimal separator. However, under the description of Numeric Separator, you will find: "When not used in a numeric context, infix separators are sentence-ending punctuation. Therefore they always prevent breaks before."

That said. Do you really want the final full stop to be separated from the preceding word and left alone at the paragraph end? Perhaps such appearances might be really desirable for your typographical rules but, if not, the best solution would be to keep "blah blah blah ... blah blah veryLongWord." including the final full stop in the same and single line. And the simplest way to achieve it would be to decrease the font size of space characters in the last line(s). Here is a macro for that purpose. Select "blah blah blah ... blah blah veryLongWord." and run the macro to see how it works.

Code: Select all

# This macro decreases the font size of space characters in the selection interactively.

# If you hit "Restore font Size to N pt" buttton, N pt will be applied on all space characters in the selection. N must be a font size available as a menu command under Format:Size.

# You can use this macro to integrate a widow word into the preceding line.

$doc = Document.active
if $doc == undefined  # if there is no open document
	Exit  # exit silently
end

$sel = $doc.textSelection

if ! $sel.length  # if there is no selection of positive width
	Exit  # exit silently
end

# Get the maximum font size of the selection
$index = $sel.location
$limit = $sel.bound
$maxFontSize = 0
while $index < $limit
	$attr = $sel.text.displayAttributesAtIndex $index
	$range = $sel.text.rangeOfDisplayAttributesAtIndex $index
	$fontSize = $attr.fontSize
	if $fontSize > $maxFontSize
		$maxFontSize = $fontSize
	end
	$index = $range.bound
end

$maxFontSize = Cast to Int $maxFontSize

$numFound = Find All '\x20+', 'Es'  # Find All spaces in the selection
if ! $numFound
	'No space char in the selection, exit...'
end
$selections = $doc.textSelections

# Construct prompt messages
$message = 'Decrease font size of spaces in the selection'
$detail = 'Hit « Stop Macro » to stop macro'
$decrease = 'Decrease'
$restore = "Restore font Size to $maxFontSize pt"

while 1
	$answer = Prompt $message, $detail, $decrease, $restore
	if $answer == $decrease
		Menu ':Format:Size:Decrease'
		# The purpose of the following two commands is to let NW Pro refresh the display.
		Select Start
		$doc.setSelections $selections
	elsif $answer == $restore
		Menu ":Format:Size:$maxFontSize"
		Exit
	end
end
I'd like to have Refresh Display command ;-)

Edit: corrected a bug by replacing "rangeOfAttributesAtIndex" with "rangeOfDisplayAttributesAtIndex".

foobar
Posts: 53
Joined: 2005-09-22 09:08:09
Location: bugmenot.com
Contact:

Re: Line-wrapping bug: space, then punctuation

Post by foobar »

Thanks for all the macro help -- but you seem to have missed, or I've been unclear, that this is happening with all the punctuation marks I've tried, certainly not just the closing element of a pair of quotation marks or parens. It's happened with opening parens and quote-marks just as well. My example's period was just an example (and in any case periods are not paired so your quoted guideline does not apply to them either). This is quite clearly incorrect behavior.
y8 | Miniclip | y3 | friv | ben10 |

Kino
Posts: 400
Joined: 2008-05-17 04:02:32

Re: Line-wrapping bug: space, then punctuation

Post by Kino »

foobar wrote:It's happened with opening parens and quote-marks just as well.
No, it does not occur to my copy of Nisus Writer Pro 1.1 with opening elements such as “, ‘, «, (, {, etc. in Times, Helvetica, AdobeGaramond Pro. If you really have the problem with those characters, you would better send a sample document to Nisus Soft together with a PDF. But... which version of OS X are you running? Mine is 10.5.5 which has updated AppKit. Perhaps this might make the difference.
(and in any case periods are not paired so your quoted guideline does not apply to them either)
I have never said that the rule applies to mirrored characters only -- but what do you mean by "paired" precisely? -- and I did quote this: "When not used in a numeric context, infix separators are sentence-ending punctuation. Therefore they always prevent breaks before". If you have any doubt about the interpretation, you can read the original document of which I gave the link.

foobar
Posts: 53
Joined: 2005-09-22 09:08:09
Location: bugmenot.com
Contact:

Re: Line-wrapping bug: space, then punctuation

Post by foobar »

If you really have the problem with those characters, you would better send a sample document to Nisus Soft together with a PDF.
Yes, I really have the problem with those characters. And yes, I have sent a sample document (quite some time ago). If this doesn't get fixed in a timely manner (as it hasn't so far, I suspect because you're right that it's an issue with Apple's text engine rather than NWP), then I'll just use the ZWNJ hack to get around it. Thanks.
y8 | Miniclip | y3 | friv | ben10 |

User avatar
martin
Official Nisus Person
Posts: 4926
Joined: 2002-07-11 17:14:10
Location: San Diego, CA
Contact:

Re: Line-wrapping bug: space, then punctuation

Post by martin »

Kino is correct, the line wrapping is determined by the system text engine, which ultimately follows the Unicode consortium's line wrapping rules. Which version of OSX you are using could affect the behavior. The consortium consists of many companies, including Apple, and I imagine the rules are designed to make text wrap in a way that displays nicely.

Another odd example I've seen is the way in which parenthesized text can't be split apart from quoted material. If you have text like:
This is “quoted text” (enclosed afterwards)
By default the word "text" and "enclosed" can't be split across lines. Eg: if the word "enclosed" wraps to another line then the word "text" is along for the ride:
This is “quoted
text” (enclosed afterwards)
I can understand the reasoning for this wrapping, but it can be confusing and leave excessive whitespace. We did file bug reports with Apple on one or two of these kinds of wrapping oddities, but were told that things behave correctly as described by the Unicode line wrapping rules. At least there's zero width non-joiner to the rescue!

Perhaps if we add the "keep together" formatting option/feature it would be appropriate to investigate overriding Apple's layout engine so that whitespace always allows breaking apart text into separate lines.

Post Reply