Mysterious Control Character

[useeger]

I found a mysterious Control Character in a Nisus document:

snap.png (50.26 KiB) Viewed 14883 times

After the digit 9 there is a unvisible character with no width represented by a small T before the return.
Does anybody know what it stands for? What does it do? How did I insert it? Can I search for it? Can I delete it without causing trouble?
Thanks for informations about it
Ulrich

[phspaelti]

The character code says it's #202C which Unicode explains as "POP DIRECTIONAL FORMATTING".

I'm pretty sure you can delete it, if you don't need it.

[useeger]

Thank you very much, Philip! That's what I wanted to know.

What tool did you use to find the character code? (so that I can help myself next time)

[adryan]

G’day, Ulrich, Philip et al

Strange characters like this often seem to arise when you copy something (generally a complete entity such as a telephone number) from an email message or a Webpage and then paste it into a text document. As Philip says, it's usually safe to delete them.

Deleting all occurrences in a document is not difficult, but it is a bit tricky because it's not easy to select one. One solution is to focus on one you can see, then perform a grep search to find it. Invoking the Character Viewer (usually in your Main Menu bar at the top of the screen) will reveal the Unicode code for the selected character. Returning to your document, you can copy the selection (you will see the character but it won't appear to be selected even though it is), paste it back into the search field and replace all such occurrences with an empty string.

I'm sure there is a way to do this that uses the Unicode codes for common characters like this, but I'm afraid I don't know it.

Cheers,
Adrian

[useeger]

Dear Adryan,

I tried to copy and paste the character in the Nisus document and it worked. I could also search for it in the document with the marvelous search function in Nisus. But when I pasted it in the search field of the character viewer nothing happened, I did not get any information about it. So the question is still: what tool tells me that this charcter is U+202C ?

[adryan]

G'day, Ulrich et al

Yeah, it's all a bit crazy. The Character Viewer is the tool you want, but (inexplicably) pasting into its search field does not give the correct character code, at least in this instance. However, if the character is selected in the Nisus document, it will automatically appear as the character of interest in the Character Viewer, whereupon its Unicode code will be revealed. (You can test this with the “9” character in your document.)

Cheers,
Adrian

[phspaelti]

Ulrich, Adrian,

I selected the character and I used Transform Text:Convert Character Codes:To Unicode Points

cheers

[useeger]

Great, there are still a lot of features in Nisus that are little known but very useful.

Thank you very much for that hint!

[adryan]

G'day, Philip et al

When I do that, the character is replaced by "\x".

However, once the aptly named "mysterious" character is selected, its Unicode code is revealed in the Formatting Examiner Palette, which is easier than using the Character Viewer.

Cheers,
Adrian

[useeger]

Yes, finally I found the "Formatting Examiner", very elegant solution for my problem.

Thank you all for your help!

[martin]

The Formatting Examiner palette is definitely the best tool for this job. Any time you select a single character (or a single composed character sequence) the palette will show you the relevant information:

uchars.png (60.46 KiB) Viewed 14814 times

You can see both the character name and code (aka Unicode code point).

The palette can also help you find other occurrences of the selected character in your document. Click the left/right arrow buttons in the palette to find and select the next/previous matching character, or click the gear button and choose a find command from the contextual menu, eg: Find All in New Window.

[phspaelti]

While the question has obviously been well answered by now, I did want to answer one point.

When I do that, the character is replaced by "\x".

I'm pretty sure you are not looking at the result correctly. Ulrich's character was in a gigantic font so it was hard to see. But what you get when you use the command is \x{202C}

You can also use the command :to Unicode Character Names in which case you get \N{POP DIRECTIONAL FORMATTING}.

Admittedly the Formatting Examiner is more convenient. I really ought to start paying more attention to it.

[adryan]

G'day, Philip et al

Yes, you're quite right, Philip. With that hint about the font size, I now see that it was set at 256 pt in Ulrich's document. As the font size is reduced, progressively more of the code string appears. At 256 pt, anything beyond the "\x" extended beyond the right margin and hence was invisible to me.

Thanks for clearing up that loose end.

Cheers,
Adrian