Bug? Nisus 3 seemingly randomly "Typesetting Text..." and sluggish, 50% CPU use, in long doc already open for hours

[withoutFeathers]

Hi,
Sometimes Nisus 3 refuses to search in my document without first spending 15-30 seconds "Typesetting Text", even though the document has been open for hours or days. Other times it searches the whole document instantly. Same document. This seems like a bug, or something that needs to be turned off that it's doing in the background.

More details:
It's a long document, 400K words.
Nisus is also often sluggish working in it, using up to 50% of my CPU, even though I'm only typing a few words of text in it. It seems it's doing something else also.

I've tried the usual -- restarting Nisus and MacOS, turning off things that it might be doing in the background. But at the moment I don't know why it's doing this.

MacOS 10.13.6 (High Sierra)
iMac dual core
Nisus 3.2.1

wF

[martin]

There's one explanation for the behavior you observed. Is your large document set to Draft View, but the search results can only be displayed in Page View? For example, if the find matches are in the headers/footers. That will force Nisus Writer to paginate your document, to show you the matching text in Page View.

[withoutFeathers]

There's one explanation for the behavior you observed. Is your large document set to Draft View, but the search results can only be displayed in Page View? For example, if the find matches are in the headers/footers. That will force Nisus Writer to paginate your document, to show you the matching text in Page View.

Hi Martin,
Thanks for the response. But no, I work in Page View also. And with the recent example of the behaviour I cited, all the find words were in the body of the document anyway.

However, there's something I perhaps didn't understand. If I edit, say, in the first thousand words of a 400K document, does that mean that all the typesetting from that point onwards is being done over again, every time I make a change? So another two minutes or more of "typesetting" (like when opening the document originally) is required? And Nisus will merrily set about that, over and over again, as I'm editing that early chapter?

If so, I understand better why it might take 50% of the CPU and slow down the editing.

But this is counterproductive, from my point of view. Is there any way to ensure that Nisus remains responsive to my actual editing needs, and only does this typesetting when I'm not doing something else?

Or, perhaps more accurate to say -- since my guess is you've already instituted that tradeoff -- is there any easy way for me to influence the tradeoff, so I can push it towards or away from background typesetting, if I find the editing too sluggish?

wF

[adryan]

G'day, wF et al

I haven't dealt with such long documents for a long time, but I too have observed sluggish typesetting behavior with NWP — for example, even on short A6 documents of less than 1,000 words. I can't recall whether it only began with NWP 3 or whether it happened with previous versions.

I haven't logged exactly what I've done, but typically a line (at the bottom of a page, I think — and I usually use Page View) becomes invisible after an edit somewhere and only reappears after the page is redrawn on scrolling. Even then, the redraw is not instantaneous, so it is somewhat annoying and not really what one expects.

Now I come to think of it, NWP didn’t always exhibit this behavior: I’m just not sure when it started to.

Cheers,
Adrian

[martin]

If I edit, say, in the first thousand words of a 400K document, does that mean that all the typesetting from that point onwards is being done over again, every time I make a change?

That's more or less correct. If you make an edit somewhere in a document, all the downstream content must be re-typeset. For example, editing on page 13 will require Nisus Writer to redo layout for pages 14 onward. This generally occurs in the background when you are idle, but it will use CPU. You may or may not notice the impact depending on Mac model, other current tasks, etc.

Or, perhaps more accurate to say -- since my guess is you've already instituted that tradeoff -- is there any easy way for me to influence the tradeoff, so I can push it towards or away from background typesetting, if I find the editing too sluggish?

I'm afraid not. Allowing the user to set some thresholds is an interesting idea, and is something we could consider adding. I'll file the potential enhancement for further investigation. It's not something that's necessarily straightforward, since background typesetting is a joint effort between Nisus Writer and the macOS text engine.

In the meantime your best workaround is to edit in Draft View, where typesetting is simply less demanding. Page View pagination and text layout is more complicated and uses more CPU.

[withoutFeathers]

If I edit, say, in the first thousand words of a 400K document, does that mean that all the typesetting from that point onwards is being done over again, every time I make a change?

That's more or less correct. If you make an edit somewhere in a document, all the downstream content must be re-typeset. For example, editing on page 13 will require Nisus Writer to redo layout for pages 14 onward. This generally occurs in the background when you are idle, but it will use CPU. You may or may not notice the impact depending on Mac model, other current tasks, etc.
...[snip]
In the meantime your best workaround is to edit in Draft View, where typesetting is simply less demanding. Page View pagination and text layout is more complicated and uses more CPU.

Thanks for the explanation. I may move to Draft View, but In order to help me stay in Page View as long as possible, I'd like to know something else that may help, about opening many documents:

1. Suppose I'm working in Long Document A. I then also open another one, Long Document B, check something in it, and then continue working in Long Document A. (I do this a lot).

a) If I leave Document B open, is Nisus occupying itself trying to Typeset all of Document B, which will then interfere with my working in Document A?

b) If so, if I instead close Document B quickly, will this improve the situation? Or will Nisus still be working on Document B in a cache somewhere?

c) And/or, will changing Document B to Draft View help the situation?

Sometimes I open several long documents in a row, checking a section of text in them relative to Document A. It will help to know if, and how, this is stealing CPU usage from my editing in Document A.

wF

[Vanceone]

Another thing that may impact this is autosave.

I frequently run into the spinning beach ball when I have a complex document filled with images. Frequently I have to add, say, 70 or so full page PDF graphics as sort of an appendix. When I do that, saving the document becomes long, not quite sure why. Like, 5 to 10 seconds long. Which, when you have autosave turned on (a good idea) yet are working on the document, can really interrupt your flow as suddenly the document freezes up while it saves. I'm actually not quite sure why autosaves take the UI thread instead of a background thread, to be honest, but at least in my copy of Nisus it does. My solution has been to limit the frequency of autosave so it doesn't bother me as much, but that's always risky.

If you have a long document, autosave may be one reason why things feel sluggish if Nisus is taking more time than normal to save your document.

[withoutFeathers]

If you have a long document, autosave may be one reason why things feel sluggish if Nisus is taking more time than normal to save your document.

Thanks Vanceone.
And in my context, this would mean repeated autosaves in all the other open documents, maybe? Even if I don't edit in them? I'm not sure if that would impact the CPU much, but perhaps it would.

wF

[Vanceone]

If Nisus makes any change, it will trigger an autosave at some point. You can watch your window title, where it has the name of your document. If it has (edited)" after your document name, an autosave is in your near future. Some things like scrolling or moving the window don't add an edit, but some things will, and I think even if you don't change anything there might be background work. I guess something like if you adjust your default printer settings it may trigger a reflow or page layout in your document and thus trigger the edited state. But that's a guess on my part. Still, it's something to watch for.

[withoutFeathers]

If Nisus makes any change, it will trigger an autosave at some point. You can watch your window title, where it has the name of your document. If it has (edited)" after your document name, an autosave is in your near future. Some things like scrolling or moving the window don't add an edit, but some things will...

Got it. Good idea. I have two extra docs open now, and I can see that just selecting passages to count words (which is what I'm doing at present) doesn't trigger edit, and so probably the autosave isn't part of the CPU issue in my current situation.

wF

[martin]

1. Suppose I'm working in Long Document A. I then also open another one, Long Document B, check something in it, and then continue working in Long Document A. (I do this a lot).

a) If I leave Document B open, is Nisus occupying itself trying to Typeset all of Document B, which will then interfere with my working in Document A?

Possibly. If opening Document B did not force typesetting for the entire document, then Nisus Writer may continue processing it in the background. For example: if you open Document B to page 50 but it has 300 pages, then typesetting for pages 51-300 will continue in the background.

b) If so, if I instead close Document B quickly, will this improve the situation? Or will Nisus still be working on Document B in a cache somewhere?

c) And/or, will changing Document B to Draft View help the situation?

Yes, both of those could help the situation to reduce the amount of background processing.

[martin]

Frequently I have to add, say, 70 or so full page PDF graphics as sort of an appendix. When I do that, saving the document becomes long, not quite sure why. Like, 5 to 10 seconds long.

I'm sorry for the autosaving delays. The problem is almost certainly the size of your images (in bytes on disk). This FAQ on large file sizes goes over the details, but the fundamental issue is that Nisus Writer saves all image data. If you insert several large images your file size may quickly balloon beyond your expectations, increasing the amount of time it takes to save the data to disk.

[withoutFeathers]

Yes, both of those could help the situation to reduce the amount of background processing.

Thanks Martin.
Today I looked closely at the CPU usage, and still I feel something is not right about this background processing.

Short version: I tested, and changed a comma to a semi-colon in Doc A. With no change in line length, this still causes 3.5 minutes of 50% CPU usage by Nisus. No typesetting should be required, so what's happening?

Details of my test:
1. I have both Docu A and Docu B open (both long). I go away for while, come back, and CPU usage is close to zero -- about 3%.
2. I make the smallest possible change I can think of in Docu A -- I change a comma to a semi-colon. This does not change anything in the document other than that. I doesn't force a line change or anything else. Everything looks the same.
3. Now CPU goes up to 53%.
4. I just watch for 3.5 minutes. I don't do anything. CPU stays at 53%. Activity Monitor shows that it's Nisus that's using the CPU.
5. Finally it goes back to 3%.

So, conclusion: I changed a comma to a semi-colon, then did nothing for 3.5 minutes, and Nisus was typesetting everything again? But why? It didn't need to do anything. There's no conceivable difference; that line is functionally identical...isn't it?

Surely if this is what Nisus is doing, there needs to be some internal tweaking about when typesetting the entire document (after edit point) is re-done?

Or else it's an actual bug, and Nisus has started to do something else that's irrelevant?

wF