multi-file search

Get help using and writing Nisus Writer Pro macros.
Post Reply
ThosGreen
Posts: 16
Joined: 2021-09-15 03:28:09

multi-file search

Post by ThosGreen »

Has anyone written a macro to search all files in a folder?

The macro repository offers 'FindReplace in all open documents' and says: "This is a legacy macro. Nisus Writer can natively search all open documents. These macros apply the find and replace expressions that have been entered into the Find panel to all documents that are currently open for editing."
[url]https://nisus.com/pro/macros/browse.php?keywords=search[/https://nisus.com/pro/macros/browse.php?keywords=search] But I can't see any way for Nisus to do that natively – have I missed something?

And I can't locate the macro that the repository entry claims to link to.

This topic came up back in 2013 ([viewtopic.php?p=23550#p23550][/viewtopic.php?p=23550#p23550]) but doesn't seem to have led anywhere.

Any suggestions / links / clues / guidance? I'm writing a book and I often need to search the several files to remind myself what I said about something.

PS added later: I've now discovered how to search all open documents, I quite overlooked it. BUT -- I have a lot of related documents, and I don't want to open them all. BBEdit can search all files in a folder (and all subfolders, recursively) – that's what I'd like to have.
User avatar
martin
Official Nisus Person
Posts: 5227
Joined: 2002-07-11 17:14:10
Location: San Diego, CA
Contact:

Re: multi-file search

Post by martin »

I'm glad you found out how to natively search all open documents in Nisus Writer. For anyone else wondering how to do it: in the Find panel set the "Where" popup to "All Open Files".

I understand the desire to search multiple files in a folder without opening them. Unfortunately this is not possible natively, sorry. I'll add your vote for the feature.

In the meantime it should be possible to write a macro that accomplishes the task. It's likely to be relatively slow, since the macro would need to fully open all documents before searching them. But perhaps this is not an issue if your documents are quick to open or few in number.
Þorvarður
Posts: 410
Joined: 2012-12-19 05:02:52

Re: multi-file search

Post by Þorvarður »

I can't locate the macro that the repository entry claims to link to.
The macros are named "Find in All Open Documents" and "Replace in All Open Documents". If you have difficulties finding them you can choose the menu command 'Show Macros Folder in Finder' from the Macro menu and then use Spotlight to locate the macros in the Macros folder.

The macros require the files that are being searched to be open. This becomes a problem when we have many files (let's say, more than one hundred); it simply takes too much time to open them all.

In this case, a much better approach is therefore to use the Document Manager. Add a Folder Group and choose 'Document Content' or 'Advanced Search'. This is how you can fast search hundreds, perhaps thousands of Nisus files without opening them! And the search is lightning fast.
BBEdit can search all files in a folder (and all subfolders, recursively) – that's what I'd like to have.
Unfortunately, when in the Document Manager, we can't search subfolders.

Martin, please add my vote for hierarchical groups in the Document Manager!

Another inconvenience is not being able to *replace* text in the Document Manager. In order to do that, the files must be opened first. It would be nice if we could search AND replace text in the Document Manager as well, … without having to first open the files.
User avatar
martin
Official Nisus Person
Posts: 5227
Joined: 2002-07-11 17:14:10
Location: San Diego, CA
Contact:

Re: multi-file search

Post by martin »

Þorvarður wrote: 2022-11-29 16:06:08In this case, a much better approach is therefore to use the Document Manager. Add a Folder Group and choose 'Document Content' or 'Advanced Search'. This is how you can fast search hundreds, perhaps thousands of Nisus files without opening them! And the search is lightning fast.
This is a nice tip! But it does have a big limitation: it uses Apple's Spotlight search index. This may be sufficient for simple searches, but it won't help with sophisticated search options, situations where you need to see the matches in context, text replacement, regex patterns, etc.
ThosGreen
Posts: 16
Joined: 2021-09-15 03:28:09

Re: multi-file search

Post by ThosGreen »

Thanks Martin and Þorvarður, very helpful answers. But recursive search is essential.

For background: the book I'm writing is divided into a few (5) parts. In each part there are several chapters/essays. Each chapter has associated with it the usual detritus of out-takes, diagrams, references, reminders of what I must remember to say, and so on. So: the book has a folder for each part; in those folders there is a folder for each chapter; in the chapter folders there are subfolders for bits and pieces. In addition, there are top level folders called things like 'bits we might use', 'papers to cite', etc - and each of those might have subfolders.

BBEdit can recursively search the whole heap in short order, just under a minute on my Mac. That's perfectly acceptable for the usual search case, which is "Have I mentioned X before and if so what did I say about it?" What BBEdit offers me is RTF, which is far from ideal, but I can make do if I have to.

If Nisus has to open every file it will take a long, long time. Definitely not acceptable – sorry, Martin.

I don't know how big the demand is for an affordable book-sized document processor, maybe not big enough to justify putting effort into this. But if that's what you're pitching Nisus as, rather than as a tool for glossy fliers (like Pages), I suggest it's worth considering, because poking around the different bits of a document being constructed is an important part of technical writing. Nisus does lots of things so well ....

And I'd like to add my vote for subfolders in the Document Manager.
User avatar
xiamenese
Posts: 543
Joined: 2006-12-08 00:46:44
Location: London or Exeter, UK

Re: multi-file search

Post by xiamenese »

I'm going to break a habit of (roughly a third) of a lifetime! I don't like to discuss alternative software on a forum like this, but ThosGreen's problem pushes me to do it.

This is precisely the situation that Scrivener is designed around. All those sidecar files, notes to self, and written-uo sections would live as separate documents in Scrivener's binder, which is like ThosGreen's folder within a folder within a folder etc. structure. But the difference is that they are all part of a single project that can be searched as a whole, including using RegEx. Although the whole can be (and is best) split up into small text units (Scrivener is designed round that basic concept) they can be pulled together in the editor and worked on as a single unit. And at the end of the day those "documents" that form part of the final book can be compiled as a single document. There's much more…

But for ThosGreen's problem: Scrivener only opens in memory those documents in the Binder hierarchy that are actively being worked on at the time, so it is memory efficient. However, project search searches all the data in the project whether open or not, and builds a search list of the documents where the terms are found and highlights the terms. It can also do search and replace across the project, but caution is needed, because I don't think there's an undo buffer on that.

And as a cherry on the cake, Scrivener uses RTF as its format and plays very nicely indeed with NWP (and Bookends).

Possible downsides are: lists and tables are pretty basic, not up to the standard of NWP; Scrivener doesn't use "Track changes". There are ways round those, however.

For all my work except website work, my workflow is: write in Scrivener; compile to a standard RTF document including named styles; open automatically in NWP; run a Macro which imports and applies a style collection of my choosing and caries out a few other processes. I then spell check, run Bookends scan, etc.

To me, Scrivener is not a rival software; it and NWP are both key apps in my workflow.

Mark
User avatar
phspaelti
Posts: 1313
Joined: 2007-02-07 00:58:12
Location: Japan

Re: multi-file search

Post by phspaelti »

Side-stepping the philosophical discussions, here is my attempt at a macro for solving this problem. It will search through all sub-folders for relevant files and check them against the find dialog. It will then let you how many it found, and ask whether you want to open them and show the results.
You'll have to be the judge whether it's too slow or not.
It could probably be modified to do replace all (perhaps without showing the open files?), if so desired.
Attachments
Search All Files in Folder.nwm
(8.99 KiB) Downloaded 140 times
philip
User avatar
phspaelti
Posts: 1313
Joined: 2007-02-07 00:58:12
Location: Japan

Re: multi-file search

Post by phspaelti »

Here is another solution to this problem to take advantage of the search feature in the Document Manager described by Þorvarður.
The macro creates a user group and adds all the files in the folder as well as all files in all subfolders. That way they will be collected in one group and can be searched easily.
The user group will have the name of the folder with a suffix " (all)" attached.
Attachments
Create DM Group for Folder.nwm
(7.64 KiB) Downloaded 139 times
philip
Þorvarður
Posts: 410
Joined: 2012-12-19 05:02:52

Re: multi-file search

Post by Þorvarður »

phspaelti wrote: 2022-12-01 00:15:45 here is my attempt at a macro for solving this problem
Awesome! … Thank you Philip.
User avatar
martin
Official Nisus Person
Posts: 5227
Joined: 2002-07-11 17:14:10
Location: San Diego, CA
Contact:

Re: multi-file search

Post by martin »

phspaelti wrote: 2022-12-01 00:15:45 Side-stepping the philosophical discussions, here is my attempt at a macro for solving this problem. It will search through all sub-folders for relevant files and check them against the find dialog. It will then let you how many it found, and ask whether you want to open them and show the results.
You'll have to be the judge whether it's too slow or not.
It could probably be modified to do replace all (perhaps without showing the open files?), if so desired.
Thanks for the nice macro Philip! Hopefully you don't mind a bit of tinkering... I'm attaching an amended macro that has a few user-experience enhancements:

• give the user a choice to search only files directly in the chosen folder (instead of considering all nested subfolders).
• show a progress bar while the search is underway.
• present the user with a list of matching files to open at the end.
Attachments
Search All Files in Folder.nwm.zip
(3.87 KiB) Downloaded 142 times
NisusUser
Posts: 317
Joined: 2011-01-12 05:32:38

Re: multi-file search

Post by NisusUser »

Philip and Martin,

You guys are so helpful! Just seeing how you provide complex macros to help someone makes me smile. I'm thankful also for all the times you've both done things like that for me.
User avatar
phspaelti
Posts: 1313
Joined: 2007-02-07 00:58:12
Location: Japan

Re: multi-file search

Post by phspaelti »

martin wrote: 2022-12-05 12:52:43 Hopefully you don't mind a bit of tinkering... I'm attaching an amended macro that has a few user-experience enhancements:
Thanks!
Actually I had been thinking about adding a status bar myself. Thanks for that.

One thing I really don't like though is that this macro swamps the "Recent Items" menu. I was wondering whether anything can be done about that? If not that would be a feature request that I would like to see: "Suspend add to Recent Items" as an option on the macro Open command.

And finally one more potential improvement might be the following: If you run the macro while some files are already open — for example as a way to look for a particular file — if the open files are part of the folder(s) you are searching the macro will close your files! So maybe should avoid closing already open files …
philip
Post Reply