3 dimentional array how can I use it?

Get help using and writing Nisus Writer Pro macros.
dakofi
Posts: 17
Joined: 2003-02-07 07:48:35

3 dimentional array how can I use it?

Post by dakofi » 2008-08-28 04:43:46

I have a text which is organized according the following pattern.

- register-number [group.subgroup.member] e.g. 03.052.013
- text

All register-numbers with their corresponding texts are in one text document.

It looks like that:
[register-number][text]
or more specific
[group][subgroup][number][text]

In an other text document I have a list with simular register-numbers. A register-number may appear several times.

Now I want to fill in the text, referenced by the register-number from the first document, into the second document (merge).
The intention is that each reference has the corresponding text appended.
The format of the second document is then the same as the format of the first document (i.e.[group][subgroup][number][text]).

How is it possible to do that?

Thank you very much for your help.

Regards

dakofi

Kino
Posts: 400
Joined: 2008-05-17 04:02:32

Re: 3 dimentional array how can I use it?

Post by Kino » 2008-08-28 06:59:31

Hello again ;-)

Am I right in assuming your document looks something like this?
03.052.013 text 1
03.055.323 text 2
07.178.923 text 3
...
In other words, does "text" follow a register number immediately and is in the same line? If so, the following macro will work. The macro uses a hash called $contents -- not $text because it is a kind of registered word: Martin uses it for a text object ;-) -- in which text is defined for each registration number. Then, $contents{<number>} will return the text associated with the number.

Code: Select all

 Require Application Version '3.1'
$docs = Document.openDocuments
if $docs.count < 2
	Exit 'This macro requires two open documents, exit...'
end
$doc = Document.active
$contents = Hash.new
$numFound = Find All '^\d+\.\d+\.\d+.+', 'E'
if ! $numFound
	Exit 'Nothing found, exit...'
end

$subTexts = $doc.selectedSubtexts
Find All '^\d+\.\d+\.\d+', 'Es'
$regNumbers = $doc.selectedSubtexts
foreach $num in $regNumbers
	$range = Range.new 0, $num.length
	$subtext = $subTexts.dequeue
	$subtext.deleteInRange $range
	$contents{$num} = $subtext
end

Document.setActive $docs[1]
$doc = Document.active
$text = $doc.text
$numFound = Find All '^\d+\.\d+\.\d+', 'E'
if ! $numFound
	Exit 'Nothing found, exit...'
end

$selections = $doc.textSelections
foreach $sel in reversed $selections
	$num = $sel.subtext
	$content = $contents{$num}
	$index = $sel.bound
	$sel.text.insertAtIndex $index, $content
end

User avatar
martin
Official Nisus Person
Posts: 4368
Joined: 2002-07-11 17:14:10
Location: San Diego, CA
Contact:

Re: 3 dimentional array how can I use it?

Post by martin » 2008-08-28 12:28:12

Kino wrote:The macro uses a hash called $contents -- not $text because it is a kind of registered word: Martin uses it for a text object ;-)
haha, well you're free to name your variables whatever you please- they don't have to make sense to me ;)

Code: Select all

$catbus = $xibalba + $jellybeans{'bubblegum'}
Thanks for all the great macro advice/code you've been posting to the forums!

dakofi
Posts: 17
Joined: 2003-02-07 07:48:35

Re: 3 dimentional array how can I use it?

Post by dakofi » 2008-09-02 00:44:53

Hi Kino,

Thank you for quickly answering my questions. I tried it several times but the macro isn’t working as I’m expecting it.

Firstly, I started the ‘Referenztext’ document containing the references as you expected
example: 02.357.057text
Secondly, I started the ‘Referenzlist’ document in which I want to fill the text in.
Lastly, I started your macro.

Doing this leads to two scenarios:
In case I start the macro in the ‘Referenztext’ document

=>the macro is working for a long time
=>the references in the ‘Referenzlist’ document get highlighted but the corresponding text is not added.
=>the references and the text in the ‘Referenztext’ document get highlighted

Or in case I start the macro in the ‘Referenzlist’ document

=>the macro is working for a long time
=>the references in the ‘Referenztext’ document get highlighted
=>the references in the ‘Referenzlist’ document are not highlighted and the corresponding text is not added


Unfortunately, the text will never be filled in

My ‘Referenztext’ document is about 4.3MB. Could that be the reason for the problems?

Thank you very much for your support

Regards

dakofi

Kino
Posts: 400
Joined: 2008-05-17 04:02:32

Re: 3 dimentional array how can I use it?

Post by Kino » 2008-09-02 03:07:27

dakofi wrote:I tried it several times but the macro isn’t working as I’m expecting it.
The macro assumes that 'Referenztext' is the frontmost document and 'Referenzlist' is the second and that reg numbers and texts in those files are formatted in the following way:

Code: Select all

# Referenztext

03.052.013Giant Pocky
03.055.145Hello Kitty Pocky
03.055.323Almond Crush Pocky
07.178.294Pretz Roast
07.178.923Pretz Basil
04.023.818Giant Cone

Code: Select all

# Referenzlist

03.052.013
03.055.145
04.023.818
04.023.818
07.178.923
03.055.323
03.055.323
03.052.013
07.178.923
07.178.294
The macro should have stopped and reported when those files are not in the correct order but it does not due to a bug: the first Find All command

Code: Select all

$numFound = Find All '^\d+\.\d+\.\d+.+', 'E'
should be

Code: Select all

$numFound = Find All '^\d+\.\d+\.\d+[^\d\n].*', 'E'
Perhaps it would be better to modify the second Find All command too. But I‌ don't know the exact data format of your 'Referenzlist'. Anyway please try the corrected macro attached here.
files.zip
(11.2 KiB) Downloaded 464 times
If your 'Referenztext' and 'Referenzlist' are formated differently, please show us some samples (for example, 10 lines from both of them). I think I can modify the macro accordingly.
My ‘Referenztext’ document is about 4.3MB. Could that be the reason for the problems?
Oh, I don't know but can you create a small 'Referenztext' by extracting, for example, 50 or 100 lines from it and try to run the macro on the small 'Referenztext'?

dakofi
Posts: 17
Joined: 2003-02-07 07:48:35

Re: 3 dimentional array how can I use it?

Post by dakofi » 2008-09-03 05:08:51

Hi Kino,

The pattern of your example text is exactly the same as in my text.
I tried your macro and it works with small files (part of 'Referenztext' document (1MB) and a tiny little of the 'Referenzlist' document (16kB)). The text was successfully filled in.

With a part of the 'Referenztext' document (1MB) and with the full 'Referenzlist' document (240kB) the macro is aborted after about 2 hours and nothing is posted to the destination document.

After analyzing your macro I suggest, it could help to close the 'Referenztext' document after having filled the text into the hash table. It is no longer needed.

Generally:
NWP is working very slowly, although it is more or less the only application running.

Thanks again for your help, I really appreciate it.

dakofi

Kino
Posts: 400
Joined: 2008-05-17 04:02:32

Re: 3 dimentional array how can I use it?

Post by Kino » 2008-09-03 08:12:27

The pattern of your example text is exactly the same as in my text.
So am I right in assuming that the format of all reg numbers is 'dd.ddd.ddd'?
With a part of the 'Referenztext' document (1MB) and with the full 'Referenzlist' document (240kB) the macro is aborted after about 2 hours and nothing is posted to the destination document.
Perhaps NW Pro stalled...
After analyzing your macro I suggest, it could help to close the 'Referenztext' document after having filled the text into the hash table. It is no longer needed.
Maybe... but the main problem is the enormal size of your files. They do not look like a love letter, a cookbook, a novel... or are they? ;-)

If you don't care of style attributes, we can use Perl. With 'Referenztext' document (3.3 MB, 100000 definitions) and 'Referenzlist' document (300 KB, 20000 data), the following macro does the job in 1 minutes 7 seconds in my old G4 Mac.

Code: Select all

Require Application Version '3.1'
$startTime = Date.now

$docs = Document.openDocuments
if $docs.count < 2
	Exit 'This macro requires two open documents, exit...'
end
$doc = Document.active
$doc.clearAndDisableUndoHistory
$numFound = Find All '^[0-9]{2}\.[0-9]{3}\.[0-9]{3}.+', 'E'
if ! $numFound
	Exit 'Nothing found, exit...'
end
$definitions = $doc.selectedSubtexts
# Menu ':File:Close'  ## Uncomment this command if you want the definition file to be closed

Document.setActive $docs[1]
$doc = Document.active
$doc.clearAndDisableUndoHistory
$text = $doc.text
$numFound = Find '^[0-9]{2}\.[0-9]{3}\.[0-9]{3}', 'EW'
if ! $numFound
	Exit 'Nothing found, exit...'
end

$path = $doc.filePath
if $path == undefined
	$fileName = $doc.displayName
else
	$fileName = $path.lastFilePathComponentWithoutExtension
end
$outputPath = File.temporaryPathWithName "$fileName (output).rtf"

$text = $doc.text
$text = Cast to String $text
$text = $text.split "\n"
$output = ''

Set Exported Perl Variables 'definitions', 'text', 'output'
begin Perl
	my %dic = ();
	foreach $def (@definitions) {
		$def =~ /(^[0-9]{2}\.[0-9]{3}\.[0-9]{3})(.+)/;
		$dic{$1} = $2;
	}
	foreach $line (@text) {
		$line =~ s/^([0-9]{2}\.[0-9]{3}\.[0-9]{3})/exists $dic{$1} ? $1 . $dic{$1} : $1/geo;
	}
	$output = join "\n", @text;
end
$definitions = ''
$text = ''

$output &= "\n"
$output = Encode RTF $output
File.writeDataToPath $output, $outputPath
Open $outputPath

$finishTime = Date.now
$seconds = $finishTime.secondsSinceUnixEpoch - $startTime.secondsSinceUnixEpoch
$message = 'Finished in '
$minutes = Cast to Int $seconds / 60, false
if $minutes
	$message &= "$minutes m "
end
$seconds = $seconds % 60
$message &= "$seconds s.\n\n"
$message &= "The output file is in a folder for temporary files.\n"
$message &= "Save it somewhere in your home directory if you want to keep it."

Exit $message
NWP is working very slowly, although it is more or less the only application running.
But you are trying to use a word processor for the task which is done usually in the Terminal (perl, ruby, phthon or a C program). I don't say it is a wrong thing. Me too, I often do something similar. However, that kind of jobs is the least suitable for NW‌ Pro which is obliged to take care of style attributes and layout. As far as the automatization of normal tasks for word processors, I think NW Pro macro is very strong and fast.

User avatar
martin
Official Nisus Person
Posts: 4368
Joined: 2002-07-11 17:14:10
Location: San Diego, CA
Contact:

Re: 3 dimentional array how can I use it?

Post by martin » 2008-09-03 11:34:37

dakofi wrote:NWP is working very slowly, although it is more or less the only application running.
In this case the macro runs very slowly because of the very large number of selections present during the text modifications. Each time you make a text change NWP is required to update all selections so they still cover the same logical characters. If you have many thousand selections and many thousand replacements, this ends up being quite a lot of work.

You can greatly reduce the running time of Kino's first macro by simply deselecting the found text before entering the modification loops, eg:

Code: Select all

Require Application Version '3.1'
$docs = Document.openDocuments
if $docs.count < 2
   Exit 'This macro requires two open documents, exit...'
end
$doc = Document.active
$contents = Hash.new
$numFound = Find All '^\d+\.\d+\.\d+.+', 'E'
if ! $numFound
   Exit 'Nothing found, exit...'
end

$subTexts = $doc.selectedSubtexts
Find All '^\d+\.\d+\.\d+', 'Es'
$regNumbers = $doc.selectedSubtexts
Select End # to make macro run quicker
foreach $num in $regNumbers
   $range = Range.new 0, $num.length
   $subtext = $subTexts.dequeue
   $subtext.deleteInRange $range
   $contents{$num} = $subtext
end

Document.setActive $docs[1]
$doc = Document.active
$text = $doc.text
$numFound = Find All '^\d+\.\d+\.\d+', 'E'
if ! $numFound
   Exit 'Nothing found, exit...'
end

$selections = $doc.textSelections
Select End # to make macro run quicker
foreach $sel in reversed $selections
   $num = $sel.subtext
   $content = $contents{$num}
   $index = $sel.bound
   $sel.text.insertAtIndex $index, $content
end
In my tests this reduced the running time of the macro from 120 seconds to about 5 seconds.

dakofi
Posts: 17
Joined: 2003-02-07 07:48:35

Re: 3 dimentional array how can I use it?

Post by dakofi » 2008-09-08 11:41:19

Thanks Kino for your help.
Your perl macro does it's work very fast.


Thanks Martin for your macro. It fills in the text with all styles.
To see the filled in text at the screen, while the macro is still working, is very impressive.

Is this feature (update of the screen), perhaps absorbing too much performance of the computer?

For small lists the macro comes good to the end

But for the big ones the macro was still working after 6 houres
Unfortunately I wasn't able to stop the macro.
That was possible in Nisus Classic.

Is there a way to abort the macro without loosing the already done work?

Is it possible to do "save as" while the macro is working, perhaps with an extended filename?


Thanks for your feedback.

dakofi

Kino
Posts: 400
Joined: 2008-05-17 04:02:32

Re: 3 dimentional array how can I use it?

Post by Kino » 2008-09-08 23:03:22

dakofi wrote:But for the big ones the macro was still working after 6 houres
Unfortunately I wasn't able to stop the macro.
NW Pro 1.1 seems to have a bug which prevents Find All in Selection (Find All '^\d+\.\d+\.\d+', 'Es') from working properly when there are a great number of selections. I reported it via feedback. Hopefully it will be fixed in a near future.

Kino
Posts: 400
Joined: 2008-05-17 04:02:32

Re: 3 dimentional array how can I use it?

Post by Kino » 2008-09-09 03:10:33

This one seems to work, fast enough thanks to Martin's great tip. Please try it if you need preserving style/formatting attributes.

Code: Select all

Require Application Version '3.1'
$docs = Document.openDocuments
if $docs.count < 2
	Exit 'This macro requires two open documents, exit...'
end
$doc = Document.active
$doc.clearAndDisableUndoHistory
$contents = Hash.new
$numFound = Find All '^\d+\.\d+\.\d+[^\d\n].*', 'E'
if ! $numFound
	Exit 'Nothing found, exit...'
end

$subTexts = $doc.selectedSubtexts
Select End

Document.setActive $doc
Find All '^\d+\.\d+\.\d+(?=[^\d\n])', 'E'

$regNumbers = $doc.selectedSubtexts
Select End # to make macro run quicker

foreach $num in $regNumbers
	$range = Range.new 0, $num.length
	$subtext = $subTexts.dequeue
	$subtext.deleteInRange $range
	$contents{$num} = $subtext
end

$regNumbers = ''

Document.setActive $docs[1]
$doc = Document.active
$doc.clearAndDisableUndoHistory
$text = $doc.text
$numFound = Find All '^\d+\.\d+\.\d+', 'E'
if ! $numFound
	Exit 'Nothing found, exit...'
end

$selections = $doc.textSelections
Select End # to make macro run quicker
foreach $sel in reversed $selections
	$num = $sel.subtext
	$content = $contents{$num}
	$index = $sel.bound
	$sel.text.insertAtIndex $index, $content
end
dakofi wrote:Is it possible to do "save as" while the macro is working, perhaps with an extended filename?
Possible but such a trick is not necessary for this version, I think.

User avatar
martin
Official Nisus Person
Posts: 4368
Joined: 2002-07-11 17:14:10
Location: San Diego, CA
Contact:

Re: 3 dimentional array how can I use it?

Post by martin » 2008-09-10 14:42:12

dakofi wrote:Is there a way to abort the macro without loosing the already done work?
You can press Command + "." (period) or just the Escape key while a macro is running to abort it before its work is complete.

dakofi
Posts: 17
Joined: 2003-02-07 07:48:35

Re: 3 dimentional array how can I use it?

Post by dakofi » 2008-09-15 08:29:51

Hi

Your macro "fill in the text with all styles" works well. It crashes at line 49 and 50 which are the last lines of the macro. If redundant returns are deleted which are at the end of the lines, the crash happens only at line 49. But after confirming by pressing the continue button, the job is finished perfectly.


Thanks for your feedback.

dakofi

dakofi
Posts: 17
Joined: 2003-02-07 07:48:35

Re: 3 dimentional array how can I use it?

Post by dakofi » 2008-09-17 06:50:45

Hi Martin hi Kino
unfortunately, the execution of your macro cannot be aborted by pressing command + . and I don't know why.

For my own makros I’d like to save the modified document as a new version automatically by the macro. But I can’t come through with "save as, filenamef", it doesn’t work.

Thanks for your feedback.

dakofi

User avatar
martin
Official Nisus Person
Posts: 4368
Joined: 2002-07-11 17:14:10
Location: San Diego, CA
Contact:

Re: 3 dimentional array how can I use it?

Post by martin » 2008-09-17 12:07:20

dakofi wrote:unfortunately, the execution of your macro cannot be aborted by pressing command + . and I don't know why.
The abort/cancel keyboard shortcut can't always immediately stop a running macro. Cancellation will only occur in between individual commands. For example, if a single "find all" operation is taking a long time it must finish before your Command + "." request is taken into consideration.
For my own makros I’d like to save the modified document as a new version automatically by the macro. But I can’t come through with "save as, filenamef", it doesn’t work.
Command arguments are not separated from the command name using a comma, so the command must look like one of the following:

Code: Select all

Save As '~/Desktop/file.rtf'
Save As $pathVariable

Post Reply