Sunday, April 7, 2013

ExtendScript Toolkit Unbearably Slow?


ExtendScript Toolkit Slowness

Note added 9-May-2017: in addition to the issues outlined below, another, now common, reason for slowness of ESTK on Macs is the 'app nap'. Background apps on Mac 'fall asleep' and become really slow (to save energy). If ESTK is talking to InDesign and InDesign falls asleep, things get really slow. Quickly switching back and forth to InDesign, then back to ESTK wakes them up. On CS6 you can disable App Nap as a 'Get Info' property on the ESTK and InDesign applications in the Finder. For CC and higher, no quick fixes. I have turned off App Nap on ESTK and InDesign CS6, and I use CS6 for script development when I need ESTK.

Recently, I was working on a fairly large script-based solution for Adobe InDesign.
Things progressed well, and the script was growing at a rapid pace.
For small tasks, I tend to write scripts more in a procedural, ad-hoc manner. For more complex tasks, I will normally define some formal object model with object classes, methods, getters, setters, inheritance,...
In this particular case, the system was teetering between the two realms, and it was clear I'd need to go for a formal object-oriented approach some time later, as the complexity started to increase.
But as I was still testing the waters and getting my head around the many aspects of the solution, I had decided not to start on an object model and a class architecture until things had become better defined. Often, with these projects, I go through a ping-pong process with the users and other stakeholders, and valuable information only emerges after a few initial prototypes have been tried out 'in real life'.
Because the script had to serve as a go-between InDesign and an outside system with a wide API (Application Programming Interface), I had a growing collection of symbolic constants, each representing some kind of 'magical' value for the outside system's API.
All was well: the solution grew, and gradually more and more functionality became available; users were impressed.
Speed and responsiveness were great, even when massive amounts of data were being chewed on.
ExtendScript, despite being an interpreted language, is amazingly fast if you know how to treat it correctly.
Until suddenly some unexpected issue came out of the blue: for some reason, ExtendScript Toolkit became unbearably slow.
At first I thought it was one of those unexplicable things that come and go. So I restarted my Mac, reset the preferences, created a brand new user account and tried that way, switched to another Mac. All to no avail.
Simply trying to run the script in ExtendScript Toolkit almost immediately caused the colored pizza to appear and everything ran like molasses.
Running the script straight from the InDesign Scripts palette worked fine. It was only when the ExtendScript Toolkit got involved that things got really slow.
Initially, it was still somewhat bearable. Having to wait a few minutes just to inspect a variable is annoying, but you can grit your teeth and wait it out. But things got out of control very rapidly: a few hours later, the minute-long waits had become half-hour waits.
Run, breakpoint, colored pizza for 30 minutes. Run, breakpoint, colored pizza for 30 minutes...
Ok, this lobster had enough: the water in the pot was becoming too hot, and something had to be done. I started to investigate.
I used the Mac's Activity Monitor to sample the ExtendScript Toolkit process, and found that the ExtendScript toolkit was working very hard. Some poking and prodding revealed that nearly all that hard work was related to the Data Browser element in the ExtendScript Toolkit.
Eventually, I figured out it had to do with the quantity of constants and variables I had created in the global variable space. The Data Browser had trouble displaying all those elements. If I closed the Data Browser, ExtendScript Toolkit's speed became normal.
For what little info I was able to glean from my poking around, I suspect the Data Browser has an extremely inefficient lookup buried somewhere inside its code: it looks like it has many levels of nested linear lookups, and as more elements start populating the global variable space, things get exponentially (or worse than exponentially) slower.
So, I had to refactor my code earlier than expected - and that fixed the issue. The main change was that instead of something like:

const kExternalSys_BoldAttribute   = 0x0001;
const kExternalSys_ItalicAttribute = 0x0002;
...

and so on, thousands of those, I made it into:

var kCST =
{
ExternalSys_BoldAttribute:   0x0001,
ExternalSys_ItalicAttribute: 0x0002
...
};
And instead of referring to kWordSys_BoldAttribute, I now referred to kKCST.WordSys_BoldAttribute and so on.
I further reduced the footprint in the global space by making the script fully object oriented, with nearly all functions as methods attached to object classes, rather than having gobs of globally defined functions.
And now it's all fine again!
So, the morale: keep the global variable space as empty as possible, or ExtendScript Toolkit will get ya!