Some baffling behavior in InDesign CS3 scripts explained - beware of relative object references

Lesson learned: be careful with references to page items in InDesign ExtendScript CS3 - you might be referring to the wrong object!

The issue described below seems fixed in InDesign ExtendScript CS4 - but if you need to support CS3, you should be aware of it.

Look at this code snippet:

var itemB = app.activeDocument.pageItems.item(1);
alert(itemB.id);
... do some stuff that does not affect itemB ...
alert(itemB.id);

What would you expect to happen? You'd expect to see the same id value displayed in two alerts, right?

itemB refers to a particular page item, and should continue to do so. Turns out that is not the case in InDesign CS3.

Try this:

Create a new document, and create three rectangular frames on the first page.

Create the following script:

var itemA = app.activeDocument.pageItems.item(0);
var itemB = app.activeDocument.pageItems.item(1);
var itemC = app.activeDocument.pageItems.item(2);
alert(itemB.id);
itemA.remove();
alert(itemB.id);

You would expect itemB not to be affected by removing itemA - but it is! Run the script and you'll get two different id values displayed in the dialog.

I don't really know for sure what is going on but I have a theory - it looks like itemB has 'become' itemC.

I suspect that InDesign CS3 does not really keep a 'hard' reference to any particular item in the itemB variable - instead it has what I'd call a 'relative reference'.

In other words, itemB is not fixed - it always remains equivalent to 'app.activeDocument.pageItems.item(1)' - and if the page item list changes 'underneath', then itemB suddenly will refer to another item.

After the remove(), the page item list of the document has changed - the item at index 1 is now a different item. So the variable itemB suddenly refers to a different item.

Can we create hard references that don't suffer from this issue? Turns out we can - so there is a solution.

Change the script to read:

var itemA = app.activeDocument.pageItems.item(0);
var itemB = app.activeDocument.pageItems.item(1);
var itemC = app.activeDocument.pageItems.item(2);
itemB = app.activeDocument.pageItems.itemByID(itemB.id);
alert(itemB.id);
itemA.remove();
alert(itemB.id);

That looks like a do-nothing: replace the reference to itemB with another reference to itemB

But internally, InDesign will now refer to 'the item with id=1234' (assuming itemB had an id equal to 1234). itemB.id uniquely identifies itemB, and it is not affected by items being added or removed from the page item list.

So this updated script works as expected - we see the same id displayed before and after the remove.

We can write a small helper function for this:

function FreezeReference(theItem)
{
// Harbs improved my original attempt by
// appending .getElements()[0] - thanks
// Harbs!
return theItem.parent.pageItems.itemByID(theItem.id).getElements()[0];
}

var itemA = app.activeDocument.pageItems.item(0);
var itemB = FreezeReference(app.activeDocument.pageItems.item(1));
var itemC = app.activeDocument.pageItems.item(2);
itemB = app.activeDocument.pageItems.itemByID(itemB.id);
alert(itemB.id);
itemA.remove();
alert(itemB.id);

This also works as expected.

Uncovering this issue was not as straightforward as it might seem - at first, it looked like I was struggling with a bug in our APIDToolAssistant plug-in: Harbs (www.in-tools.com) who is an advanced user of APIDToolAssistant reported some issues with the 'move into' function provided by APIDToolAssistant.

This particular ExtendScript enhancement allows you to move one page item into another very quickly; much faster than could be achieved with pure scripting - you'd do something like this:


// Some 'magic' numbers - C++ meets ExtendScript
const IID_IACTIVEPAGEITEMSCRIPTUTILITIESEXTENSION = 0x90B6C;
const kOpCode_MoveInto = 10016;

...

function MoveItemIntoItem(newParent,newChild)
{
app.callExtension(
IID_IACTIVEPAGEITEMSCRIPTUTILITIESEXTENSION,
kOpCode_MoveInto,
newParent,
newChild);
}

var itemA = app.activeDocument.pageItems.item(0);
var itemB = app.activeDocument.pageItems.item(1);
var itemC = app.activeDocument.pageItems.item(2);
alert(itemB.id);
MoveItemIntoItem(itemA,itemB);
alert(itemB.id);


app.callExension() 'calls into' the APIDToolAssistant plug-in to perform the move.

He noticed that he got invalid object references after calling this function in CS3.

Turned out APIDToolAssistant was not the culprit - after some digging I finally figured out what was going on.

Hope this is useful!

Cheers,

Kris

VMware Fusion and TimeMachine

Been a while since I blogged; swamped in work.

This quick post to share a solution I found on Mac OS X to make VMware Fusion (or Parallels) and Apple's Time Machine backup work better together.

I use a fair amount of virtual Windows and Linux machines, and these tend to be primarily stored in large virtual hard disk files - 40GB, 60GB,... pretty massive.

Because Time Machine finds and copies any modified file, and the mere act of running the virtual Windows machines caused these large files to be marked as 'modified', I was faced with endless copies of massive virtual hard disk files to my Time Machine hard drive.

Figuratively speaking, a one-byte change in one of those massive Windows virtual hard disk files would cause a 60GB copy operation - which also indirectly forced older backup files to be erased from the Time Machine drive to make room for these behemoths.

Initially, I put all my virtual machines in a single folder, and explicitly excluded that folder from the Time Machine backup - not a good solution, but at least, my backup drive was not swamped with those big files.

But then, a few days ago it dawned on me - I could have my cake and eat it too!

The solution is to use the 'snapshot' feature of VMware (Parallels also has it) - you can make a snapshot of a virtual machine, so you can always 'undo' whatever happened to that virtual machine since the last snapshot.

The way VMware handles this is by 'freezing' the underlying virtual hard disk, and storing any changes made since the snapshot to the frozen hard disk in separate files.

And that's the solution: I first make sure my virtual machine is in a useful, stable state, and then I make a snapshot (I call it 'Base').

That effectively 'locks' the massive many-GB virtual drive - so running VMware does not cause it to be modified any more, and any changes from then on are kept in a bunch of much smaller snapshot files.

Time Machine now makes a copy of my 'frozen' many-GB virtual drive once, and from then on only backs up the changes that are kept in the snapshot files - which results in a much smaller backup set.

After a while, when the snapshot file size grows to many GB in size, I make a new snapshot, and delete the old snapshot. That 'merges' all changes that occurred 'between' the two snapshots into the main virtual drive, and starts with a fresh slate.

After I do that, Time Machine backs up the behemoth once again, and from then again only backs up the changes in the new set of snapshot files.

I now have a good backup of my virtual machines again without having to jump through hoops!

Cheers,

Kris

May 11-15 InDesign Developer Training Sessions

If you’re involved in automation around the Adobe Creative Suite, you need to mark the week of May 11 - 15, 2009 on your calendar - there’s another Creative Suite Developer Summit coming up in Seattle, on the Adobe Fremont Campus

We’re running two training sessions this year - click the lines below for more info:

On May 11: InDesign Plugin Development Workshop - Getting Started With The InDesign SDK

On May 15: Feature Development for InDesign Using ExtendScript

Both workshops are limited to 12 seats, which will be allocated on a first come, first served basis - so don’t delay booking.

More about the Adobe Creative Suite Developer Conference can be found here:

http://niemannross.host.adobe.com/2009csbuDeveloperSummit/

As space is limited to 12 attendees per course, you should make sure you enroll early. There's also a 10% price reduction for people who enroll in both courses before April, 15.

Cheers,

Kris

New Cookbook Blog

I've started a new blog which will be mainly like a 'cookbook' for Adobe Creative Suite scripters - how-to articles that show how to write much more powerful scripts.

http://www.rorohiko.com/wordpress

The first entry in the blog is up - it's about how you can 'attach' little icons or labels to the four sides of page items as a means of feedback to the user of your scripts. The sample script adds a little floating label with the word count to each text frame - so the user can see the amount of words in each text frame in the blink of an eye.

Check it out!

Cheers,

Kris

Using State Machines: Web Access From Adobe InDesign CS3 ExtendScript - click to listen to podcast

Sample files for this podcast can be downloaded from:

http://www.rorohiko.com/podcast/geturl.zip

This podcast will explain how you can query web services from within InDesign CS3 ExtendScript. No need for plug-ins, external libraries - just Adobe InDesign ExtendScript, pure and simple.

>>> Edit#3:

Adjusted the script to handle redirections, by interpreting the HTTP 1.0/301 Moved Permanently return status.

<<<

>>> Edit#2:

Adjusted the script to give much faster downloads in case the Content-Length header is not present in the web server headers. Also changed the protocol to HTTP/1.0 instead of HTTP/1.1 to sidestep the issue of 'chunked' downloads - support for 'chunked' HTTP is left as an exercise.

<<<

>>> Edit#1:
With regards to InDesign CS5: the downloadable code won't work as-is with CS5 due to some oddity with the String.replace function.

To work around the issue, you need to change the ParseURL function to read:

...
function ParseURL(url)
{
url=url.replace(/([a-z]*):\/\/([-\._a-z0-9A-Z]*)(:[0-9]*)?\/?(.*)/,"$1/$2/$3/$4");
url=url.split("/");

// ADD THE LINE BELOW FOR INDESIGN CS5
if (url[2] == "undefined") url[2] = "80";

var parsedURL = 
...

That fixes it up so it works with CS5 too...
<<<

I'll also present some useful routines I wrote, called GetURL() and ParseURL(). GetURL() is a fairly large routine which demonstrates how you can use a programming pattern called 'a state machine' to process data on-the-fly as it is received from a network connection.

To demonstrate how to use the GetURL() function, I've also added a useful sample script. The script will search your active InDesign document for any page items that have a URL as their script label (entered via Window - Automation - Script Label). It will then fetch the data 'behind' the URL and place that data into the page item.

Install the script GetURLs.jsx in the InDesign scripts folder (the easiest is to bring up the scripts palette from InDesign: select Window - Automation - Scripts, and then right-click the User folder).

Select Reveal in Finder or Reveal in Explorer. Then copy the script into the Scripts Panel folder you should now see appear.

Switch back to InDesign, and open up the sample document GetURLSample.indd. Run the script GetURLs.jsx from the palette by double-clicking it on the palette. The empty frames should fill up with images or text (at least, if you are connected to the Internet).

So, how does it all work?

At the heart of it all is the standard ExtendScript object called Socket. More info about the Socket can be found in the JavaScript Tools Guide for CS3:

http://www.adobe.com/devnet/bridge/pdfs/javascript_tools_guide_cs3.pdf

The socket object gives us the ability to perform low-level network communications - we can set up TCP/IP connections with other computers on the network.

The problem is that the Socket object has no higher-level functionality - it has no support for any protocols, like HTTP for example.

To fix that, you could try and use the protocol support that is available via Bridge with the HttpConnection object. You could also forcibly give InDesign access to the webaccesslib that is used by Bridge, through some 'fiddling around'.

However, personally I am not too keen on either of these approaches - they're either a bit too big or too brittle to my liking; I wanted to have an 'InDesign all by itself' solution.

The alternative approach I used was to provide HTTP support in pure ExtendScript to InDesign.

Now, before diving into this: be warned, this is NOT a fully fledged, fully compliant HTTP client. I've only implemented a subset of the protocol, just enough to let me do what I needed to do.

For example, the code only supports UTF-8 text encoding. If your target web server does not offer that you'll have to add some additional code to the scripts to cope with that. Also, I've only implemented HTTP 'GET' requests, not 'POST'. Adding that functionality would be fairly easy to do - it's left as an exercise.

So, the Socket object allows us to send out requests and receive replies via TCP/IP.

The HTTP protocol is quite extensive, but the basis of it is simple - it is mainly a plain text-based, ping-pong protocol. You send out a request, and you get a reply.

The (incoming) reply is composed of three parts: a start (or status) line, zero or more header lines, and then an optional body (which can be binary data or text).

The (outgoing) request is similarly composed of a request line, and zero or more header lines, and an optional body (for POST requests, which I am not implementing here).

Immediately after the start line, you get zero or more header lines.

Header lines are themselves separated from the following body of the request or reply by an empty line - so a request or reply always has the same 'rhythm' to it: start line, header lines, empty line, body.

The GetURL() script code in the GetURLs.jsx implements this in a rudimentary fashion: the request is a simple multi-line text string, which is fired off to the web server via a Socket object.

Then the bulk of the code is for interpreting the reply from the web server - there are three levels of decoding that need to happen.

First of all, we need to decode the reply itself - separate the start/status line from the headers and from the body.

The headers also will contain some important information: the length of the body that will follow. So we need to interpret that header line in order to know exactly how many bytes to read from the socket.

At a lower level, we need to 'chop' the reply up into individual lines until we reach the body - the status line and the individual lines are separated by CRLF character pairs (CR = ASCII character 13, LF = ASCII character 10).

And finally, at the lowest level, while reading a text-based body, we need to interpret UTF-8 code and convert the UTF-8 codes into plain Unicode. This means that we need to read through codes that are 1, 2, 3 or 4 bytes long, and each of them encodes a single Unicode character.

In the GetURL() routine, these three levels of encoding are decoded concurrently, through three nested 'state machines'. Using state machines makes the code fairly fast - much faster than could ever be achieved through string matching.

I won't go into the intricate details of how GetURL() works - the script is fairly well documented and you should be able to figure out how it works by careful reading and stepping through it with the ExtendScript debugger.

Instead, I want to explain a little bit more about state machines - they are a very powerful technique for fast pattern matching and parsing, and once you 'get' them, they are easy to use. They are used in mechanisms like GREP, in compilers and interpreters, in all kinds of text parsers,...

All too often I see code that uses straight string functions to achieve some matching goals.

A simple example: you get some data thrown at you which has line endings that might be either CR (ASCII 13), LF (ASCII 10) or one of each (CRLF).

Many people will handle that by reading in all the data into a buffer, then do some pattern search-and-replace. For example,

first replace all CRLF with CR
then replace all LF with CR

After that, the line ending has become a CR throughout the text.

This approach is not necessarily the best. Especially if you receive the data character by character, you could do the clean-up on-the-fly, as you receive the data. No need to buffer or no global search-and-replaces - so it might greatly reduce the amount of memory you need and also run a lot faster to boot.

With a state machine, you would go about it as follows. First of all, you create a variable (say, myState), and you create some symbolic numerical constants (for example, kNormalState could be a symbolic name for 0, and kSeenCR could be a symbolic name for 1).

For more complex state machines there might be hundreds, even thousands of different states - but in this case, two states will do.

All we'll now do is play with a simple integer variable, and we'll keep track of where we're at by manipulating the state. The idea is that we don't assemble strings or 'memorize' any other input data - we encode the relevant info about 'what has been' into the state variable.

Data flows through our state machine - we read input data, and immediately get rid of the data - we write or store or process it - and we keep as little data as possible inside our state machine logic.

So, our little state machine is happily reading and writing character after character.

After each character we read and process we also check whether it was a CR (ASCII 13) or not, and we change our state to either kNormalState or kSeenCR.

Now, suppose we now read a line feed character (ASCII 10). Before doing anything with a new character, the state machine will always check its current state first.

If the state is kSeenCR we know that this is a line feed after a preceding CR, so we simply don't write the LF out.

If we read a LF and the state is kNormalState, we know that this is a 'stand alone' LF without preceding CR - so we output a CR character to replace it instead.

The state machine is just simple enough to express in words:

Initialize myState to kNormalState
read character (loop until end of file)

if character is LF then
if myState is NOT kSeenCR then
output CR
end if
else
output character
end if

if character is CR then
myState becomes kSeenCR
else
myState becomes kNormalState
end if

end loop

This might seem overkill, but the advantages of state machines become apparent when you try more complex things - for example, interpreting a quote JavaScript string. That string might contain escape-sequences (backslash, followed by a letter, or 1-3 octal digits). Properly interpreting such an 'escaped' string is hard work without a state machine. With a state machine it's a breeze, with hardly any overhead.

So, that was a quick introduction to state machines - I hope it was enough to pique your interest, and entice you to do a bit of research; once you've added them to your arsenal of techniques, you'll find that some difficult tasks have become a lot easier.

You can download the sample files from the following URL (which is mentioned in the podcast transcript):

http://www.rorohiko.com/podcast/geturl.zip

Lightning Brain Podcast: Click here to listen to "How To Get Paid For Writing ExtendScripts For Adobe® InDesign®"

In this podcast, I'll be highlighting a single feature of Rorohiko's Active Page Items family of tools that can help you get paid for ExtendScripts you write for Adobe InDesign CS, CS2 or CS3.

To demonstrate how easy it is, I've first created an ExtendScript called FlontFipper. The script itself, and its name, are a bit tongue-in-cheek, but actually might be useful, who knows?

What the script does is look at the currently active document and determine which two fonts are used most often in the document. It then swaps these fonts.

For example, if your document is quite simple, and has headlines in Helvetica and body text in Times Roman, this script would make the headlines Times Roman, and the body text would become Helvetica.

To use the script, the end-user would install the script into the appropriate folder for InDesign scripts. Then he would open a document, and double-click the script name in the InDesign Scripts panel.

Now assume I would like to sell this script to InDesign users.

First of all, I need to download a copy of the APIDToolkit (Active Page Items Developer Toolkit). A functional, time-limited demo version of APIDToolkit is available from the Rorohiko web site.

This toolkit contains lots of stuff, but the single item I am interested in for this podcast is called the InDesignScriptCompiler. Mac and PC versions are provided in the downloadable archive file.

Second, my potential customer needs to install a special runtime plug-in (the APIDKernel) which will enforce the demo- and licensing restrictions I want to impose on the use of my FlontFipper script.

To protect my FlontFipper.jsx script, I simply drag/drop it onto the InDesignScriptCompiler icon, and fill in some parameters - mainly to tell APID what the limitations are for the demo mode and via which URL a license can be purchased from me.

The result is an encrypted and protected version of my script - FlontFipper.compiled.js - which I can then send to my prospective customer, the end user.

My end user then installs FlontFipper.compiled.js just like any normal script, and uses it just like he would use the non-encrypted original. Note that this is also supported on InDesign CS and CS2.

The main difference with an uncompiled script is that each time InDesign is restarted, the first time FlontFipper is used, a 'beg' dialog will appear, which will inform the end-user of the time-limited nature of the FlontFipper demo. There are also two buttons on the dialog: Get License... and Import License File....

If the end-user clicks Get License... his system's web browser will be directed to display the embedded URL, and it will pass through all the data I would need to create a personalized license file for my end-user.

I then proceed through some monetary transaction to receive the payment from the end-user.

How this transaction is conducted is up to the script developer - for example, I could set up a fully automatic web-based pay-and-license system, or I could simply use a manual system based on e-mail.

APID does not dictate how this transaction should be implemented - this part of the system is left open, and for the script developer to fill in.

Once I receive payment from the end-user I will then use the data I received via the URL to generate a license file, which I then e-mail to the end-user.

The end-user uses the license file to convert the time-limited demo of my script into a fully functional version.

The next time the beg dialog appears, he clicks Import License File... and imports the license file I sent. From then on FlontFipper.compiled.js will not show any more beg dialogs, and also will not lapse.

If you want to try these things out: go to

www.rorohiko.com/flontfipper.html

(all lowercase): you can see the transcript of this podcast with additional screenshots, and you can download the source code, the compiled script, and a license file to test things out.

I'll now dive a bit more into the details of the compilation. After I drag-drop my script onto the script compiler, I get to see a main dialog named APID Standalone Script Compiler.

Here I must define how the compiled script will behave. The most important field is a 'Component ID' string. It has quite a few subfields separated by semicolons and commas and is most easily edited by clicking the 'Edit...' button next to it.

Clicking the Edit... button shows the 'Component ID' dialog. In the 'Component ID' Dialog, I need to define a number of parameters

- a 'component name' which I set to be FlontFipper - this is the name that will appear in various dialogs
- a 'compilation password' which will be used in the encryption of the script - in this sample, I made te password guessWhat
- a copyright string

These first three strings are mainly about identifying your compiled script.

In this same dialog there are also a number of fields to restrict the use of your compiled script - there is a minimum APID version (which I set to 1.0.44 or 1.044 - the current version at the time of this podcast - the InDesignScriptCompiler will drop the leading zeroes automatically and display 1.44).

There is also a hard cut-off date for demo versions. Because I don't want to use such a date in this demo, I've set the three date-subfields (year, month, day) to zero.

The most interesting fields for me are the number of 'actual use' days and number of demo days.

#Demo days represents a number of calendar days since the first use of your script by the end user; 30 days here would mean 'about a month'.

#Actual Use Days represent days of actual use; these days are not necessarily consecutive. This is provided to cater for situations with 'infrequent use' - people that only test the script every so often, and might leave long gaps between uses; you could leave #Demo days set to -1, and #Actual Use Days for example set to 5 and the user would be able to try things out for 5 non-consecutive days.

You can use either, or both fields to limit how long a demo version of your script should be usable.

In this sample, I've set the actual use days to 20, and the calendar days to 30 - the demo will time out after 20 days of actual use, or 30 calendar days, whichever comes first.

There is also a checkbox Free which I'd only use if I don't want to sell my script - for example if I just want to give encrypted versions away without divulging my source code. This checkbox is deselected in my example - I want to sell my script.

On the main screen, I must also enter some messages and a License URL.

The URL is set to link to a web page on my web server, and it passes 4 parameters as part of the URL: the serial number of InDesign, the name of the script (as defined in the Component ID dialog), the system identifier (a unique identifier for the computer requesting a license), and a license level (which will be a letter 'D' or 'R' depending on whether the end-user already has purchased a proper license for APIDKernel or not) - the strings ^1, ^2, ^3, and ^4 are placeholders that are replaced by real data when the URL is needed.

There are also two messages that can appear in the beg dialog. In these messages, ^1 is a placeholder for the compiled script name (as defined in the component ID earlier on), and ^2 is a placeholder for the remaining number of demo days.

Once I click Compile for APIDKernel two things will happen: a compiled version of my script will be created (FlontFipper.compiled.js), and my original source code will also be prefixed with a few comment lines that store my compilation parameters - this to avoid having to re-enter the same stuff next time around.

When a user installs FlontFipper.compiled.js in one of the proper script locations for Adobe InDesign, the script will behave mostly like any other script, except for the beg dialog. Clicking Get License... on the beg dialog makes the end-users' system browser connect to the URL embedded into the compiled script.

Once I receive payment from the end-user, I use a command-line tool that is part of the APIDToolkit to generate a license file for him. Mac and Windows versions of this tool are provided. If so desired, this command-line tool can be embedded into a web server setup - this is how Rorohiko does automated software sales, for example of our Sudoku Generator.

Note that the command line tool is not provided with the demo version of APIDToolkit - you need to purchase the APIDToolkit to get access to this generator - a license for the APIDToolkit costs US$149.00.

For simple set-ups you can also create 'generic' license files which are not linked to any particular system ID or InDesign serial number. The demo license file for this podcast is such a generic license file. It will enable FlontFipper.compiled.js on any copy of InDesign.

Costwise, the APIDKernel runtime for your end-user is not free - there is a one-time cost of US$25 or less per seat.

Note: there are multi-seat APIDKernel bundles available. These should only be used for a single end-user company. For example, you're not meant to purchase a 100-seat APIDKernel and then break these up and sell individual seats to different end-user companies.

Another point worth noting is that APIDKernel licenses are linked to a particular serial number of InDesign, as well as to a unique system identifier. That means that if your customer is using both InDesign CS2 and CS3 on the same computer, he'll need to purchase two licenses for APIDKernel.

Depending on how you set things up, there can be two payments to be made by a first-time end-user. First of all, the end-user needs a license for the APIDKernel, and second, he needs a license for your script.

It is up to you, as a script developer, to decide how to handle the cost for APIDKernel.

You can bundle a pre-purchased coupon code for a license for the kernel when you sell your script, or you can ask your customer to license the kernel directly from Rorohiko. That means you can choose to either have a single monetary transaction between the end-user and you, or there can be two separate transactions
- between the end-user and you for your script
- between the end-user and Rorohiko for APIDKernel

Since version 1.0.44, there is also an option to provide your end-user with a single, combined license file which contains both a license for APIDKernel and a license for your script - contact APIDlicenses@rorohiko.com for more info; for this system to be available to you, you need to register with Rorohiko as an APID developer.

Lastly, if you expect to sell more than a few thousand copies of your script, you should probably consider our APIE/APIR combo - these are two other family members of the Active Page Items family. APIR is a runtime which is very similar to APIDKernel, except that it is free for end-users - there is no cost to the end-user for the APIR runtime.

To generate solutions that can use the free APIR instead of APIDKernel, you'd need to license APIE - which is a more expensive version of APIDToolkit, and can create solutions that work with the free APIR instead of the non-free APIDKernel.

Lightning Brain Podcast: Click here to listen to "Refactoring ExtendScripts"

Today we'll talk about cleaning up scripts, and as a bonus, we'll work on a script that adds a 'hand-written' quality to text: changing something like this:



into



By clicking the link provided in this sentence, you can download this real-life example of a small script being refactored, from experimental form to a cleaner form - click here to download JitterScripts.zip

Listen to or read the podcast transcript for more info...

On to the podcast - click here to listen to it!

So, the script works, everyone is happy - but maybe you are not done yet...

Imagine that you will have to pick that same script up again twelve months from now. How much time will it take you to 'get back up to speed' and rebuild a mental picture of the script and what it is doing? Probably it will take you many hours of browsing, debugging and fiddling around before you regain enough understanding of the script's inner workings to make a 'safe' change.

Now, what if the circumstances have changed, and the script needs to be adjusted to cope with a new environment - it might need to be moved from InDesign to InDesign Server, there might be new requirements...

How about spending some time now - when your head is still filled with knowledge about how it all works - and make sure the script becomes more future-proof.

In this podcast, I'll list a few of the techniques I personally use to 'future-proof' my scripts.

Before diving into the techniques, I need to explain how we approach ExtendScript development here at Rorohiko; we've adapted an approach that delivers very good quality at a reasonable price - we're not cheap, but as opposed to many other custom developments, our solutions work and work well.

When we're developing custom scripts, we use a 'no cure, no pay' approach, for a number of reasons.

The main reason is, that for this type of development, building a sufficiently accurate quote often costs us more than the development itself.

For an accurate quote, we would first and foremost need an accurate, extensive project brief.

But in our business we're often dealing with creative, fairly a-technical people, and we invariably found it very hard, even impossible to zoom in on an accurate enough technical description of the functionality being looked for.

On top of that, what we're asked to do is often at odds with what is really needed. We're often asked to develop a specific solution, rather than being asked to solve a problem.

From experience we've learned that it pays to dig deeper, and try and find out what the underlying problem is - quite often the asked-for 'solution' only cures a symptom, and leaves the underlying problem unfixed.

The most efficient way we found to get the technical information we need from creative people is to use an iterative approach. Instead nagging people and trying to wring a technical brief out of them, we put the cart before the horse instead, and we create something, anything, as good as we can, based on the still limited understanding we have of the problem at hand.

We present our customer with the attempted solution, and get their feedback - it is much easier for them to explain what is wrong or what is missing from some tangible, real software, rather than trying to come up with blueprint.

Based on the feedback, we adjust the solution (or throw it out and start over), and we go through a few iterations, until we have the thing sussed.

Eventually, we reach a good, smooth solution, and by that time we also know exactly what the cost of that solution is.

At that point, the 'no cure, no pay' system kicks in: our customer can choose to purchase and continue to use the software, or alternatively, in case our solution were to not live up to its promise, the software is simply destroyed, and there is no cost to the customer.

This approach works really well, but the successive iterations cause the software to go through a few swings and roundabouts, and along the way, grow a whole collection of warts if we're not careful.

We'll typically spend some time refactoring the scripts to make sure they're future-proof and self-explanatory - making a small investment of time now in return for a substantial time-saving later.

Here are some of the things we do:

1) Don't rely on app.activeDocument

While in the heat of experimentation, iteration and script development, it's easy to assume that the functionality being created will be applied to the current document, and hence to refer to app.activeDocument

However, there are sizable benefits to removing the reliance on the active document. Our scripts will typically contain a number of functions, and whenever a function uses app.activeDocument, we rework that function to not use 'app.activeDocument', but instead take a 'document' parameter.

The idea is that you get hold of the document 'under consideration' in one single spot in the script, and that from then on you pass a document parameter to any function that is supposed to work on that document.

The two biggest advantages are:

First, your script becomes a lot easier to convert to an InDesign Server environment (where there is no such thing as app.activeDocument), and second, all of your functions have now suddenly become much more re-usable: they can now also be used when the document to be modified is not the active document.

For example, the document might be a temporary, invisible document you're opening in the background - and by NOT using app.activeDocument, you can pass such a document to your functions as a parameter.

2) Test, test, test your preconditions

When using a function, it pays to add tests for preconditions - make sure all parameters being passed as what they are supposed to be, and display an error message if they are not. Whenever possible, we leave all this testing code in the script - so if something goes wrong at the customer's end we get good, specific information about where things go off the rails.

Typically, we'll have a global constant - something like kDebugging - which can be set to true or false to indicate debugging more.

We'll also add a messaging function similar to alert() which can display a dialog box with a message. The difference with alert() is that the dialog box is conditional on kDebugging being set to true. If kDebugging is set to false, the message is ignored.

And then we'll test all the function parameters being passed into a function. Is the document non-null? Is it instanceof Document? Is the percentage a number between 0 and 100? If any of these tests fail, a debug message is emitted, and the function 'bails out'. This guarantees that any unexpected condition can be caught early on.

This works well by wrapping most of the function body inside a do{}while(false) construct, which mis-uses the do-while loop to build a construct that allows a 'ladder-like' function construction.

Inside the do{}while(false) there is a whole series of if tests which verify if all is well, and display a debug message followed by a 'break' statement if not. The break causes the function to 'fall off' the ladder for any failing precondition. The debug message being displayed is specific enough to pinpoint the spot where things went wrong - it will include the name of the function where the problem occurs, and a short description of what is wrong.

This construct is quite similar to using try-catch, but it is 'cheaper' in a number of respects; it causes less overhead than using try-catch, and does not cause the InDesign Debug version to emit assert statements during script execution.

3) Do not spend time optimizing unless it is really necessary.

Now, you'd think that all that debug code from the previous point must cause a lot of overhead.

Well, turns out that is not true most of the time - a typical script will spend 95% of its time in 5% of the script, and all that debug code has very little impact on the script execution time.

In practice, we'll leave all our debug code in the script - all we might do is to set kDebugging (or whatever the constant is called) to false - but even that we often don't do: it's better to be informed of unexpected circumstances, than to have a script silently and mysteriously fail.

Only when there are speed issues might we consider removing some debugging code - but only if we can clearly see that this code is part of the bottleneck.

The current ExtendScript Toolkit contains a nice profiling tool that allows you to see where a script is spending most of its time. Our recommendation is to not bother with optimizing unless there is a time issue, and when optimizing, use proper profiling to solve the bottleneck - but nothing else. Any debug code that you can leave alone should be left alone; it's part of your safety net.

It is very common for our scripts to have 50% or more debugging/testing code in them.

4) Avoid global variables

While experimenting, it is very common and easy to introduce some global variables to keep track of things.

However, global variables can be a recipe for disaster - especially when you need to revisit an older script and make some modifications to it.

Global variables represent a form of communication between different areas of your script - functions can communicate with one another by stuffing data into global variables, and getting it back out again.

Problem is: that type of interaction is very easy to get overlooked, and causes all kinds of unexpected side effects - for example, you add an extra call to a particular function somewhere, the function changes a value of some global variable, and then other functions that rely on that same variable go off the rails.

Because functions don't clearly 'advertise' what global data they consult or modify, it becomes very hard to keep track of interactions. That makes for fun debug sessions, chasing weird bugs after making a 'tiny change' to a year-old script.

Like everyone else, during the initial phase of a project, we often start out stuffing data into globals - but unless there is good reason to, we'll rework the script and move the global variables into function parameters. If there is a lot of data, we'll introduce one or more data structures which are then passed around as a parameter.

An example: we might be parsing a text string, and keep track of where we're at in a global variable gTextPos, and store the string in a global gParseText.

During cleanup, that will be reworked (or 'refactored' as it is often called) - we'll get rid of the globals, and instead we'll put the current 'parse state' into a JavaScript object with at least two attributes: parseText and textPos.

Then we pass that object to the relevant routines using a parameter - say 'parseState'.

This way it becomes immediately clear to the human reader of the script which routines access that data (they need the parameter) and which ones don't access that data (they don't need the parameter) - it's a self-enforcing cleanup. From this moment on, each function that needs access to that data does 'advertise' the fact via its parameter list.

Imagine every JavaScript function as a gob of code floating in space. Then imagine what outside factors influence the function's operation and how, in return, the function influences its environment. There are the parameters coming in at the top, the return value coming out at the bottom. Most of the time these two relations are pretty easy to see.

Then there are any global variables that are modified or consulted by the function - using globals leaves a lot of room for unseen interaction between the function and its environment. Things like app.something and $.something are also globals - they are provided by InDesign, but they are still globals.

The more 'isolated' you can make your functions, the easier it will be to re-use in a different script.

Functions that interact with global data are like a beating heart - very difficult to transplant because there is a lot of stuff to disconnect and reconnect.

Functions that take data via their parameters, and return data via their return value and/or via some of their parameters are much easier to transplant: a few easy connections to their environment; they easily snap in and out.

5) Each function should do one thing well

We always try to create functions that do one thing well; during the 'frantic' phase of a project we often end up with functions that do lots of stuff. These multi-headed monsters need to be divvied up into smaller functions - each doing just one thing. Functions that are initially called something like 'ImportFileAndColorFramesAndDeleteOverrun' are split up into multiple smaller functions.

This increases the chances of making things re-usable - any 'good' function eventually ends up in our growing function library and will be reused, which cuts down our development time on future projects. Multi-headed monsters are never reusable - so cutting them up has distinct advantages.

6) Name constants and move them to the header of the file for easier customization

During the trial and error phase, you'll typically add all kinds of literal constants to the code - it is worthwhile to try and isolate these constants from the code and move them to a seaparate section near the top.

This makes the script easier to adjust, and it also makes it more robust.

Now, if a certain string constant is used twice in the script, there seems to be little advantage to creating a symbolic constant for the string and then use the constant instead of the literal string. Many people think this is a pedantic use of constants - on cursory inspection, the two approaches look not all that different.

However, the advantage is that with a symbolic constant, typing errors can be immediately caught by the computer, whereas with literal strings the computer would not know that these two strings are supposed to be equal.

So, if you'd type two literal strings "TextFrame" and "TextFrome", the computer would accept that - but if you typed two symbolic constants kTextFrame and kTextFrome, the second one would be undefined and cause an error.

By clicking the link provided in this sentence, you can download a real-life example of a small script being refactored, from experimental form to a cleaner form - click here to download JitterScripts.zip