The IDL Path Problem from Hell

QUESTION: OK, I don't have a friggin' clue how the IDL 7 Workbench is managing my IDL path information. Does anyone have the foggiest idea?

ANSWER: Oh, you poor soul, come here. Calm down. Take a few deep breaths. No matter how bad it is, it can't be worse than my own journey along the IDL path. Let me tell you about it and how I found my way home, eventually. Here, take these. You don't want to know, just take them. I found them in the medicine cabinet. They were probably Coyote's. They'll do you some good.

OK, it was Spring. The daffodils were just coming up, although it was still chilly outside....

The Synopsis

The real story is still too painful to recall, so let me just give you the synopsis. I was trying to give the IDL Workbench the benefit of the doubt and I was allowing it to control the IDL paths as I opened and closed projects in the IDL workspace. But I was getting increasingly frustrated. Sometimes programs would run correctly and sometimes they wouldn't. I got to the point where I wasn't even sure which files I had open in the editor. But the straw that broke the camel's back was my frustration trying to get a set of files organized to send to a client. I couldn't figure out which files needed to be in the distribution and which didn't because sometimes it would get a file from one project and sometimes it would get it from another. Sometimes (and this is the nature of supporting software from many clients) the files were named the same, but were different internally. It was a nightmare. I had to get a better handle on this.

The first thing I tried was right clicking on each of my projects in the Project Explorer, selecting their Properties sheet, and then changing their IDL Project Properties. I unchecked the "Update IDL Path Preference" button for each project. I would manage the paths for the projects myself.

Imagine my surprise, then, to find that when I started IDL up after saving all the changes I made that two of my projects were still on my path. I re-checked my work, confirmed all the changes I made previously, restarted IDL several times, and the projects were still on the path!

OK, I then deleted all the projects, every single one of them, from within the Project Explorer. Of course, I didn't delete the files, just the projects. Started IDL up and the two projects were still there.

OK, I moved all the IDL files from the projects I just deleted out of the IDL Workspace folder. Now there is nothing that IDL could be getting information from, so I should be go to go. But, no, the two projects were still on my path. In a frenzy, I clicked every Restore Defaults button on every preference sheet I could find in the entire IDL Workbench. Months of customizations were completely wiped out. (Probably a good thing, but more on this later.) The projects were still on my path.

The hell! I then created an entirely new IDL workspace, far away from the one I have been using. The projects were still on my path!

I was in a bit of a panic now, and was starting to question my sanity, when Ceasar Ordonez reported that he had experienced a similar problem the week before, and had found that the IDL Workbench keeps a "secret" IDL preference file. And that the IDL path that was stored in this secret file was apparently copied over from his IDL 6.4 installation when he installed IDL 7. Sure enough, I found this secret file here:

   C:\Documents and Settings\David\.idl\itt\pref-10-idl_7_0-windows\idl.pref

And, sure enough, inside this file is where the two projects were being added to my IDL path. I removed the offending projects here, restarted IDL, and had, finally, what I considered a clear path to start from. I carefully put everything back together (a couple of hours, it seemed like), taking care that when I added projects this time that they did not modify the IDL path when they were opened or closed. Finally, I was ready to start managing the paths myself.

By the way, the equivalent UNIX file is found in your home directory and has the same name. That is, it starts in the .idl directory, which is found in your home directory.

Managing the IDL Path Myself

Reasoning, correctly I think, that allowing IDL anywhere near my paths was risky business and could only result in disappointment, I decided that I would simply control my IDL path with my IDL start-up file. I created a start-up file, and pointed to it by setting the Startup File preference in the IDL Preferences dialog. I put this code into it:

    !PATH = 'G:\IDL\Coyote;' + $
           'G:\IDL\CoyotePlus;' + $            'G:\IDL\Utilities;' + $
           Expand_Path('+G:\IDL\JHUAPL') + Path_Sep(/SEARCH_PATH) + $
           Expand_Path('+G:\IDL\Astronomy') + Path_Sep(/SEARCH_PATH) + $
           !PATH 

I also had some other things and a Print statement, so I could tell the code in the start-up file was executed, in the file.

I started IDL up, and my start-up file executed. Oh, joy! But wait, I can't run any programs. IDL can't find them. I check the IDL path. My directories are not on the path. Say what!? I check to see the other things in the start-up file have been done. They have. I run the start-up file from the IDL command line. It runs perfectly. I check the path. My directories are there!

I must have made a mistake. I exit IDL and try it again. Same result. My start-up file is good, but somehow between the time it runs and the microsecond later when I check my path, the directories I added to the path have been deleted. Aaauughhhh! How the #*%@& does one change a path in this piece of s..., this IDLDE anyway?

I'll just give you the bottom line: not the way I was doing it.

If you want to set your path properly, you can use one of the following three methods. First, change your IDL path from the IDL command line once you get into IDL. You can use code like I tried to use above. This method fails if you try to do this from a start-up file, at least if you try this on a Windows machine. It appears to work fine if run in a start-up file on a UNIX machine, although I still wouldn't recommend it as a method for version 7 of IDL.

Second, if you want to change your IDL path from a start-up file, you must use the Pref_Set command to do it. Your code might look like this:

    aPath = 'G:\IDL\Coyote;' + $            'G:\IDL\CoyotePlus;' + $
           'G:\IDL\Utilities;' + $
           Expand_Path('+G:\IDL\JHUAPL') + Path_Sep(/SEARCH_PATH) + $
           Expand_Path('+G:\IDL\Astronomy') + Path_Sep(/SEARCH_PATH) + $
           !PATH   Pref_Set, 'IDL_PATH', aPath, /COMMIT 

Third, you can use the IDL Path preferences in Window -> Preferences -> IDL -> Paths to add your directories to the IDL path, as shown in the figure below. If you have projects that manipulate the IDL path, those project directories will be added below the ones on this list in the path order. They will never be ordered ahead of these. Another way of saying this is that the directories specified as on the path here (or which are set with Pref_Set) constitute the "default" directory path. Any changes to the path after IDL starts up, start with this as the default path. This default directory is stored in the secret IDL preferences file referenced above.

You should choose either method 2 or method 3, if you think method 1 is as lame as I think it is. Don't try to do both! Two or three. Not both. Settle on one or the other, or your life will remain as difficult as it is now. I chose method 3 and life seems to be good for the moment.

Setting the IDL paths from the Preferences dialog.

 

Managing the IDL Path the Mike Galloy Way

Mike Galloy has also struggled with how to manage the IDL path with IDL 7. He has come up with another method that he recommends and describes on his web page.

Version of IDL used to prepare this article: IDL 7.0.1.