Reading: An Introduction to Programming with IDL, Chapters 1 through 4
INTRODUCTION
· IDL stands for Interactive Data Language
· IDL can be run in either interpreted mode or compiled mode.
o In Interpreted mode each line of the source code text file is read and executed as it is encountered.
o In compiled mode the source code text file is compiled into an object file that is then executed.
o Below are some advantages and disadvantages for IDL versus Fortran
|
|
IDL |
Fortran |
|
Advantages |
· Interactive or compiled modes. · Built in graphics and plotting capability. · Excellent for image analysis and manipulation. · Many built in mathematical functions (FFT, statistics, matrices) available without linking to external libraries. · Can easily build graphical user interfaces (GUI) to view data sets. · Includes object-oriented programming (OOP) features. |
· Compiled mode only. Cannot be run interactively. · Compiler is optimized to execute quickly. · Compiler optimized for large-scale parallel processing. · Open source, freely available.
|
|
|
|
|
|
Disadvantages |
· Proprietary – Must purchase expensive license. · Compiler not optimized for quick execution (wouldn’t be a good choice for a large computational atmospheric or ocean model.) |
· No built in graphics capability. · Not interactive. · Many consider it an almost “dead” language (though it is going strong for large-scale, massively parallel scientific calculations.) |
· Though there are many programming tasks that can be accomplished by both Fortran and IDL, they each have their place. For example:
o If I want to program a complex numerical weather prediction model I would choose Fortran, because a lot of time, money, and effort has been put into the compilers for Fortran to optimize them for high-speed, large-scale parallel computing.
o If I want to write a quick program to view a graph of some function, or to perform mathematical calculations on a large data set I would choose IDL.
· In my own mind, the single largest drawback to IDL is the fact that it is proprietary, and requires an expensive license (though student licenses are available at a reduced rate.) Thus, unless your employer or university has an IDL license, you’re screwed.
· Reasons to learn IDL:
o It is a very easy and quick way to generate plots and graphs, and to do numerical calculations.
o It is that it is widely used by NASA and many universities. A knowledge of IDL may help you get a job, and can also make your life in graduate school easier.
o Even if you get a job or attend a university that doesn’t have an IDL license, the programming techniques and experience you gain with IDL will make it easier to learn other languages and graphical visualization packages.
o It can be fun! Yes, this is true…trust me!
MY PERSONAL OPINION ON COMPUTER LANGUAGES
· Fortran – I have used Fortran extensively, primarily to program numerical weather prediction models, which is its strength. Most people outside of the atmospheric and oceanic sciences consider Fortran obsolete, though it is still widely used for numerical weather prediction models owing to the large amount of time, effort, and money put into optimizing compilers for large-scale, parallel processing. The only thing for which I have used Fortran in the past 7 years or so is when teaching my course in numerical modeling, and am finding I could even replace Fortran with IDL in that course if need be (since model run time isn’t really an issue for the small-scale models used in that course).
· IDL – I began using IDL in 1996, when I started working on my Ph.D. I used it primarily for data visualization, and it wasn’t until later that I realized it also had great computational and analysis capabilities. I now use it exclusively whenever I need to do a calculation or plot a graph. I’ve been fortunate that everywhere I’ve been since 1996 I’ve had an IDL license available to me. Even in my numerical modeling course I could do everything that needs to be done using IDL, and am minimizing my use of Fortran.
· Ruby – In 2006 I decided that I wanted to learn an object-oriented language. I considered C++, Perl, Java, Visual Basic, etc., but happened to chance upon Ruby. It has turned out to be extremely easy and fun to learn, and very capable. I’ve used it to build GUI applications, calculate and manipulate my grade sheets, send automated e-mail messages to students with their grades, and many other tasks. It is so fun that I look for excuses to program in Ruby.
· I now pretty much use IDL and Ruby exclusively, and though I am sad to say it, I could live quite nicely without Fortran.
A NOTE ON THE INTERACTIVE DEVELOPMENT ENVIRONMENT
· IDL comes with an interactive development environment. I generally avoid interactive development environments, preferring to run my programs the way God intended…from the command line.
o I do use the development environment to compile and run IDL programs on Windows machines, but only because this is the only way to compile and run them in the Windows environment.
· You are free to experiment with and use the development environment for your projects, but its use will not be covered in the course.
DOCUMENTATION AND BOOKS
· IDL has an extensive on-line documentation and help system, which is accessed by entering a question mark “?” at the IDL prompt on the command line.
· There are also some books, in addition to our class textbook, that you may want to look into if you want to continue learning about IDL and its capabilities. A list of IDL books can be found at http://www.ittvis.com/training/books.asp (Note: There are other places to purchase these books, and I am note endorsing this site. It just has a convenient list of IDL books on the market.)
SETTING UP IDL
· IDL is already installed on all of our departments Linux computers, so you don’t need to do any sort of set-up in advance. However, there are some nice things you can do to make IDL easier and friendlier, and so I encourage you to follow the instructions in Section 3.2.2 of the textbook. When you are finished you should have:
o A directory called “idl” that is underneath your home directory.
o A subdirectory called “bowman” that contains the files you downloaded from http://idl.tamu.edu.
o A copy of the “startup.pro” file in both your “idl” directory and the “bowman” subdirectory.
· Referring to Section 3.2.3, you then need to edit the “.tcshrc” file that is in your home directory to include the following line:
setenv IDL_STARTUP ~/idl/startup.pro
· The “startup.pro” sets certain compiler options, and also allows you to define some system variables to be used as constants. The “startup.pro” file that you copied has some nice constants in it already. Feel free to add more. I also recommend making the following modifications (just my personal preference):
o Change the value of the Universal Gas Constant to 8.3144720D0, and change the “kmol” to “mol”.
o Change the value of Avogadro’s number to 6.02214199D+23, and change the “kmol” to “mol”.
A NOTE ON THE COMPILE_OPT IDL2 COMMAND
· The first command in the “startup.pro” file that you copied is COMPILE_OPT IDL2, which requires square brackets rather than parentheses for referencing array elements, and also uses 4-Byte (32-bit) integers by default. Although I agree with the textbook that this is a good idea in theory, in practicality it may come back to haunt you. This is because older versions of IDL used parenthesis for array referencing, and there is a lot of older code out there that still has parenthesis. If you try to run this older code using the COMPILE_OPT IDL2 option it will give you an error.
· I RECOMMEND replacing the “COMPILE_OPT IDL2” statement in the startup.pro file with “COMPILE_OPT DEFINT32”.
o This still sets 4-Byte integers by default, but will allow the use of parentheses OR square brackets for array indexes.
· For any new coded you write during this class I will expect you to use square brackets for array referencing.
STARTING, INTERRUPTING, AND EXITING IDL
· To start IDL, type idl from the command line.
· To exit IDL, type exit from the command line.
· To activate the online help, type ? from the command line.
· To suspend execution of a program, script, or calculation use the keystroke combination <Ctrl>-c.
· To continue program, script, or calculation type .c or .continue.
SOME OTHER USEFUL COMMANDS
· While in IDL you can execute Linux shell commands by preceding them with a “$” character.
o For example, you could get a listing all the IDL source files in the current working directory by typing $ls *.pro.
· The CD, new_directory command can be used to change the working directory within IDL.
· The “$” character is also used as a continuation line character (similar to the “&” used in Fortran for this purpose).
· Multiple IDL statements can be typed on the same line using the character & as a separator.
o NOTE: Putting multiple IDL statements on a single line is not good programming practice, and should only be used if necessary. The only time I can see this as being necessary would be to execute a loop or if-then-else block from the command line or in a script.
· It is sometimes useful to reset an IDL session. This is done using the .reset_session or .reset commands.
o Resetting clears all variable definitions, and gets rid of any compiled programs, etc. It is basically like starting from scratch, without having to exit and then restart IDL.
DATA TYPES
· IDL has several different numeric data types.
o The number of bits used for storage dictates the range of numbers that can be stored.
o Unsigned integers are positive values only, whereas signed integers can be positive or negative.
· The table below lists the numeric data types available, along with how many bits are used for storage and the range of numbers available (after Gumley, Practical IDL Programming).
|
Data type |
Remarks |
Bits |
Range of numbers |
|
byte |
Unsigned integer |
8 |
0 to 255 |
|
int |
Signed short integer |
16 |
-32,768 to 32,767 |
|
uint |
Unsigned short integer |
16 |
0 to 65,535 |
|
long |
Signed long integer |
32 |
-231 to 231 - 1 |
|
ulong |
Unsigned long integer |
32 |
0 to 232 - 1 |
|
long64 |
Signed very-long integer |
64 |
-263 to 263 - 1 |
|
ulong64 |
Unsigned very-long integer |
64 |
0 to 264 - 1 |
|
float |
Single-precision floating-point |
32 |
-1038 to 1038 |
|
double |
Double-precision floating-point |
64 |
-10308 to 10308 |
|
complex |
Complex (real-imaginary) pair |
64 |
-1038 to 1038 (each member) |
|
dcomplex |
Double-precision complex |
128 |
-10308 to 10308 (each member) |
· There are also 4 non-numeric data types shown below (again, after Gumley).
|
Data type |
Remarks |
|
string |
Character data (up to 32,767 characters) |
|
struct |
Container for variables |
|
pointer |
Reference to a variable |
|
objref |
Reference to an object structure |
· Unlike with Fortran, you don’t have to declare variable types ahead of time.
o IDL uses dynamic typing, which means that the variable type is determined by its input. The same variable name can be used as an integer at one point, and then if a string is assigned to it, it then becomes a string or character variable.
· You can always see what type a variable is by typing help, variable_name at the command line.
ARRAYS
· Arrays can be created from any of the variable types, but they must be declared ahead of time.
· Arrays can have one or more dimension.
· The size of the array must be declared when creating an array.
· If the COMPILE_OPT IDL2 command is used in your “startup.pro” file, then array elements must be referenced using square brackets (e.g., x[2]). Otherwise, either parentheses or square brackets will work.
o It is recommended to always use square brackets for referencing array elements, and that is what I will expect to see in all your assignments.
· Array indices must begin with 0, and must be positive.
o This is different than Fortran, so be ware!
· There are some nice built-in functions for creating arrays of evenly spaced numbers. For example, the command x = findgen(10) will create an array of floating-point number from 0 to 9.
· Numerical arrays can be multiplied by scalars, and the multiplication will be applied to every element of the array.
PLOTS
· Generating plots in IDL is very simple.
o If you want a plot of the elements of an array called x, just type the command plot, x and the plot will appear on the screen.
o If you want to plot one array named x against another array named y, the command is simply plot, x, y.
· Note that the comma is required after the plot procedure, as it is for all IDL procedures.
· IDL will also plot surfaces and 3-D images, which we will learn about later.
SCRIPTS AND JOURNALING
· If you have a series of IDL commands that you need to routinely execute you can save them in a file with a “.pro” suffix. You can then execute these commands by typing @scriptname from the IDL prompt.
o When executing a script the IDL interpreter is executing the commands in the script line-by-line, without any pre-compiling.
· Scripts differ from procedures and functions, about which we will learn more later. Important limitations to scripts are:
o You cannot pass arguments to a script.
o Scripts cannot execute blocks of code such as loops or “if-then-else” blocks.
· Scripts can be created using a text editor, or through “journaling.”
· By typing journal, filename any commands typed at the IDL prompt will be saved to the file named filename until another journal command is encountered.
· The file created can then be run as a script and can also be edited if need be.
TYPES OF IDL PROGRAMS
· Besides using IDL from the command line or executing scripts, IDL allows the ability to write program modules that can be compiled and executed.
· There are three types of IDL program modules:
· Main programs
o Main programs can be thought of as equivalent to main programs written in other languages such as Fortran.
o Main programs must always end with the END statement.
o Main programs can call procedures and functions
o Main programs are compiled and executed from the command line using the .run filename or .r filename command.
o To compile a main program, but not run it, use the .compile filename command.
o If a main program has already been compiled, it can be executed from the command line using the .go or .g command (without specifying a filename).
· Procedures
o An IDL procedure is equivalent to what would be called a subroutine in Fortran.
o A procedure can return more than one argument, or none at all.
o Procedures can be called from a main program or other procedures.
o A procedure is called with its name and then a comma-separated list of arguments.
§ plot, x, y, title = “Nice Plot”
o In order to be a procedure the text file must begin with a PRO statement and end with an END statement.
o In addition to being called from main programs or other procedures, a procedure may also be executed from the command line by just typing its name (and any arguments needed).
· Functions
o Functions in IDL behave just like functions in Fortran.
o A function returns only a single argument.
o A function is called as part of a variable assignment, and has the arguments in parenthesis, with comma-separation.
§ a = FINDGEN(4, 6)
o In order to be a function the text file must begin with a FUNCTION statement and end with an END statement.
o As with a procedure, a function can be called from a main program or a procedure, or can be executed directly from the command line.
PROGRAMMING STYLE
· Read the IDL Style Guide printed in Appendix A of the textbook. Although we won’t follow this style guide exactly, it does have some good points.
· Here are some style issues that are very important, as you will lose points if you don’t follow them:
o ALWAYS USE SQUARE BRACKETS FOR ARRAY INDEXES!
o The final END statement of any program or module must be capitalized.
o The PRO or FUNCTION statement at the beginning of a module should be capitalized.
o Any function calls should be capitalized!
· Capitalization
o IDL is case insensitive.
o When using IDL interactively I never bother with capitalization.
o When writing programs, however, I do bother.
o The style guide suggests that you use upper-case for procedure and function names, and IDL reserved words such as “PRINT”, “FOR”, etc.
o There is good reason for doing this, as it looks nice and makes your code easier to read. I would appreciate if you do this as it makes your code easy to read. However, you don’t have to, because I don’t.
§ I don’t do this because many other programming languages are case-sensitive, and predominantly use lower-case letters. For instance, I write a lot of Ruby code, and in Ruby words like print, for, etc. must be lower-case. In order to make my transition easier between the two languages, I generally use lower case in both.
o As mentioned above, I do expect you to capitalize PRO and FUNCTION statements at the beginning of modules, and the terminal END statement in any program or module.
o I also expect you to capitalize and calls to functions (but not procedures).
o Whatever you choose to do, BE CONSISTENT! Don’t capitalize these words in some places and not in others.
· Indenting – Blocks of code within if-then-else statements, for loops, etc. should be indented.
o Although the textbook’s style guide recommends a 3-space indent, you are free to choose any indent between 2 and 5 spaces (I personally use 2 spaces, because that’s fairly standard in the Ruby community).
o Whatever indent you choose, BE CONSISTENT throughout your programs. Do not use 2 spaces in some spots and 5 spaces in others.
· Comments – I expect to see liberal use of comment lines in your code.
o At the beginning of any program file (but after any PRO or FUNCTION statements) I want you to include your full name, the assignment number, and the date.
o I want every for loop, if-then-else statement, etc. to have comment lines explaining what it is for.
o I expect every variable, when it is first introduced, to include a comment line explaining what it is used for, INCLUDING its UNITS!