| Spatial and temporal image correlation [message #85543] |
Thu, 15 August 2013 09:19  |
Cornelio Zolin
Messages: 5
Registered: August 2013
|
Junior Member |
|
|
Dear all,
Hi all,
I have just started using IDL and I’m having hard times even with simple tasks. I’m trying to do the following task:
I have 12 folders (2001, 2002 ….2012) and in each folder I have 35 files .dat of anomalies (7 months NDVI, 7 months Surface temperature and so on) and I would like to do a correlation (pixel by pixel) for each index in each month for each year.
After that I want to make one plot per year: each index in a different column and each row representing a different month.
I started writing the code bellow, but I’m confused now. I think the next step would be REFORM the images, so for each image I will have a vector that I could use to do the correlation.
Is there anyone that could help me on that?
Thanks a lot,
PRO ImgCorr,
CD, 'C:\Anomalies\year'
theseFiles = FindFile('*.dat', Count=numFiles)
Print, 'Number of files found: ', numFiles
FOR j=0,numFiles-1 DO BEGIN
OpenR, lun, theseFiles(j), /Get_Lun
File = fltarr(620, 500)
ReadU, lun, File
JustNumbers= where(File eq -9999, count)
File[JustNumbers]=!VALUES.F_NAN
*
*
*
*
Free_Lun, lun
ENDFOR
END
|
|
|
|
| Re: Spatial and temporal image correlation [message #85546 is a reply to message #85543] |
Thu, 15 August 2013 10:48  |
Andy Sayer
Messages: 127
Registered: February 2009
|
Senior Member |
|
|
Looks like a good start!
I use file_search instead of findfile and believe file_search is considered preferable for some reason, although forget what it actually is. Note you don't need the CD command and could do e.g. thesefiles=file_search('C:\anomalies\year\*.dat',count=numfi les) instead.
Are you trying to do a spatial correlation or a temporal correlation? If spatial (e.g. correlate NDVI and SST anomalies for June 2003) then you can pass the 2D array containing your data to correlate(), you don't need to reform() it to a 1D array first.
You will also want to modify your loop a bit. As written, the variable 'file' will be redefined and overwritten each time and the contents lost. So for example you might want to define ndvi=fltarr(nyears,nmonths,620,500) and then loop over years and months to populate the appropriate part of the array (and ditto for SST and your other variables). Also, (I think) you'll want to close,lun before free_lun,lun .
Hope this helps,
Andy
On Thursday, August 15, 2013 12:19:01 PM UTC-4, Cornelio Zolin wrote:
> Dear all,
>
> Hi all,
>
> I have just started using IDL and I’m having hard times even with simple tasks. I’m trying to do the following task:
>
>
>
> I have 12 folders (2001, 2002 ….2012) and in each folder I have 35 files .dat of anomalies (7 months NDVI, 7 months Surface temperature and so on) and I would like to do a correlation (pixel by pixel) for each index in each month for each year.
>
>
>
> After that I want to make one plot per year: each index in a different column and each row representing a different month.
>
>
>
> I started writing the code bellow, but I’m confused now. I think the next step would be REFORM the images, so for each image I will have a vector that I could use to do the correlation.
>
> Is there anyone that could help me on that?
>
>
>
> Thanks a lot,
>
>
>
>
>
> PRO ImgCorr,
>
> CD, 'C:\Anomalies\year'
>
> theseFiles = FindFile('*.dat', Count=numFiles)
>
> Print, 'Number of files found: ', numFiles
>
> FOR j=0,numFiles-1 DO BEGIN
>
> OpenR, lun, theseFiles(j), /Get_Lun
>
> File = fltarr(620, 500)
>
> ReadU, lun, File
>
> JustNumbers= where(File eq -9999, count)
>
> File[JustNumbers]=!VALUES.F_NAN
>
> *
>
> *
>
> *
>
> *
>
> Free_Lun, lun
>
> ENDFOR
>
> END
|
|
|
|
|
|
|
|
| Re: Spatial and temporal image correlation [message #85550 is a reply to message #85543] |
Thu, 15 August 2013 11:16 |
Cornelio Zolin
Messages: 5
Registered: August 2013
|
Junior Member |
|
|
On Thursday, August 15, 2013 12:19:01 PM UTC-4, Cornelio Zolin wrote:
> Dear all,
>
> Hi all,
>
> I have just started using IDL and I’m having hard times even with simple tasks. I’m trying to do the following task:
>
>
>
> I have 12 folders (2001, 2002 ….2012) and in each folder I have 35 files .dat of anomalies (7 months NDVI, 7 months Surface temperature and so on) and I would like to do a correlation (pixel by pixel) for each index in each month for each year.
>
>
>
> After that I want to make one plot per year: each index in a different column and each row representing a different month.
>
>
>
> I started writing the code bellow, but I’m confused now. I think the next step would be REFORM the images, so for each image I will have a vector that I could use to do the correlation.
>
> Is there anyone that could help me on that?
>
>
>
> Thanks a lot,
>
>
>
>
>
> PRO ImgCorr,
>
> CD, 'C:\Anomalies\year'
>
> theseFiles = FindFile('*.dat', Count=numFiles)
>
> Print, 'Number of files found: ', numFiles
>
> FOR j=0,numFiles-1 DO BEGIN
>
> OpenR, lun, theseFiles(j), /Get_Lun
>
> File = fltarr(620, 500)
>
> ReadU, lun, File
>
> JustNumbers= where(File eq -9999, count)
>
> File[JustNumbers]=!VALUES.F_NAN
>
> *
>
> *
>
> *
>
> *
>
> Free_Lun, lun
>
> ENDFOR
>
> END
Thank you folks,
I'll try to move forward using your suggestions.
Zolin
|
|
|
|
|
|
| Re: Spatial and temporal image correlation [message #85554 is a reply to message #85543] |
Thu, 15 August 2013 12:44 |
Phillip Bitzer
Messages: 223
Registered: June 2006
|
Senior Member |
|
|
Besides the aforementioned use of file_search and not using CD, let me add the following about your loop:
1) Generally speaking, I like to open the file I need, read it in, and free the lun for file immediately before doing any processing. This especially goes while you are debugging your code. The reason is so you don't have any "hanging" luns that aren't freed properly.
2) In addition, you are making file a float array for the data in the file. Is that the case, or is it integers? If it *is* floats, it's generally not a good idea to check for equality between floats (see http://www.idlcoyote.com/code_tips/comparearray.html for a discussion). I'll assume below -9999 is your missing value, and all good data is definitely greater than -9990.0. This may not really affect you here, but at some point checking if two floats are equal will get you in trouble.
3) You should use square brackets to access array elements (I see you do that once, so it may be a typo in the OPENR call). Although not strictly required, this will help you down the line and help you differentiate between arrays and functions. The use of the compiler option idl2 (see http://www.exelisvis.com/docs/COMPILE_OPT.html) is highly recommended by several notable IDL programmers.
4) Be sure to check the count keyword before assigned the null values to NaN. In IDL 7 and below, if there are no null values, the code as you've written it will throw an error. In IDL 8, if there are no null values, your code will assign the last element to NaN.
Since you're starting out in IDL, I reccomend you check out this page:http://www.idlcoyote.com/code_tips/mostcommon.html
So, I would modify the loop slightly to this:
FOR j=0,numFiles-1 DO BEGIN
OpenR, lun, theseFiles[j], /Get_Lun
File = fltarr(620, 500)
ReadU, lun, File
Free_Lun, lun
JustNumbers= where(File lt -9990.0, count)
IF count NE 0 THEN File[JustNumbers]=!VALUES.F_NAN
*
* ;do more processing
*
*
ENDFOR
Good luck!
|
|
|
|
| Re: Spatial and temporal image correlation [message #85562 is a reply to message #85554] |
Fri, 16 August 2013 06:47 |
Cornelio Zolin
Messages: 5
Registered: August 2013
|
Junior Member |
|
|
On Thursday, August 15, 2013 3:44:36 PM UTC-4, Phillip Bitzer wrote:
> Besides the aforementioned use of file_search and not using CD, let me add the following about your loop:
>
>
>
> 1) Generally speaking, I like to open the file I need, read it in, and free the lun for file immediately before doing any processing. This especially goes while you are debugging your code. The reason is so you don't have any "hanging" luns that aren't freed properly.
>
>
>
> 2) In addition, you are making file a float array for the data in the file. Is that the case, or is it integers? If it *is* floats, it's generally not a good idea to check for equality between floats (see http://www.idlcoyote.com/code_tips/comparearray.html for a discussion). I'll assume below -9999 is your missing value, and all good data is definitely greater than -9990.0. This may not really affect you here, but at some point checking if two floats are equal will get you in trouble.
>
>
>
> 3) You should use square brackets to access array elements (I see you do that once, so it may be a typo in the OPENR call). Although not strictly required, this will help you down the line and help you differentiate between arrays and functions. The use of the compiler option idl2 (see http://www.exelisvis.com/docs/COMPILE_OPT.html) is highly recommended by several notable IDL programmers.
>
>
>
> 4) Be sure to check the count keyword before assigned the null values to NaN. In IDL 7 and below, if there are no null values, the code as you've written it will throw an error. In IDL 8, if there are no null values, your code will assign the last element to NaN.
>
>
>
> Since you're starting out in IDL, I reccomend you check out this page:http://www.idlcoyote.com/code_tips/mostcommon.html
>
>
>
> So, I would modify the loop slightly to this:
>
>
>
> FOR j=0,numFiles-1 DO BEGIN
>
> OpenR, lun, theseFiles[j], /Get_Lun
>
> File = fltarr(620, 500)
>
> ReadU, lun, File
>
> Free_Lun, lun
>
> JustNumbers= where(File lt -9990.0, count)
>
> IF count NE 0 THEN File[JustNumbers]=!VALUES.F_NAN
>
> *
>
> * ;do more processing
>
> *
>
> *
>
>
>
> ENDFOR
>
>
>
> Good luck!
Thanks a lot for the tips.
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
