| Re: ENVI_CONVERT_FILE_MAP_PROJECTION processing times [message #77477] |
Tue, 06 September 2011 00:16  |
Allard de Wit
Messages: 7
Registered: December 2002
|
Junior Member |
|
|
Dear Devin,
Thanks for pointing this out.
Interactive ENVI has a feature for reprojecting images called "Match
existing file" which is great for aligning sets of images on exactly
the same grid. Your post explains why I have never been able to find
out how to use this feature in ENVI Batch mode. Of course, I would
appreciate if ITTVIS could expose more of its internal APIs in ENVI
batch mode.
|
|
|
|
| Re: ENVI_CONVERT_FILE_MAP_PROJECTION processing times [message #77490 is a reply to message #77477] |
Sat, 03 September 2011 05:35  |
devin.white
Messages: 50
Registered: March 2007
|
Member |
|
|
The reprojection routines used by interactive ENVI are indeed
different from (and superior to) those you can access via the API.
Especially for the Triangulation and Rigorous options. The
interactive version handles those cases much more efficiently. You'll
see the biggest difference when working with large images. The API
does certain preprocessing steps in memory (and in a different way)
than interactive ENVI and, as a result, you can run into memory
allocation problems for the same image that processed just fine in
interactive ENVI.
You'll want to avoid the Layer Stacking option, as that tends to use a
lowest common denominator approach to reprojection. Sure, it's fast,
but you pay a heavy price for that speed. It's better to get your
layers into the same projection and GSD (pixel size) first, then layer
stack.
The main issue I see with your code is the DIMS keyword usage and your
lack of the GRID keyword. Your call should look like this:
ENVI_CONVERT_FILE_MAP_PROJECTION, fid=gcov_fid, $
pos=pos, dims=dims, o_proj=srtm_proj, $
o_pixel_size=srtm_pix_size, $
out_name=out_name, warp_method=2, $
resampling=2, background=0, $
grid=[50,50]
You can't rely on the API to know that it only needs the first five
elements of whatever array you pass it. Some routines do check to
make sure the DIMS array has only five elements. The next time you
run interactive reprojection with Triangulation, note the number of
grid points it uses in X and Y and supply those to your call. I
*think* the interactive defaults are [50,50], but I don't recall what
the API defaults are. If they are higher than that, it would go a
long way towards explaining the increased processing time (denser grid
= longer processing time). FYI: WARP_METHOD=3 will always give you
the best results, but it is definitely the slowest option.
On Sep 2, 6:09 pm, David Fanning <n...@dfanning.com> wrote:
> Martin Landsfeld writes:
>> I am remapping some GeoCover imagery to conform to our SRTM data. The
>> GeoCover is in the UTM projection at 28.5 meters and the SRTM is in a
>> Geographic projection at 0.0008333 degree resolution. I can do the
>> conversion interactively in ENVI and it takes about 67 seconds. The
>> files are large with:
>
>> SRTM at 7208 x 9608 pixels
>> GeoCover at 26857x19517 pixels
>
>> I would like to run the conversions in IDL using the batch ENVI
>> function ENVI_CONVERT_FILE_MAP_PROJECTION but when I do, it takes
>> nearly 2 hours to process the same files on the same machine, a
>> Windows 2008 Server with 4, 2.2GHz. Quad-Core processors and 32GB RAM.
>
> I seriously doubt you are doing anything wrong. But, it
> has been my experience that ENVI does not use its own
> API. So, the way they do things internally, and the
> way they tell you to do them programmatically are
> really two different things. It wouldn't surprise me
> to find this is one of those cases.
>
> Cheers,
>
> David
>
> --
> David Fanning, Ph.D.
> Fanning Software Consulting, Inc.
> Coyote's Guide to IDL Programming:http://www.dfanning.com/
> Sepore ma de ni thui. ("Perhaps thou speakest truth.")
|
|
|
|
| Re: ENVI_CONVERT_FILE_MAP_PROJECTION processing times [message #77491 is a reply to message #77490] |
Fri, 02 September 2011 16:23 |
Maxwell Peck
Messages: 61
Registered: February 2010
|
Member |
|
|
On Sep 3, 7:23 am, Martin Landsfeld <mlandsf...@gmail.com> wrote:
> Hello,
>
> I am remapping some GeoCover imagery to conform to our SRTM data. The
> GeoCover is in the UTM projection at 28.5 meters and the SRTM is in a
> Geographic projection at 0.0008333 degree resolution. I can do the
> conversion interactively in ENVI and it takes about 67 seconds. The
> files are large with:
>
> SRTM at 7208 x 9608 pixels
> GeoCover at 26857x19517 pixels
>
> I would like to run the conversions in IDL using the batch ENVI
> function ENVI_CONVERT_FILE_MAP_PROJECTION but when I do, it takes
> nearly 2 hours to process the same files on the same machine, a
> Windows 2008 Server with 4, 2.2GHz. Quad-Core processors and 32GB RAM.
>
> Has anyone else noticed this slowdown when using ENVI batch?
>
> I ran some other tests by subsetting the GeoCover file and got these
> results:
>
> time for 10000x10000 subset = 290 s, 4.8 min
> time for 15000x15000 subset = 1648.2 s, 27 min
> time for entire 26857x19517 = 6336.2 seconds, 105 min
>
> I don't think there is anything wrong with my code as the output looks
> fine. I'll throw that up anyway.
> Thanks!
>
> init_time = SYSTIME(/sec)
>
> ENVI_OPEN_FILE, srtm_file_path, R_FID=srtm_fid
> if (srtm_fid eq -1) then message, 'SRTM file not found'
>
> ENVI_FILE_QUERY, srtm_fid, DIMS=dims, NB=nb, $
> DATA_TYPE=data_type, NS=ns, NL=nl
>
> srtm_proj = ENVI_GET_PROJECTION(fid = srtm_fid,
> pixel_size=srtm_pix_size)
> ENVI_FILE_MNG, ID=srtm_fid, /REMOVE
>
> ENVI_OPEN_FILE, gcov_file_path, R_FID=gcov_fid
> if (gcov_fid eq -1) then message, 'GeoCover file not found'
>
> ENVI_FILE_QUERY, gcov_fid, DIMS=dims, NB=nb
> pos = LINDGEN(nb)
>
> ENVI_CONVERT_FILE_MAP_PROJECTION, fid=gcov_fid, $
> pos=pos, dims=[dims,dims,dims], o_proj=srtm_proj, $
> o_pixel_size=srtm_pix_size, $
> out_name=out_name, warp_method=2, $
> resampling=2, background=0
>
> PRINT, 'processing time = ', SYSTIME(/sec) - init_time, format='(a,
> f6.1)'
> ENVI_FILE_MNG, ID=gcov_fid, /REMOVE
I assume the resampling and warping methods are the same.. The other
option is to use the layer stack doit function instead and see if it
makes any difference (from memory I have seen signficant improvements
in using the layer stacker instead of straight out map conversion).
Max
|
|
|
|
| Re: ENVI_CONVERT_FILE_MAP_PROJECTION processing times [message #77494 is a reply to message #77491] |
Fri, 02 September 2011 15:09 |
David Fanning
Messages: 11724
Registered: August 2001
|
Senior Member |
|
|
Martin Landsfeld writes:
> I am remapping some GeoCover imagery to conform to our SRTM data. The
> GeoCover is in the UTM projection at 28.5 meters and the SRTM is in a
> Geographic projection at 0.0008333 degree resolution. I can do the
> conversion interactively in ENVI and it takes about 67 seconds. The
> files are large with:
>
> SRTM at 7208 x 9608 pixels
> GeoCover at 26857x19517 pixels
>
> I would like to run the conversions in IDL using the batch ENVI
> function ENVI_CONVERT_FILE_MAP_PROJECTION but when I do, it takes
> nearly 2 hours to process the same files on the same machine, a
> Windows 2008 Server with 4, 2.2GHz. Quad-Core processors and 32GB RAM.
I seriously doubt you are doing anything wrong. But, it
has been my experience that ENVI does not use its own
API. So, the way they do things internally, and the
way they tell you to do them programmatically are
really two different things. It wouldn't surprise me
to find this is one of those cases.
Cheers,
David
--
David Fanning, Ph.D.
Fanning Software Consulting, Inc.
Coyote's Guide to IDL Programming: http://www.dfanning.com/
Sepore ma de ni thui. ("Perhaps thou speakest truth.")
|
|
|
|
| Re: ENVI_CONVERT_FILE_MAP_PROJECTION processing times [message #77558 is a reply to message #77490] |
Wed, 07 September 2011 19:07 |
Martin Landsfeld
Messages: 2
Registered: September 2011
|
Junior Member |
|
|
On Sep 3, 5:35 am, "devin.wh...@gmail.com" <devin.wh...@gmail.com>
wrote:
>
> The main issue I see with your code is the DIMS keyword usage and your
> lack of the GRID keyword. Your call should look like this:
>
> ENVI_CONVERT_FILE_MAP_PROJECTION, fid=gcov_fid, $
> pos=pos, dims=dims, o_proj=srtm_proj, $
> o_pixel_size=srtm_pix_size, $
> out_name=out_name, warp_method=2, $
> resampling=2, background=0, $
> grid=[50,50]
>
> You can't rely on the API to know that it only needs the first five
> elements of whatever array you pass it. Some routines do check to
> make sure the DIMS array has only five elements. The next time you
> run interactive reprojection with Triangulation, note the number of
> grid points it uses in X and Y and supply those to your call. I
> *think* the interactive defaults are [50,50], but I don't recall what
> the API defaults are. If they are higher than that, it would go a
> long way towards explaining the increased processing time (denser grid
> = longer processing time). FYI: WARP_METHOD=3 will always give you
> the best results, but it is definitely the slowest option.
>
Thanks Devin,
It turns out that the grid parameter is crucial to the processing
times I was getting. The grid default for the API is 25x25, whereas
the default for ENVI batch is 10x10. On my smallest test case,
10000x10000 pixels, I ran the following tests:
grid=[10,10], time = 176 sec
grid=[15,15], time = 55 sec
grid=[20,20], time = 18 sec
grid=[50,50], time = 5 sec
Obviously the 10 pixel spacing default of control points for the
warping is too fine for large files!
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
