| Solve memory problems [message #64623] |
Tue, 13 January 2009 05:28  |
corinnefrey
Messages: 18
Registered: November 2006
|
Junior Member |
|
|
Hi everybody,
Again I am confrontend with memory problems in IDL. I am working with
satellite images and their derivatives and therefore I am often
confronted with huge datasets.
Well, running a routine once, some common tricks may help to enhance
the available memory:
- Use the temporary-function,
- Put the variables, I don't need anymore, to zero,
- Working as much as possible with interger or byte arrays,
- etc.
However, specially when running an already optimized routine several
times (e.g. for several scenes), available memory will disappear over
short or long. I guess, this has to do with the growing fragmentation
of the memory.
A solution could be to use the same variables for different grids in a
subroutine. However, I prefer to name the variables according to their
physical (or whatever) meaning. Otherwise, after some time, I don't
have any chance to understand my programms anymore.
In my dreams, delvar is available also in routines. However, it's not,
is it?
So, after this long introduction: What strategies are you using, to
have more memory available, or to prevent the fragmentation of the
memory?
Thanks for any comments.
Regards,
Corinne
|
|
|
|
| Re: Solve memory problems [message #64674 is a reply to message #64623] |
Wed, 14 January 2009 07:38  |
pgrigis
Messages: 436
Registered: September 2007
|
Senior Member |
|
|
But probably this is important only on windows 32-bit systems?
Ciao,
Paolo
Jean H. wrote:
> Corinne,
>
> setting var=0 or *ptr_var = 0 will have the same effect on memory.
> Now, let's say you have 10 bands, 4000*5000. If you try to create an
> array like this data = bytarr(4000,5000,10), you might run out of
> memory, or, for the same reason, can not create any other variable (not
> enough contiguous space in memory). With this example, you would need
> about 8 bits * 4000 * 5000 * 10 = 1 600 000 000 bits of contiguous
> memory. Now, if you use points, you can create an array of 10 pointers,
> each holding a band.
> ptr_data = ptrarr(10)
> ptr_data[0] = ptr_new(bytarr(4000,5000)
> ptr_data[1] = ptr_new(bytarr(4000,5000)
> ....
>
> so now, the contiguous memory you need is only 8 bits * 4000 * 5000 =
> 160 000 000 bits.
>
> If band 2 has a different size, no problem:
> ptr_data[2] = ptr_new(bytarr(12,25)
>
> Jean
>
> Corinne wrote:
>> hi jean,
>>
>> i have never used pointers, so my question is: do you use separate
>> pointers or whole pointer-arrays for your bands? what do you do, if
>> your bands have different sizes due to different spatial resolutions
>> (which you might interpolate later in the programme)?
>>
>> i'm still trying to figure out, what the advantage of pointers is.
>>
>> example: i have created an float array a with 4000x5000 elements and
>> don't need it anymore. so i want to get rid of it. does it make a
>> difference, if i put a=0 or if i set the value of the pointer of the
>> array to zero?
>>
>> regards,
>> corinne
>>
>>
>>> In my own program, I do all the analysis on a modified version of my
>>> original image (a classified land-use map, with the background values
>>> removed so the data is a 1D array), then, at the very end, I
>>> re-transform it to save and display it. I save a lot of memory!
>>> Moreover, all bands are saved in pointers, allowing the program to run
>>> on almost any computer, while the original version, which did not use
>>> much pointer, was making my work-beast run out of memory fairly quickly!
>>>
>>> Jean
>>
|
|
|
|
|
|
| Re: Solve memory problems [message #64679 is a reply to message #64623] |
Wed, 14 January 2009 06:53 |
Jean H.
Messages: 472
Registered: July 2006
|
Senior Member |
|
|
Corinne,
setting var=0 or *ptr_var = 0 will have the same effect on memory.
Now, let's say you have 10 bands, 4000*5000. If you try to create an
array like this data = bytarr(4000,5000,10), you might run out of
memory, or, for the same reason, can not create any other variable (not
enough contiguous space in memory). With this example, you would need
about 8 bits * 4000 * 5000 * 10 = 1 600 000 000 bits of contiguous
memory. Now, if you use points, you can create an array of 10 pointers,
each holding a band.
ptr_data = ptrarr(10)
ptr_data[0] = ptr_new(bytarr(4000,5000)
ptr_data[1] = ptr_new(bytarr(4000,5000)
....
so now, the contiguous memory you need is only 8 bits * 4000 * 5000 =
160 000 000 bits.
If band 2 has a different size, no problem:
ptr_data[2] = ptr_new(bytarr(12,25)
Jean
Corinne wrote:
> hi jean,
>
> i have never used pointers, so my question is: do you use separate
> pointers or whole pointer-arrays for your bands? what do you do, if
> your bands have different sizes due to different spatial resolutions
> (which you might interpolate later in the programme)?
>
> i'm still trying to figure out, what the advantage of pointers is.
>
> example: i have created an float array a with 4000x5000 elements and
> don't need it anymore. so i want to get rid of it. does it make a
> difference, if i put a=0 or if i set the value of the pointer of the
> array to zero?
>
> regards,
> corinne
>
>
>> In my own program, I do all the analysis on a modified version of my
>> original image (a classified land-use map, with the background values
>> removed so the data is a 1D array), then, at the very end, I
>> re-transform it to save and display it. I save a lot of memory!
>> Moreover, all bands are saved in pointers, allowing the program to run
>> on almost any computer, while the original version, which did not use
>> much pointer, was making my work-beast run out of memory fairly quickly!
>>
>> Jean
>
|
|
|
|
| Re: Solve memory problems [message #64681 is a reply to message #64623] |
Wed, 14 January 2009 07:16 |
pgrigis
Messages: 436
Registered: September 2007
|
Senior Member |
|
|
David Fanning wrote:
> Craig Markwardt writes:
>
>> Uh, like using any other high-level language that doesn't force you to
>> free your own variables?
>
> My goodness, people, whatever happened to craftsmanship?
> I feel like I'm working with a bunch of Wal-Mart furniture
> builders here. Quick, easy, cheap. :-(
>
> Cheers,
>
> David
>
> P.S. I guess you probably know it is time to give it up when
> you get to the point where you believe the world will go
> straight to hell in a handbasket if you *do* give it up.
> Just don't be sending me any code with HEAP_GC in it. Even
> Coyote wouldn't lower himself *that* much. :-)
Well, I must say that garbage collection is not such a bad idea,
or else I would have an unpleasant and smelly pile of stuff in
front of my house... ;-)
Ciao,
Paolo
>
> --
> David Fanning, Ph.D.
> Fanning Software Consulting, Inc.
> Coyote's Guide to IDL Programming: http://www.dfanning.com/
> Sepore ma de ni thui. ("Perhaps thou speakest truth.")
|
|
|
|
|
|
| Re: Solve memory problems [message #64683 is a reply to message #64623] |
Wed, 14 January 2009 06:59 |
Paul Van Delst[1]
Messages: 1157
Registered: April 2002
|
Senior Member |
|
|
Corinne wrote:
> hi guys,
>
> now i got lots of inputs! thanks so much! will check memtest.pro, as
> well as this pointer stuff and heap_gc. if it helps, why not?
Your unit test suite should include a test case that checks for valid pointers (and
objects if appropriate) after your cleanup routines. If there are still non-null pointers
floating around, your cleanup is incomplete. It's very simple to test for this stuff --
even in what I would refer to as "research grade" code.
I second David's incredulity at suggestions that generic garbage collection is an
acceptable solution when you have memory issues. When the problems stem from
structures/variables that the programmer created, it behooves said programmer to fix the
problems. Otherwise it's just sloppy programming -- and you're learning bad habits that,
when transferred to other languages, may not be so easily corrected.
It reminds me of the old chestnut about three properties of software: fast, good, cheap.
Pick any two. :o)
cheers,
paulv
>
> @david: my 'images' contain normally up to 9 bands. normally i handle
> them separate (read in only exactly the band i need), as the file
> would be too big otherwise.
>
> best regards,
> corinne
>
>
> On Jan 14, 7:26 am, Craig Markwardt <cbmarkwa...@gmail.com> wrote:
>> On Jan 13, 10:33 am, David Fanning <n...@dfanning.com> wrote:
>>
>>> Jean H. writes:
>>>> As Carsten has mentioned, play with memtest.pro (from ITTVIS) to find
>>>> out what is happening. It could as well be a memory leak (you create a
>>>> pointer but don't destroy it). In this case, make a call to "heap_gc"
>>>> after your function.
>>> What!? What kind of advice is this!
>>> Uh, do NOT be making a call to HEAP_GC unless your program
>>> has completely and utterly failed and it is late Friday
>>> afternoon and you are at wit's end. Believe me when I tell
>>> you there are MUCH better ways to handle this!
>> Uh, like using any other high-level language that doesn't force you to
>> free your own variables?
>
|
|
|
|
|
|
| Re: Solve memory problems [message #64686 is a reply to message #64623] |
Wed, 14 January 2009 05:37 |
corinnefrey
Messages: 18
Registered: November 2006
|
Junior Member |
|
|
hi jean,
i have never used pointers, so my question is: do you use separate
pointers or whole pointer-arrays for your bands? what do you do, if
your bands have different sizes due to different spatial resolutions
(which you might interpolate later in the programme)?
i'm still trying to figure out, what the advantage of pointers is.
example: i have created an float array a with 4000x5000 elements and
don't need it anymore. so i want to get rid of it. does it make a
difference, if i put a=0 or if i set the value of the pointer of the
array to zero?
regards,
corinne
> In my own program, I do all the analysis on a modified version of my
> original image (a classified land-use map, with the background values
> removed so the data is a 1D array), then, at the very end, I
> re-transform it to save and display it. I save a lot of memory!
> Moreover, all bands are saved in pointers, allowing the program to run
> on almost any computer, while the original version, which did not use
> much pointer, was making my work-beast run out of memory fairly quickly!
>
> Jean
|
|
|
|
| Re: Solve memory problems [message #64687 is a reply to message #64623] |
Wed, 14 January 2009 04:55 |
Jean H.
Messages: 472
Registered: July 2006
|
Senior Member |
|
|
David Fanning wrote:
> Jean H. writes:
>
>> if not already in use, using pointers can be of great help. You can save
>> each band in a different pointer, or even each line etc. Therefore, the
>> size of the required contiguous memory space decreases.
>
> I was so upset with the HEAP_GC suggestion this morning
> that I forgot to respond to this. But I don't immediately
> see how this will help, unless, of course you have some
> ability to read all three bands from the file as bands
> and not as an image.
>
> Otherwise, I think you would have to allocate twice
> the image size to do the transfer to pointers.
>
> Plus, you have to put the bands together again to display
> the image, so I don't see where I am ahead of anything
> here. What am I missing?
>
> Cheers,
>
> David
Hi,
Agreed for the memory cost associated with loading the data... but down
the road (or down the program), it might still be useful to save
memory.. for example, if the routine that is being called needs 10 times
the memory of the input.
In my own program, I do all the analysis on a modified version of my
original image (a classified land-use map, with the background values
removed so the data is a 1D array), then, at the very end, I
re-transform it to save and display it. I save a lot of memory!
Moreover, all bands are saved in pointers, allowing the program to run
on almost any computer, while the original version, which did not use
much pointer, was making my work-beast run out of memory fairly quickly!
Jean
|
|
|
|
| Re: Solve memory problems [message #64689 is a reply to message #64623] |
Wed, 14 January 2009 03:56 |
corinnefrey
Messages: 18
Registered: November 2006
|
Junior Member |
|
|
hi guys,
now i got lots of inputs! thanks so much! will check memtest.pro, as
well as this pointer stuff and heap_gc. if it helps, why not?
@david: my 'images' contain normally up to 9 bands. normally i handle
them separate (read in only exactly the band i need), as the file
would be too big otherwise.
best regards,
corinne
On Jan 14, 7:26 am, Craig Markwardt <cbmarkwa...@gmail.com> wrote:
> On Jan 13, 10:33 am, David Fanning <n...@dfanning.com> wrote:
>
>> Jean H. writes:
>>> As Carsten has mentioned, play with memtest.pro (from ITTVIS) to find
>>> out what is happening. It could as well be a memory leak (you create a
>>> pointer but don't destroy it). In this case, make a call to "heap_gc"
>>> after your function.
>
>> What!? What kind of advice is this!
>
>> Uh, do NOT be making a call to HEAP_GC unless your program
>> has completely and utterly failed and it is late Friday
>> afternoon and you are at wit's end. Believe me when I tell
>> you there are MUCH better ways to handle this!
>
> Uh, like using any other high-level language that doesn't force you to
> free your own variables?
|
|
|
|
|
|
| Re: Solve memory problems [message #64698 is a reply to message #64623] |
Tue, 13 January 2009 13:01 |
David Fanning
Messages: 11724
Registered: August 2001
|
Senior Member |
|
|
Jean H. writes:
> if not already in use, using pointers can be of great help. You can save
> each band in a different pointer, or even each line etc. Therefore, the
> size of the required contiguous memory space decreases.
I was so upset with the HEAP_GC suggestion this morning
that I forgot to respond to this. But I don't immediately
see how this will help, unless, of course you have some
ability to read all three bands from the file as bands
and not as an image.
Otherwise, I think you would have to allocate twice
the image size to do the transfer to pointers.
Plus, you have to put the bands together again to display
the image, so I don't see where I am ahead of anything
here. What am I missing?
Cheers,
David
--
David Fanning, Ph.D.
Coyote's Guide to IDL Programming (www.dfanning.com)
Sepore ma de ni thui. ("Perhaps thou speakest truth.")
|
|
|
|
|
|
|
|
| Re: Solve memory problems [message #64709 is a reply to message #64623] |
Tue, 13 January 2009 07:56 |
Allan Whiteford
Messages: 117
Registered: June 2006
|
Senior Member |
|
|
David Fanning wrote:
> Jean H. writes:
>
>
>> As Carsten has mentioned, play with memtest.pro (from ITTVIS) to find
>> out what is happening. It could as well be a memory leak (you create a
>> pointer but don't destroy it). In this case, make a call to "heap_gc"
>> after your function.
>
>
> What!? What kind of advice is this!
>
> Uh, do NOT be making a call to HEAP_GC unless your program
> has completely and utterly failed and it is late Friday
> afternoon and you are at wit's end. Believe me when I tell
> you there are MUCH better ways to handle this!
>
> Cheers,
>
> David
Perhaps a compromise:
Do a "help,/heap" and see how many pointers you have sitting, then do a
heap_gc followed immediately by a "help,/heap" again. If you're leaking
memory by not freeing pointers or destroying objects then chances are
the two results of help,/heap will be different. If they are the same
then the heap_gc didn't do anything and the problem is elsewhere.
help,/heap will even give you an idea of what heap variable is causing
the problem.
Thanks,
Allan
|
|
|
|
|
|
|
|
| Re: Solve memory problems [message #64715 is a reply to message #64623] |
Tue, 13 January 2009 07:03 |
Jean H.
Messages: 472
Registered: July 2006
|
Senior Member |
|
|
> Well, running a routine once, some common tricks may help to enhance
> the available memory:
>
> - Use the temporary-function,
> - Put the variables, I don't need anymore, to zero,
> - Working as much as possible with interger or byte arrays,
> - etc.
if not already in use, using pointers can be of great help. You can save
each band in a different pointer, or even each line etc. Therefore, the
size of the required contiguous memory space decreases.
> However, specially when running an already optimized routine several
> times (e.g. for several scenes), available memory will disappear over
> short or long. I guess, this has to do with the growing fragmentation
> of the memory.
As Carsten has mentioned, play with memtest.pro (from ITTVIS) to find
out what is happening. It could as well be a memory leak (you create a
pointer but don't destroy it). In this case, make a call to "heap_gc"
after your function.
> A solution could be to use the same variables for different grids in a
> subroutine. However, I prefer to name the variables according to their
> physical (or whatever) meaning. Otherwise, after some time, I don't
> have any chance to understand my programms anymore.
>
> In my dreams, delvar is available also in routines. However, it's not,
> is it?
>
> So, after this long introduction: What strategies are you using, to
> have more memory available, or to prevent the fragmentation of the
> memory?
Under windows, try to assign an array as big as possible, 1st thing in
your program. At least you are "reserving" the contiguous memory for IDL.
Switching to Linux made my life soooooo much easier, for any
memory-related problem!
Jean
>
> Thanks for any comments.
>
> Regards,
> Corinne
>
|
|
|
|
|
|
| Re: Solve memory problems [message #64719 is a reply to message #64623] |
Tue, 13 January 2009 06:34 |
corinnefrey
Messages: 18
Registered: November 2006
|
Junior Member |
|
|
hi carsten,
indeed, this is the case in my actual problem. i am using a programme,
which uses several subprogrammes and subsub-programmes. between each
level there is an exchange of variables. if i defined all variables
and return-variables at the beginning, i would run out of memory just
by defining the variables. but probably i could make a compilation of
the "most wanted" variables to have them at a constant place from the
beginning. could be worth checking out!
thanks,
corinne
> All the memory you use in a function or procedure is given back when that
> function returns, except for any variables that are handed back via
> arguments or function return values.
>
> This is pure conjecture, but maybe you use some memory in your function, then
> allocate more memory for the return value, then return, freeing the earlier
> memory, but leaving you with the return value allocated somewhere "in the
> middle" of the heap memory, thereby fragmenting it more and more with each
> call of the function. Then, it should make a difference if you allocate
> your return array at the beginning of your function.
>
> Maybe you could use memtest.pro to investigate the fragmentation problem?
>
> chl
|
|
|
|
| Re: Solve memory problems [message #64720 is a reply to message #64623] |
Tue, 13 January 2009 05:57 |
Carsten Lechte
Messages: 124
Registered: August 2006
|
Senior Member |
|
|
Corinne wrote:
> However, specially when running an already optimized routine several
> times (e.g. for several scenes), available memory will disappear over
> short or long.
All the memory you use in a function or procedure is given back when that
function returns, except for any variables that are handed back via
arguments or function return values.
This is pure conjecture, but maybe you use some memory in your function, then
allocate more memory for the return value, then return, freeing the earlier
memory, but leaving you with the return value allocated somewhere "in the
middle" of the heap memory, thereby fragmenting it more and more with each
call of the function. Then, it should make a difference if you allocate
your return array at the beginning of your function.
Maybe you could use memtest.pro to investigate the fragmentation problem?
chl
|
|
|
|
| Re: Solve memory problems [message #64741 is a reply to message #64679] |
Thu, 15 January 2009 04:10 |
Guillaume Potdevin
Messages: 1
Registered: January 2009
|
Junior Member |
|
|
Hi!
I followed with great interest this discussion, as I sometimes run into
the same sort of problems, though for other sort of applications.
For reason of commodity, I often use structures. Do structures require a
contiguous space in memory? For example: does
struct = {image1 : BYTARR(4000,5000), image2 : BYTARR(4000,5000) }
require 8 bits * 4000 * 5000 * 2 of continuous space?
And, also the same question in the case of structure arrays: if we have
struct_array = REPLICATE(struct , many_times)
Does struct_array require many_times the space for struct as contiguous
space (in case struct needs a contiguous memory slot)?
But I see the immediate solution here to use arrays of pointers...
Guillaume.
On 14.01.2009 15:53, Jean H. wrote:
> Corinne,
>
> setting var=0 or *ptr_var = 0 will have the same effect on memory.
> Now, let's say you have 10 bands, 4000*5000. If you try to create an
> array like this data = bytarr(4000,5000,10), you might run out of
> memory, or, for the same reason, can not create any other variable (not
> enough contiguous space in memory). With this example, you would need
> about 8 bits * 4000 * 5000 * 10 = 1 600 000 000 bits of contiguous
> memory. Now, if you use points, you can create an array of 10 pointers,
> each holding a band.
> ptr_data = ptrarr(10)
> ptr_data[0] = ptr_new(bytarr(4000,5000)
> ptr_data[1] = ptr_new(bytarr(4000,5000)
> ....
>
> so now, the contiguous memory you need is only 8 bits * 4000 * 5000 =
> 160 000 000 bits.
>
> If band 2 has a different size, no problem:
> ptr_data[2] = ptr_new(bytarr(12,25)
>
> Jean
>
|
|
|
|
| Re: Solve memory problems [message #64742 is a reply to message #64679] |
Thu, 15 January 2009 03:45 |
corinnefrey
Messages: 18
Registered: November 2006
|
Junior Member |
|
|
Hi Jean,
Jap, got it! This is indeed an advantage. Will try to implement that
in my next programme.
Have a nice day,
Corinne
On Jan 14, 3:53 pm, "Jean H." <jghas...@DELTHIS.ucalgary.ANDTHIS.ca>
wrote:
> Corinne,
>
> setting var=0 or *ptr_var = 0 will have the same effect on memory.
> Now, let's say you have 10 bands, 4000*5000. If you try to create an
> array like this data = bytarr(4000,5000,10), you might run out of
> memory, or, for the same reason, can not create any other variable (not
> enough contiguous space in memory). With this example, you would need
> about 8 bits * 4000 * 5000 * 10 = 1 600 000 000 bits of contiguous
> memory. Now, if you use points, you can create an array of 10 pointers,
> each holding a band.
> ptr_data = ptrarr(10)
> ptr_data[0] = ptr_new(bytarr(4000,5000)
> ptr_data[1] = ptr_new(bytarr(4000,5000)
> ....
>
> so now, the contiguous memory you need is only 8 bits * 4000 * 5000 =
> 160 000 000 bits.
>
> If band 2 has a different size, no problem:
> ptr_data[2] = ptr_new(bytarr(12,25)
>
> Jean
>
> Corinne wrote:
>> hi jean,
>
>> i have never used pointers, so my question is: do you use separate
>> pointers or whole pointer-arrays for your bands? what do you do, if
>> your bands have different sizes due to different spatial resolutions
>> (which you might interpolate later in the programme)?
>
>> i'm still trying to figure out, what the advantage of pointers is.
>
>> example: i have created an float array a with 4000x5000 elements and
>> don't need it anymore. so i want to get rid of it. does it make a
>> difference, if i put a=0 or if i set the value of the pointer of the
>> array to zero?
>
>> regards,
>> corinne
>
>>> In my own program, I do all the analysis on a modified version of my
>>> original image (a classified land-use map, with the background values
>>> removed so the data is a 1D array), then, at the very end, I
>>> re-transform it to save and display it. I save a lot of memory!
>>> Moreover, all bands are saved in pointers, allowing the program to run
>>> on almost any computer, while the original version, which did not use
>>> much pointer, was making my work-beast run out of memory fairly quickly!
>
>>> Jean
>
>
|
|
|
|
| Re: Solve memory problems [message #64748 is a reply to message #64623] |
Wed, 14 January 2009 23:41 |
Craig Markwardt
Messages: 1869
Registered: November 1996
|
Senior Member |
|
|
On Jan 14, 12:55 pm, "mgal...@gmail.com" <mgal...@gmail.com> wrote:
>
> That said, IDL does not have the same garbage collection algorithms.
> HEAP_GC is *slow* when you have a lot of heap variables. I would
> support IDL adding real garbage collection, but what is currently in
> IDL is a debugging tool only.
It doesn't really have to be garbage collection. Python uses
reference-counting, and IDL could easily do the same kind of thing,
with extremely low overhead. Internal to IDL, what is needed is one
additional integer for each heap variable and three extra instructions
per pointer assignment.
Garbage collection might still be useful for when cycles occur.
Craig
|
|
|
|
| Re: Solve memory problems [message #64762 is a reply to message #64623] |
Wed, 14 January 2009 09:55 |
Michael Galloy
Messages: 1114
Registered: April 2006
|
Senior Member |
|
|
On Jan 14, 9:23 am, Craig Markwardt <cbmarkwa...@gmail.com> wrote:
> On Jan 14, 9:03 am, David Fanning <n...@dfanning.com> wrote:
>
>> Craig Markwardt writes:
>>> Uh, like using any other high-level language that doesn't force you to
>>> free your own variables?
>
>> My goodness, people, whatever happened to craftsmanship?
>> I feel like I'm working with a bunch of Wal-Mart furniture
>> builders here. Quick, easy, cheap. :-(
>
> Uh, I wasn't defending the use of HEAP_GC, but it's monumentally silly
> that the IDL language designers had the choice to implement automatic
> freeing of dangling pointers ("garbage collection") and did not [*].
> I mean, would you really enjoy the "privilege" of freeing every
> *regular* variable before returning from each IDL procedure? Of
> course not. The IDL runtime has enough information to know *exactly*
> when a pointer becomes dangling, so why not use that information?
>
> I think I understand craftmanship -- I hope my public code speaks for
> itself. But I don't think that has anything to do with masochistic
> worship at the alter of POINTER_FREE.
>
> Snark-Craig
>
> [*] - I understand that for debugging purposes, automatic garbage
> collection may be a nuisance. It would be straightforward to disable
> it with a system variable.
I must say that garbage collection is one of the features I really
appreciate when I use Python. Garbage collection is now even in OS X's
objective C.
That said, IDL does not have the same garbage collection algorithms.
HEAP_GC is *slow* when you have a lot of heap variables. I would
support IDL adding real garbage collection, but what is currently in
IDL is a debugging tool only.
Mike
--
www.michaelgalloy.com
Tech-X Corporation
Associate Research Scientist
|
|
|
|
| Re: Solve memory problems [message #64763 is a reply to message #64623] |
Wed, 14 January 2009 08:48 |
David Fanning
Messages: 11724
Registered: August 2001
|
Senior Member |
|
|
Craig Markwardt writes:
> I think I understand craftmanship -- I hope my public code speaks for
> itself. But I don't think that has anything to do with masochistic
> worship at the alter of POINTER_FREE.
>
> Snark-Craig
Yeah, I feel guilty snarking at you, of all people.
Best curve fitting code on the planet, etc., etc.
Please don't take it personally.
I've scheduled an appointment with the therapist to
see if I can get to the bottom of why this HEAP_GC
thing bothers me so much. :-)
> [*] - I understand that for debugging purposes, automatic garbage
> collection may be a nuisance. It would be straightforward to disable
> it with a system variable.
I guess I would be more concerned with how it would impact
the speed of FOR loops. ;-)
Cheers,
David
--
David Fanning, Ph.D.
Fanning Software Consulting, Inc.
Coyote's Guide to IDL Programming: http://www.dfanning.com/
Sepore ma de ni thui. ("Perhaps thou speakest truth.")
|
|
|
|
| Re: Solve memory problems [message #64765 is a reply to message #64684] |
Wed, 14 January 2009 08:23 |
Craig Markwardt
Messages: 1869
Registered: November 1996
|
Senior Member |
|
|
On Jan 14, 9:03 am, David Fanning <n...@dfanning.com> wrote:
> Craig Markwardt writes:
>> Uh, like using any other high-level language that doesn't force you to
>> free your own variables?
>
> My goodness, people, whatever happened to craftsmanship?
> I feel like I'm working with a bunch of Wal-Mart furniture
> builders here. Quick, easy, cheap. :-(
Uh, I wasn't defending the use of HEAP_GC, but it's monumentally silly
that the IDL language designers had the choice to implement automatic
freeing of dangling pointers ("garbage collection") and did not [*].
I mean, would you really enjoy the "privilege" of freeing every
*regular* variable before returning from each IDL procedure? Of
course not. The IDL runtime has enough information to know *exactly*
when a pointer becomes dangling, so why not use that information?
I think I understand craftmanship -- I hope my public code speaks for
itself. But I don't think that has anything to do with masochistic
worship at the alter of POINTER_FREE.
Snark-Craig
[*] - I understand that for debugging purposes, automatic garbage
collection may be a nuisance. It would be straightforward to disable
it with a system variable.
>
> Cheers,
>
> David
>
> P.S. I guess you probably know it is time to give it up when
> you get to the point where you believe the world will go
> straight to hell in a handbasket if you *do* give it up.
> Just don't be sending me any code with HEAP_GC in it. Even
> Coyote wouldn't lower himself *that* much. :-)
>
> --
> David Fanning, Ph.D.
> Fanning Software Consulting, Inc.
> Coyote's Guide to IDL Programming:http://www.dfanning.com/
> Sepore ma de ni thui. ("Perhaps thou speakest truth.")
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
