Messages from Usenet group comp.lang.idl-pvwave, compiled by Paulo Penteado
Re: objects, crashes, and negative memory oh my
Pete Warner writes: > On the other hand, if anyone wants to buy a program that uses negative > memory this is your lucky day. Yeah, I think maybe a couple of things are wrong here. Is this a 64-bit version of IDL? Running on what? Sounds like a call to RSI is in order. Cheers, David -- David Fanning, Ph.D. Fanning Software Consulting, Inc. Coyote's Guide to IDL Programming: http://www.dfanning.com/ Sepore ma de ni thui. ("Perhaps thou speakest truth.")
Pete Warner wrote: > Hopefully one day I'll join the crowd of you helpful people who answer > questions instead of coming up with them, but that day is not today. > > I've got an object that should hold and organize a large number of > arrays and pointers and arrays of pointers and then clean them up when > I'm done. I know I have enough memory for the data in question because > I can load it without using pointers. I want the objects and pointers > for flexibility and to make the overall program easier to work with. > > The problem is that I'm crashing IDL. I create the object, and then > destroy it to check for leaks. Using help, /heap_variables I show 0 > pointers and objects. Then if I create and destroy it again IDL locks > up or crashes to desktop with some memory errors. I figured there must > be crazy leaking going on but I can't find it. > > 1. Restart computer > 2. Load IDL > 3. Load project with objects > 4. Compile project > 5. IDL> print, memory(/l64, /current) > 845185 > 6. IDL> test = obj_new('mtgv_lodat', 'model.txt', > 'c:/rsi/projects/gicvu idl/system models/') > 7. IDL> print, memory(/l64, /current) > 412234 > 8. IDL> obj_destroy, test > 9. IDL> print, memory(/l64, /current) > -1077816 Maybe is not negative, but just overflowing for 18446744073708473800LL... now that's a nice rig you have, just one megabyte short of 4 exabytes ;-) Ciao, Paolo > > If I repeat the process starting at 5 IDL blows up after a repetition > or two. If anyone has had this problem before and knows a solution I'd > be grateful. I couldn't find a solution searching the board, the IDL > help, or the ITTVIS IDL memory allocation FAQs. > > On the other hand, if anyone wants to buy a program that uses negative > memory this is your lucky day. >
Paolo Grigis wrote: > > Pete Warner wrote: > >> Hopefully one day I'll join the crowd of you helpful people who answer >> questions instead of coming up with them, but that day is not today. >> >> I've got an object that should hold and organize a large number of >> arrays and pointers and arrays of pointers and then clean them up when >> I'm done. I know I have enough memory for the data in question because >> I can load it without using pointers. I want the objects and pointers >> for flexibility and to make the overall program easier to work with. >> >> The problem is that I'm crashing IDL. I create the object, and then >> destroy it to check for leaks. Using help, /heap_variables I show 0 >> pointers and objects. Then if I create and destroy it again IDL locks >> up or crashes to desktop with some memory errors. I figured there must >> be crazy leaking going on but I can't find it. >> >> 1. Restart computer >> 2. Load IDL >> 3. Load project with objects >> 4. Compile project >> 5. IDL> print, memory(/l64, /current) >> 845185 >> 6. IDL> test = obj_new('mtgv_lodat', 'model.txt', >> 'c:/rsi/projects/gicvu idl/system models/') >> 7. IDL> print, memory(/l64, /current) >> 412234 >> 8. IDL> obj_destroy, test >> 9. IDL> print, memory(/l64, /current) >> -1077816 > > > Maybe is not negative, but just overflowing for > 18446744073708473800LL... now that's a nice > rig you have, just one megabyte short of 4 exabytes ;-) well, that should have been 16... > > Ciao, > Paolo > >> >> If I repeat the process starting at 5 IDL blows up after a repetition >> or two. If anyone has had this problem before and knows a solution I'd >> be grateful. I couldn't find a solution searching the board, the IDL >> help, or the ITTVIS IDL memory allocation FAQs. >> >> On the other hand, if anyone wants to buy a program that uses negative >> memory this is your lucky day. >>
I guess I've been thinking about asking for an upgrade. Maybe I already got it. Anybody know if Windows Vista can address 16 exabytes or will I need Linux for that? On Dec 1, 2:53 am, Paolo Grigis <pgri...@astro.phys.ethz.ch> wrote: > Paolo Grigis wrote: > >> Pete Warner wrote: > >>> Hopefully one day I'll join the crowd of you helpful people who answer >>> questions instead of coming up with them, but that day is not today. > >>> I've got an object that should hold and organize a large number of >>> arrays and pointers and arrays of pointers and then clean them up when >>> I'm done. I know I have enough memory for the data in question because >>> I can load it without using pointers. I want the objects and pointers >>> for flexibility and to make the overall program easier to work with. > >>> The problem is that I'm crashing IDL. I create the object, and then >>> destroy it to check for leaks. Using help, /heap_variables I show 0 >>> pointers and objects. Then if I create and destroy it again IDL locks >>> up or crashes to desktop with some memory errors. I figured there must >>> be crazy leaking going on but I can't find it. > >>> 1. Restart computer >>> 2. Load IDL >>> 3. Load project with objects >>> 4. Compile project >>> 5. IDL> print, memory(/l64, /current) >>> 845185 >>> 6. IDL> test = obj_new('mtgv_lodat', 'model.txt', >>> 'c:/rsi/projects/gicvu idl/system models/') >>> 7. IDL> print, memory(/l64, /current) >>> 412234 >>> 8. IDL> obj_destroy, test >>> 9. IDL> print, memory(/l64, /current) >>> -1077816 > >> Maybe is not negative, but just overflowing for >> 18446744073708473800LL... now that's a nice >> rig you have, just one megabyte short of 4 exabytes ;-)well, that should have been 16... > > > >> Ciao, >> Paolo > >>> If I repeat the process starting at 5 IDL blows up after a repetition >>> or two. If anyone has had this problem before and knows a solution I'd >>> be grateful. I couldn't find a solution searching the board, the IDL >>> help, or the ITTVIS IDL memory allocation FAQs. > >>> On the other hand, if anyone wants to buy a program that uses negative >>> memory this is your lucky day.
Some more information: I'm running 32 bit IDL 6.3 on Windows 2000 with only 512MB of non imaginary RAM. The file I'm reading and storing data from with this object is only about 2MB, and the Windows Task Manager does not show idlde.exe using unusual amounts of memory. If I step through my obj::destroy routine IDL locks up only after the final END statement or if I try to use /RETALL before the end of the cleanup routine. Once it is locked up I must use Task Manager to end idlde.exe. If instead of returning to $MAIN$ I type .reset I get 1 to 3 "The instruction at bla referenced memory at bla. The memory could not be read/written. Click on OK to terminate the program" messages and I'm back to the desktop. I'll find out what tech support can do for me here. On Dec 1, 8:27 am, "Pete Warner" <warner.p...@yahoo.com> wrote: > I guess I've been thinking about asking for an upgrade. Maybe I already > got it. Anybody know if Windows Vista can address 16 exabytes or will I > need Linux for that? > > On Dec 1, 2:53 am, Paolo Grigis <pgri...@astro.phys.ethz.ch> wrote: > >> Paolo Grigis wrote: > >>> Pete Warner wrote: > >>>> Hopefully one day I'll join the crowd of you helpful people who answer >>>> questions instead of coming up with them, but that day is not today. > >>>> I've got an object that should hold and organize a large number of >>>> arrays and pointers and arrays of pointers and then clean them up when >>>> I'm done. I know I have enough memory for the data in question because >>>> I can load it without using pointers. I want the objects and pointers >>>> for flexibility and to make the overall program easier to work with. > >>>> The problem is that I'm crashing IDL. I create the object, and then >>>> destroy it to check for leaks. Using help, /heap_variables I show 0 >>>> pointers and objects. Then if I create and destroy it again IDL locks >>>> up or crashes to desktop with some memory errors. I figured there must >>>> be crazy leaking going on but I can't find it. > >>>> 1. Restart computer >>>> 2. Load IDL >>>> 3. Load project with objects >>>> 4. Compile project >>>> 5. IDL> print, memory(/l64, /current) >>>> 845185 >>>> 6. IDL> test = obj_new('mtgv_lodat', 'model.txt', >>>> 'c:/rsi/projects/gicvu idl/system models/') >>>> 7. IDL> print, memory(/l64, /current) >>>> 412234 >>>> 8. IDL> obj_destroy, test >>>> 9. IDL> print, memory(/l64, /current) >>>> -1077816 > >>> Maybe is not negative, but just overflowing for >>> 18446744073708473800LL... now that's a nice >>> rig you have, just one megabyte short of 4 exabytes ;-)well, that should have been 16... > >>> Ciao, >>> Paolo > >>>> If I repeat the process starting at 5 IDL blows up after a repetition >>>> or two. If anyone has had this problem before and knows a solution I'd >>>> be grateful. I couldn't find a solution searching the board, the IDL >>>> help, or the ITTVIS IDL memory allocation FAQs. > >>>> On the other hand, if anyone wants to buy a program that uses negative >>>> memory this is your lucky day.
Pete Warner writes: > I'm running 32 bit IDL 6.3 on Windows 2000 with only 512MB of non > imaginary RAM. The file I'm reading and storing data from with this > object is only about 2MB, and the Windows Task Manager does not show > idlde.exe using unusual amounts of memory. Well, this explains why the L64 keyword is acting strangely, as this is *expressly* for the 64-bit version of IDL. > I'll find out what tech support can do for me here. I'm back to pilot error as the most likely explanation, but a bug in memory management certainly can't be ruled out. :-) Cheers, David -- David Fanning, Ph.D. Fanning Software Consulting, Inc. Coyote's Guide to IDL Programming: http://www.dfanning.com/ Sepore ma de ni thui. ("Perhaps thou speakest truth.")
Just to be sure there's no hardware problem, you might want to do a tough memory test (e.g. memtest86) to see if the physical RAM is ok... Ciao, Paolo Pete Warner wrote: > Some more information: > I'm running 32 bit IDL 6.3 on Windows 2000 with only 512MB of non > imaginary RAM. The file I'm reading and storing data from with this > object is only about 2MB, and the Windows Task Manager does not show > idlde.exe using unusual amounts of memory. > > If I step through my obj::destroy routine IDL locks up only after the > final END statement or if I try to use /RETALL before the end of the > cleanup routine. Once it is locked up I must use Task Manager to end > idlde.exe. > > If instead of returning to $MAIN$ I type .reset I get 1 to 3 "The > instruction at bla referenced memory at bla. The memory could not be > read/written. Click on OK to terminate the program" messages and I'm > back to the desktop. > > I'll find out what tech support can do for me here. > > > On Dec 1, 8:27 am, "Pete Warner" <warner.p...@yahoo.com> wrote: > >> I guess I've been thinking about asking for an upgrade. Maybe I already >> got it. Anybody know if Windows Vista can address 16 exabytes or will I >> need Linux for that? >> >> On Dec 1, 2:53 am, Paolo Grigis <pgri...@astro.phys.ethz.ch> wrote: >> >> >>> Paolo Grigis wrote: >> >>>> Pete Warner wrote: >> >>>> >Hopefully one day I'll join the crowd of you helpful people who answer >>>> >questions instead of coming up with them, but that day is not today. >> >>>> >I've got an object that should hold and organize a large number of >>>> >arrays and pointers and arrays of pointers and then clean them up when >>>> >I'm done. I know I have enough memory for the data in question because >>>> >I can load it without using pointers. I want the objects and pointers >>>> >for flexibility and to make the overall program easier to work with. >> >>>> >The problem is that I'm crashing IDL. I create the object, and then >>>> >destroy it to check for leaks. Using help, /heap_variables I show 0 >>>> >pointers and objects. Then if I create and destroy it again IDL locks >>>> >up or crashes to desktop with some memory errors. I figured there must >>>> >be crazy leaking going on but I can't find it. >> >>>> >1. Restart computer >>>> >2. Load IDL >>>> >3. Load project with objects >>>> >4. Compile project >>>> >5. IDL> print, memory(/l64, /current) >>>> > 845185 >>>> >6. IDL> test = obj_new('mtgv_lodat', 'model.txt', >>>> >'c:/rsi/projects/gicvu idl/system models/') >>>> >7. IDL> print, memory(/l64, /current) >>>> > 412234 >>>> >8. IDL> obj_destroy, test >>>> >9. IDL> print, memory(/l64, /current) >>>> > -1077816 >> >>>> Maybe is not negative, but just overflowing for >>>> 18446744073708473800LL... now that's a nice >>>> rig you have, just one megabyte short of 4 exabytes ;-)well, that should have been 16... >> >>>> Ciao, >>>> Paolo >> >>>> >If I repeat the process starting at 5 IDL blows up after a repetition >>>> >or two. If anyone has had this problem before and knows a solution I'd >>>> >be grateful. I couldn't find a solution searching the board, the IDL >>>> >help, or the ITTVIS IDL memory allocation FAQs. >> >>>> >On the other hand, if anyone wants to buy a program that uses negative >>>> >memory this is your lucky day. > >
Here's a program that crashes me every time. The key part seems to be the "file_search" followed by the "and" statement. Take either out and it doesn't cause me trouble. I'm creating the .txt file because I don't know what to search for on your computers, and file_search needs to find something. I run between 1 and 10 loops before IDL dies on my computer. Tested on 6.1 and 6.3. pro testfailure a = 'test_file.txt' openw, 1, a close, 1 free_lun, 1 key = 'b' count = 0 print, 'Starting Memory: ', memory(/current) while (key ne 'a') do begin count++ b = file_search(a) print, 'anystring' and b print, 'Memory after iteration ', count, ', ', memory(/current) print, 'Hit "a" to exit, anykey to continue' key = get_kbrd(1) endwhile print, 'ending' end On Dec 1, 9:42 am, Paolo Grigis <pgri...@astro.phys.ethz.ch> wrote: > Just to be sure there's no hardware problem, you might > want to do a tough memory test (e.g. memtest86) to see > if the physical RAM is ok... > > Ciao, > Paolo > > Pete Warner wrote: >> Some more information: >> I'm running 32 bit IDL 6.3 on Windows 2000 with only 512MB of non >> imaginary RAM. The file I'm reading and storing data from with this >> object is only about 2MB, and the Windows Task Manager does not show >> idlde.exe using unusual amounts of memory. > >> If I step through my obj::destroy routine IDL locks up only after the >> final END statement or if I try to use /RETALL before the end of the >> cleanup routine. Once it is locked up I must use Task Manager to end >> idlde.exe. > >> If instead of returning to $MAIN$ I type .reset I get 1 to 3 "The >> instruction at bla referenced memory at bla. The memory could not be >> read/written. Click on OK to terminate the program" messages and I'm >> back to the desktop. > >> I'll find out what tech support can do for me here. > >> On Dec 1, 8:27 am, "Pete Warner" <warner.p...@yahoo.com> wrote: > >>> I guess I've been thinking about asking for an upgrade. Maybe I already >>> got it. Anybody know if Windows Vista can address 16 exabytes or will I >>> need Linux for that? > >>> On Dec 1, 2:53 am, Paolo Grigis <pgri...@astro.phys.ethz.ch> wrote: > >>>> Paolo Grigis wrote: > >>>> >Pete Warner wrote: > >>>> >>Hopefully one day I'll join the crowd of you helpful people who answer >>>> >>questions instead of coming up with them, but that day is not today. > >>>> >>I've got an object that should hold and organize a large number of >>>> >>arrays and pointers and arrays of pointers and then clean them up when >>>> >>I'm done. I know I have enough memory for the data in question because >>>> >>I can load it without using pointers. I want the objects and pointers >>>> >>for flexibility and to make the overall program easier to work with. > >>>> >>The problem is that I'm crashing IDL. I create the object, and then >>>> >>destroy it to check for leaks. Using help, /heap_variables I show 0 >>>> >>pointers and objects. Then if I create and destroy it again IDL locks >>>> >>up or crashes to desktop with some memory errors. I figured there must >>>> >>be crazy leaking going on but I can't find it. > >>>> >>1. Restart computer >>>> >>2. Load IDL >>>> >>3. Load project with objects >>>> >>4. Compile project >>>> >>5. IDL> print, memory(/l64, /current) >>>> >> 845185 >>>> >>6. IDL> test = obj_new('mtgv_lodat', 'model.txt', >>>> >>'c:/rsi/projects/gicvu idl/system models/') >>>> >>7. IDL> print, memory(/l64, /current) >>>> >> 412234 >>>> >>8. IDL> obj_destroy, test >>>> >>9. IDL> print, memory(/l64, /current) >>>> >> -1077816 > >>>> >Maybe is not negative, but just overflowing for >>>> >18446744073708473800LL... now that's a nice >>>> >rig you have, just one megabyte short of 4 exabytes ;-)well, that should have been 16... > >>>> >Ciao, >>>> >Paolo > >>>> >>If I repeat the process starting at 5 IDL blows up after a repetition >>>> >>or two. If anyone has had this problem before and knows a solution I'd >>>> >>be grateful. I couldn't find a solution searching the board, the IDL >>>> >>help, or the ITTVIS IDL memory allocation FAQs. > >>>> >>On the other hand, if anyone wants to buy a program that uses negative >>>> >>memory this is your lucky day.
On Fri, 1 Dec 2006, Pete Warner wrote: > Here's a program that crashes me every time. The key part seems to be > the "file_search" followed by the "and" statement. Take either out and > it doesn't cause me trouble. I'm creating the .txt file because I don't > know what to search for on your computers, and file_search needs to > find something. I run between 1 and 10 loops before IDL dies on my > computer. Tested on 6.1 and 6.3. > > pro testfailure > > a = 'test_file.txt' > openw, 1, a > close, 1 > free_lun, 1 > > key = 'b' > count = 0 > print, 'Starting Memory: ', memory(/current) > while (key ne 'a') do begin > count++ > b = file_search(a) > print, 'anystring' and b > print, 'Memory after iteration ', count, ', ', memory(/current) > print, 'Hit "a" to exit, anykey to continue' > key = get_kbrd(1) > endwhile > print, 'ending' > > end > valgrind shows invalid read/free, so it is definitely an IDL bug. regards, lajos $ valgrind /usr/local/rsi/idl/bin/bin.linux.x86/idl ==26303== Memcheck, a memory error detector. ==26303== Copyright (C) 2002-2006, and GNU GPL'd, by Julian Seward et al. ==26303== Using LibVEX rev 1658, a library for dynamic binary translation. ==26303== Copyright (C) 2004-2006, and GNU GPL'd, by OpenWorks LLP. ==26303== Using valgrind-3.2.1, a dynamic binary instrumentation framework. ==26303== Copyright (C) 2000-2006, and GNU GPL'd, by Julian Seward et al. ==26303== For more details, rerun with: -v ==26303== IDL Version 6.3 (linux x86 m32). (c) 2006, Research Systems, Inc. Installation number: 8209. Licensed for use by: KFKI RMKI IDL> testfailure testfailure % Compiled module: TESTFAILURE. Starting Memory: 580997 test_file.txt Memory after iteration 1, 581085 Hit "a" to exit, anykey to continue : ==26303== Invalid read of size 4 ==26303== at 0x40DDE05: IDL_MemFreeMSG_LONGJMP (in /usr/local/rsi/idl_6.3/bin/bin.linux.x86/libidl.so.6.3) ==26303== by 0x42788F7: IDL_StrDelete (in /usr/local/rsi/idl_6.3/bin/bin.linux.x86/libidl.so.6.3) ==26303== by 0x42A1B80: IDL_Delvar (in /usr/local/rsi/idl_6.3/bin/bin.linux.x86/libidl.so.6.3) ==26303== by 0x4293E73: IDL_VarCopy (in /usr/local/rsi/idl_6.3/bin/bin.linux.x86/libidl.so.6.3) ==26303== by 0x42A7084: _IDL_interpreter (in /usr/local/rsi/idl_6.3/bin/bin.linux.x86/libidl.so.6.3) ==26303== by 0x40EE093: IDL_Executive (in /usr/local/rsi/idl_6.3/bin/bin.linux.x86/libidl.so.6.3) ==26303== by 0x4485D5A: IDL_Main (in /usr/local/rsi/idl_6.3/bin/bin.linux.x86/libidl.so.6.3) ==26303== by 0x8051CB8: main (in /usr/local/rsi/idl_6.3/bin/bin.linux.x86/idl) ==26303== Address 0x51B6A68 is 0 bytes inside a block of size 18 free'd ==26303== at 0x401C005: free (vg_replace_malloc.c:233) ==26303== by 0x40DDE21: IDL_MemFreeMSG_LONGJMP (in /usr/local/rsi/idl_6.3/bin/bin.linux.x86/libidl.so.6.3) ==26303== by 0x42788F7: IDL_StrDelete (in /usr/local/rsi/idl_6.3/bin/bin.linux.x86/libidl.so.6.3) ==26303== by 0x42A1B80: IDL_Delvar (in /usr/local/rsi/idl_6.3/bin/bin.linux.x86/libidl.so.6.3) ==26303== by 0x428AEC6: IDL_Deltmp (in /usr/local/rsi/idl_6.3/bin/bin.linux.x86/libidl.so.6.3) ==26303== by 0x42A6753: _IDL_interpreter (in /usr/local/rsi/idl_6.3/bin/bin.linux.x86/libidl.so.6.3) ==26303== by 0x40EE093: IDL_Executive (in /usr/local/rsi/idl_6.3/bin/bin.linux.x86/libidl.so.6.3) ==26303== by 0x4485D5A: IDL_Main (in /usr/local/rsi/idl_6.3/bin/bin.linux.x86/libidl.so.6.3) ==26303== ==26303== Invalid free() / delete / delete[] ==26303== at 0x401C005: free (vg_replace_malloc.c:233) ==26303== by 0x40DDE21: IDL_MemFreeMSG_LONGJMP (in /usr/local/rsi/idl_6.3/bin/bin.linux.x86/libidl.so.6.3) ==26303== by 0x42788F7: IDL_StrDelete (in /usr/local/rsi/idl_6.3/bin/bin.linux.x86/libidl.so.6.3) ==26303== by 0x42A1B80: IDL_Delvar (in /usr/local/rsi/idl_6.3/bin/bin.linux.x86/libidl.so.6.3) ==26303== by 0x4293E73: IDL_VarCopy (in /usr/local/rsi/idl_6.3/bin/bin.linux.x86/libidl.so.6.3) ==26303== by 0x42A7084: _IDL_interpreter (in /usr/local/rsi/idl_6.3/bin/bin.linux.x86/libidl.so.6.3) ==26303== by 0x40EE093: IDL_Executive (in /usr/local/rsi/idl_6.3/bin/bin.linux.x86/libidl.so.6.3) ==26303== by 0x4485D5A: IDL_Main (in /usr/local/rsi/idl_6.3/bin/bin.linux.x86/libidl.so.6.3) ==26303== Address 0x51B6A68 is 0 bytes inside a block of size 18 free'd ==26303== at 0x401C005: free (vg_replace_malloc.c:233) ==26303== by 0x40DDE21: IDL_MemFreeMSG_LONGJMP (in /usr/local/rsi/idl_6.3/bin/bin.linux.x86/libidl.so.6.3) ==26303== by 0x42788F7: IDL_StrDelete (in /usr/local/rsi/idl_6.3/bin/bin.linux.x86/libidl.so.6.3) ==26303== by 0x42A1B80: IDL_Delvar (in /usr/local/rsi/idl_6.3/bin/bin.linux.x86/libidl.so.6.3) ==26303== by 0x428AEC6: IDL_Deltmp (in /usr/local/rsi/idl_6.3/bin/bin.linux.x86/libidl.so.6.3) ==26303== by 0x42A6753: _IDL_interpreter (in /usr/local/rsi/idl_6.3/bin/bin.linux.x86/libidl.so.6.3) ==26303== by 0x40EE093: IDL_Executive (in /usr/local/rsi/idl_6.3/bin/bin.linux.x86/libidl.so.6.3) ==26303== by 0x4485D5A: IDL_Main (in /usr/local/rsi/idl_6.3/bin/bin.linux.x86/libidl.so.6.3) test_file.txt Memory after iteration 2, 581073 Hit "a" to exit, anykey to continue :
Pete Warner wrote: > Here's a program that crashes me every time. The key part seems to be > the "file_search" followed by the "and" statement. Take either out and > it doesn't cause me trouble. I'm creating the .txt file because I don't > know what to search for on your computers, and file_search needs to > find something. I run between 1 and 10 loops before IDL dies on my > computer. Tested on 6.1 and 6.3. even a 1 line IDL> a = file_search() & print, 'test ' and a makes IDL crash... IDL> print, !version { x86 Win32 Windows Microsoft Windows 6.3 Mar 23 2006 32 64} Jean > > pro testfailure > > a = 'test_file.txt' > openw, 1, a > close, 1 > free_lun, 1 > > key = 'b' > count = 0 > print, 'Starting Memory: ', memory(/current) > while (key ne 'a') do begin > count++ > b = file_search(a) > print, 'anystring' and b > print, 'Memory after iteration ', count, ', ', memory(/current) > print, 'Hit "a" to exit, anykey to continue' > key = get_kbrd(1) > endwhile > print, 'ending' > > end > > On Dec 1, 9:42 am, Paolo Grigis <pgri...@astro.phys.ethz.ch> wrote: > >> Just to be sure there's no hardware problem, you might >> want to do a tough memory test (e.g. memtest86) to see >> if the physical RAM is ok... >> >> Ciao, >> Paolo >> >> Pete Warner wrote: >> >>> Some more information: >>> I'm running 32 bit IDL 6.3 on Windows 2000 with only 512MB of non >>> imaginary RAM. The file I'm reading and storing data from with this >>> object is only about 2MB, and the Windows Task Manager does not show >>> idlde.exe using unusual amounts of memory. >> >>> If I step through my obj::destroy routine IDL locks up only after the >>> final END statement or if I try to use /RETALL before the end of the >>> cleanup routine. Once it is locked up I must use Task Manager to end >>> idlde.exe. >> >>> If instead of returning to $MAIN$ I type .reset I get 1 to 3 "The >>> instruction at bla referenced memory at bla. The memory could not be >>> read/written. Click on OK to terminate the program" messages and I'm >>> back to the desktop. >> >>> I'll find out what tech support can do for me here. >> >>> On Dec 1, 8:27 am, "Pete Warner" <warner.p...@yahoo.com> wrote: >> >>>> I guess I've been thinking about asking for an upgrade. Maybe I already >>>> got it. Anybody know if Windows Vista can address 16 exabytes or will I >>>> need Linux for that? >> >>>> On Dec 1, 2:53 am, Paolo Grigis <pgri...@astro.phys.ethz.ch> wrote: >> >>>> >Paolo Grigis wrote: >> >>>> >>Pete Warner wrote: >> >>>> >>>Hopefully one day I'll join the crowd of you helpful people who answer >>>> >>>questions instead of coming up with them, but that day is not today. >> >>>> >>>I've got an object that should hold and organize a large number of >>>> >>>arrays and pointers and arrays of pointers and then clean them up when >>>> >>>I'm done. I know I have enough memory for the data in question because >>>> >>>I can load it without using pointers. I want the objects and pointers >>>> >>>for flexibility and to make the overall program easier to work with. >> >>>> >>>The problem is that I'm crashing IDL. I create the object, and then >>>> >>>destroy it to check for leaks. Using help, /heap_variables I show 0 >>>> >>>pointers and objects. Then if I create and destroy it again IDL locks >>>> >>>up or crashes to desktop with some memory errors. I figured there must >>>> >>>be crazy leaking going on but I can't find it. >> >>>> >>>1. Restart computer >>>> >>>2. Load IDL >>>> >>>3. Load project with objects >>>> >>>4. Compile project >>>> >>>5. IDL> print, memory(/l64, /current) >>>> >>> 845185 >>>> >>>6. IDL> test = obj_new('mtgv_lodat', 'model.txt', >>>> >>>'c:/rsi/projects/gicvu idl/system models/') >>>> >>>7. IDL> print, memory(/l64, /current) >>>> >>> 412234 >>>> >>>8. IDL> obj_destroy, test >>>> >>>9. IDL> print, memory(/l64, /current) >>>> >>> -1077816 >> >>>> >>Maybe is not negative, but just overflowing for >>>> >>18446744073708473800LL... now that's a nice >>>> >>rig you have, just one megabyte short of 4 exabytes ;-)well, that should have been 16... >> >>>> >>Ciao, >>>> >>Paolo >> >>>> >>>If I repeat the process starting at 5 IDL blows up after a repetition >>>> >>>or two. If anyone has had this problem before and knows a solution I'd >>>> >>>be grateful. I couldn't find a solution searching the board, the IDL >>>> >>>help, or the ITTVIS IDL memory allocation FAQs. >> >>>> >>>On the other hand, if anyone wants to buy a program that uses negative >>>> >>>memory this is your lucky day. > >
On Fri, 01 Dec 2006 11:41:08 -0700, Jean H. wrote: > Pete Warner wrote: >> Here's a program that crashes me every time. The key part seems to be >> the "file_search" followed by the "and" statement. Take either out and >> it doesn't cause me trouble. I'm creating the .txt file because I don't >> know what to search for on your computers, and file_search needs to >> find something. I run between 1 and 10 loops before IDL dies on my >> computer. Tested on 6.1 and 6.3. > > even a 1 line > IDL> a = file_search() & print, 'test ' and a > makes IDL crash... > IDL> print, !version > { x86 Win32 Windows Microsoft Windows 6.3 Mar 23 2006 32 64} > > Jean Same result here for IDL Version 6.3 (linux x86 m32). GLIBC catches a double free: *** glibc detected *** /usr/local/rsi/idl_6.3/bin/bin.linux.x86/idl: double free or corruption (out): 0x08f8c7f0 *** ======= Backtrace: ========= /lib/libc.so.6[0xc031e0] /lib/libc.so.6(__libc_free+0x77)[0xc0372b] /usr/local/rsi/idl_6.3/bin/bin.linux.x86/libidl.so.6.3(IDL_M emFreeMSG_LONGJMP+0x3f)[0x452e22] ... Should we assume ITTVIS is reading? JD
JD Smith writes: > Should we assume ITTVIS is reading? No. :-) Cheers, David -- David Fanning, Ph.D. Fanning Software Consulting, Inc. Coyote's Guide to IDL Programming: http://www.dfanning.com/ Sepore ma de ni thui. ("Perhaps thou speakest truth.")
On Fri, 01 Dec 2006 11:41:08 -0700, Jean H. wrote: > Pete Warner wrote: >> Here's a program that crashes me every time. The key part seems to be >> the "file_search" followed by the "and" statement. Take either out and >> it doesn't cause me trouble. I'm creating the .txt file because I don't >> know what to search for on your computers, and file_search needs to >> find something. I run between 1 and 10 loops before IDL dies on my >> computer. Tested on 6.1 and 6.3. > > even a 1 line > IDL> a = file_search() & print, 'test ' and a > makes IDL crash... > IDL> print, !version > { x86 Win32 Windows Microsoft Windows 6.3 Mar 23 2006 32 64} > > Jean Ouch. Ok I'll open a bug report and we'll try to fix it for next release. I've spent a little time looking for a workaround and didn't have much luck. About all I can suggest is looking at how the AND operator works on non-numeric values and maybe changing your code to do the same thing without using AND. Docs say: For operations on other types, the result is equal to the second operand if the first operand is not equal to zero or the null string; otherwise, the result is zero or the null string. So you might replace: print, 'anystring' and b with print, 'anystring' ne '' ? b : '' Now 'anystring' ne '' doesn't make sense unless 'anystring' is another variable. Entire program, with a workaround: pro testfailure a = 'test_file.txt' openw, 1, a close, 1 free_lun, 1 key = 'b' any = 'anystring' count = 0 print, 'Starting Memory: ', memory(/current) while (key ne 'a') do begin count++ b = file_search(a) print, any ne '' ? b : '' print, 'Memory after iteration ', count, ', ', memory(/current) print, 'Hit "a" to exit, anykey to continue' key = get_kbrd(1) endwhile print, 'ending' end
The whole problem started when I replaced file_test() with file_search() and then forgot to change the checking line from if file_a_good and file_b_good then... to if (file_a ne '') and (file_b ne '') then... I got obsessed with the thought that it must be an object problem because I don't have much experience there and it looked like I must be leaking memory somewhere. When stepping through my code the crash never happened immediately after the bad "and", but maybe that was just luck. Thanks everybody. Karl, I have been in contact with RSI support. How can I make sure there aren't two different people working on this separately? I'll send them the latest from the group here. On Dec 1, 2:19 pm, "Karl Schultz" <k_remove_schu...@ittvis.com> wrote: > On Fri, 01 Dec 2006 11:41:08 -0700, Jean H. wrote: >> Pete Warner wrote: >>> Here's a program that crashes me every time. The key part seems to be >>> the "file_search" followed by the "and" statement. Take either out and >>> it doesn't cause me trouble. I'm creating the .txt file because I don't >>> know what to search for on your computers, and file_search needs to >>> find something. I run between 1 and 10 loops before IDL dies on my >>> computer. Tested on 6.1 and 6.3. > >> even a 1 line >> IDL> a = file_search() & print, 'test ' and a >> makes IDL crash... >> IDL> print, !version >> { x86 Win32 Windows Microsoft Windows 6.3 Mar 23 2006 32 64} > >> JeanOuch. Ok I'll open a bug report and we'll try to fix it for next release. > > I've spent a little time looking for a workaround and didn't have much > luck. > > About all I can suggest is looking at how the AND operator works on > non-numeric values and maybe changing your code to do the same thing > without using AND. > > Docs say: > > For operations on other types, the result is equal to the second > operand if the first operand is not equal to zero or the null string; > otherwise, the result is zero or the null string. > > So you might replace: > > print, 'anystring' and b > > with > > print, 'anystring' ne '' ? b : '' > > Now 'anystring' ne '' doesn't make sense unless 'anystring' is another > variable. > > Entire program, with a workaround: > > pro testfailure > > a = 'test_file.txt' > openw, 1, a > close, 1 > free_lun, 1 > > key = 'b' > any = 'anystring' > count = 0 > print, 'Starting Memory: ', memory(/current) > while (key ne 'a') do begin > count++ > b = file_search(a) > print, any ne '' ? b : '' > print, 'Memory after iteration ', count, ', ', memory(/current) > print, 'Hit "a" to exit, anykey to continue' > key = get_kbrd(1) > endwhile > print, 'ending' > > end
"Pete Warner" <warner.pete@yahoo.com> wrote in message news:1164983221.424081.106940@f1g2000cwa.googlegroups.com... > I guess I've been thinking about asking for an upgrade. Maybe I already > got it. Anybody know if Windows Vista can address 16 exabytes or will I > need Linux for that? Has mankind even measured 16 exabytes of data in all of human history? I wonder what the spectrum of that looks like?
R.G. Stockwell wrote: > "Pete Warner" <warner.pete@yahoo.com> wrote in message > news:1164983221.424081.106940@f1g2000cwa.googlegroups.com... >> I guess I've been thinking about asking for an upgrade. Maybe I already >> got it. Anybody know if Windows Vista can address 16 exabytes or will I >> need Linux for that? > > Has mankind even measured 16 exabytes of data in all of human history? > I wonder what the spectrum of that looks like? I know it's O(6) less, but I was in a meeting the other day when a colleague walked in with a 4TB portable HDD (containing global reanalysis "nature run" data) and casually mentioned that the other four disks will be arriving soon. It wasn't that long ago that a terabyte was something that involved a sizeable budget and a tape robot.... now you buy multiples at the local Circuit City. I also remember thinking how on earth could I *ever* fill the 20MB hard drive on my new 8088-CPU PC back in the 80's.... Maybe exabyte HDD's aren't too far away once some new physics is discovered (writing into the other 8 or so spatial dimensions of the universe? :o) -- Paul van Delst Ride lots. CIMSS @ NOAA/NCEP/EMC Eddy Merckx Ph: (301)763-8000 x7748 Fax:(301)763-8545
"Paul van Delst" <Paul.vanDelst@noaa.gov> wrote in message news:ekq7e2$u4o$1@news.nems.noaa.gov... > R.G. Stockwell wrote: >> "Pete Warner" <warner.pete@yahoo.com> wrote in message >> news:1164983221.424081.106940@f1g2000cwa.googlegroups.com... >>> I guess I've been thinking about asking for an upgrade. Maybe I already >>> got it. Anybody know if Windows Vista can address 16 exabytes or will I >>> need Linux for that? >> >> Has mankind even measured 16 exabytes of data in all of human history? >> I wonder what the spectrum of that looks like? > > I know it's O(6) less, but I was in a meeting the other day when a > colleague walked in with a 4TB portable HDD (containing global reanalysis > "nature run" data) and casually mentioned that the other four disks will > be arriving soon. NICE! anyways, to throw out some numbers, 16 exabytes would be roughly equivalent to making a measurement for every square kilometer on earth, every second, for the past 1000 years. Or equivalently, for 100 heights at each kilometer each second over the past 10 years. That would be awesome.
R.G. Stockwell wrote: > "Paul van Delst" <Paul.vanDelst@noaa.gov> wrote in message > news:ekq7e2$u4o$1@news.nems.noaa.gov... >> R.G. Stockwell wrote: >>> "Pete Warner" <warner.pete@yahoo.com> wrote in message >>> news:1164983221.424081.106940@f1g2000cwa.googlegroups.com... >>>> I guess I've been thinking about asking for an upgrade. Maybe I already >>>> got it. Anybody know if Windows Vista can address 16 exabytes or will I >>>> need Linux for that? >>> Has mankind even measured 16 exabytes of data in all of human history? >>> I wonder what the spectrum of that looks like? >> I know it's O(6) less, but I was in a meeting the other day when a >> colleague walked in with a 4TB portable HDD (containing global reanalysis >> "nature run" data) and casually mentioned that the other four disks will >> be arriving soon. > > NICE! > > anyways, to throw out some numbers, 16 exabytes would be roughly > equivalent to making a measurement for every square kilometer on earth, > every second, for the past 1000 years. Or equivalently, for 100 heights at > each kilometer each second over the past 10 years. That would be awesome. Wow. I never thought about it like that. If you took into account the "spreading" of data as the height increased (i.e. increased the spatial density to ensure at least 1 sq.km coverage and each 100 levels) and decreased the temporal sampling to, say, every five minutes (so you could measure more things like the temperature, h2o, o3, co2, ch4, no2, so2, etc, cloud water content, cloud ice content, cloud particle size, shape would be nice too, aerosol concentration for dust, woc,doc,wbc,dbc, sea salt, and sulphites, etc.. etc.. along with all the surface property measurements) *that* would be a neato dataset. Only problem is all the wires would get in the way of the weather. :oD -- Paul van Delst Ride lots. CIMSS @ NOAA/NCEP/EMC Eddy Merckx Ph: (301)763-8000 x7748 Fax:(301)763-8545
On Fri, 1 Dec 2006, R.G. Stockwell wrote: > Has mankind even measured 16 exabytes of data in all of human history? > I wonder what the spectrum of that looks like? > The LHC (Large Hadron Collider) at CERN will produce about 12-14 Petabytes of data yearly. This is filtered data, only very small part of the raw datastream. regards, lajos
"F�LDY Lajos" <foldy@rmki.kfki.hu> wrote in message news:Pine.LNX.4.64.0612021048440.8135@bifur.rmki.kfki.hu... > > On Fri, 1 Dec 2006, R.G. Stockwell wrote: > >> Has mankind even measured 16 exabytes of data in all of human history? >> I wonder what the spectrum of that looks like? >> > > The LHC (Large Hadron Collider) at CERN will produce about 12-14 Petabytes > of data yearly. This is filtered data, only very small part of the raw > datastream. > > regards, > lajos And after looking up petabytes, found this tidbit: "the Google cluster may now have 4 petabytes of RAM" sweet