| Allocating memory in LINKIMAGE program [message #20680] |
Wed, 19 July 2000 00:00  |
Virginia Rogers
Messages: 4
Registered: July 2000
|
Junior Member |
|
|
I have a C program which queries a database and returns a list of all matching
entries. I am making this routine callable in IDL with LINKIMAGE. I'd like to
return a string array to the calling IDL program. My question is, how do I
allocate the memory to store the strings? The C program has allocated space for
all of the strings with malloc(). Can I call IDL_MakeTempArray() to create the
string array and then set each of the string pointers to point to the location
allocated by malloc()? Is there another source of information about using
LINKIMAGE other than the IDL Advanced Development Guide?
Thanks! Ginny
|
|
|
|
| Re: Allocating memory in LINKIMAGE program [message #20761 is a reply to message #20680] |
Thu, 20 July 2000 00:00  |
Randall Frank
Messages: 17
Registered: October 1999
|
Junior Member |
|
|
Virginia,
You can use something like this to return an array of (n)
strings:
s = (IDL_STRING *)IDL_MakeTempVector(IDL_TYP_STRING,
n, IDL_BARR_INI_ZERO, &vpTmp);
for(i=0;i<n;i++) IDL_StrStore(s++,YourCStringArray[i]);
IDL_VarCopy(vpTmp,argv[0]);
In general, do not pass memory you allocated with malloc back
to IDL, especially if you want IDL to clean it up later for you.
Hope it helps.
Virginia Rogers wrote:
>
> I have a C program which queries a database and returns a list of all matching
> entries. I am making this routine callable in IDL with LINKIMAGE. I'd like to
> return a string array to the calling IDL program. My question is, how do I
> allocate the memory to store the strings? The C program has allocated space for
> all of the strings with malloc(). Can I call IDL_MakeTempArray() to create the
> string array and then set each of the string pointers to point to the location
> allocated by malloc()? Is there another source of information about using
> LINKIMAGE other than the IDL Advanced Development Guide?
>
> Thanks! Ginny
--
rjf.
Randy Frank | ASCI Visualization
Lawrence Livermore National Laboratory | rjfrank@llnl.gov
B451 Room 2039 L-561 | Voice: (925) 423-9399
Livermore, CA 94550 | Fax: (925) 423-8704
|
|
|
|
| Re: allocating memory [message #56346 is a reply to message #20680] |
Thu, 18 October 2007 11:52 |
Karl Schultz
Messages: 341
Registered: October 1999
|
Senior Member |
|
|
R.G. Stockwell <noemail@please.com> wrote:
>
> <jtmcahill@gmail.com> wrote in message
> news:1192673434.544923.139390@i38g2000prf.googlegroups.com.. .
>> So, I have this large model file that I need to open. Although it is
>> 650MB I should be able to open it in IDL on my pc computer which has
>> ~4GB of Memory. However, it keeps telling me insufficient memory.
>> However, if I try to open it in IDL on a linux machine with ~2GB
>> memory I can open it no problem. Is there a way to make my windows
>> based pc cooperate and allow me to open this file that should be no
>> problem to open?
>>
>> Thanks,
>> Hawaiianite
>
>
> The problem may be fragmentation of your ram by the many
> dlls loaded by windows and other programs. I'd remove everything
> you can from the startup (and other automatically loading programs)
> and reboot (remove spyware, antivirus, firewalls, mail programs, but
> be careful not to forget to turn them back on). That may help.
Each Windows process gets its own 32-bit virtual address space. So,
eliminating other processes won't help the IDL process in terms of
virtual address space fragmentation, but might help overall system
performance by reducing paging demand.
What you do need to look out for is various types of "junkware" that
causes extra DLL's to get loaded either for every process on the machine
or only for processes with windows. The Windows OS does a pretty good job
of placing system DLL's in the virtual address space of each process in
order to leave a big contiguous free space in the middle. But these
non-Microsoft add-ons are sometimes notorious for plopping themselves
into the middle of this free space, which directly effects the size
of the biggest chunk you can allocate later from IDL.
One of the biggest problem programs used to be the software that you
could optionally install with nVidia and ATI video cards. Some of these
"value-add" packages would add items and do-dads to the window title bar
to let you tweak things, etc. I found that by turning these off or rebasing
the offending DLL's, I could ease the problem. I don't know if these
sorts of programs are causing these problems today, but it's worth turning
off these extra gadgets just to see if it helps.
--
Karl Schultz kws@frii.com
There are 844,739 ways to enjoy a Waffle House hamburger.
|
|
|
|
| Re: allocating memory [message #56350 is a reply to message #20680] |
Thu, 18 October 2007 09:24 |
rtowler
Messages: 28
Registered: June 2006
|
Junior Member |
|
|
On Oct 18, 3:41 pm, Jean H wrote:
> Now, with windows, you can only have up to 2Gb of memory for any
> process... even if your computer has 42 Gb available!. There is a way to
> allocate up to 3 Gb, but this solution might make your system instable
> (I use it quite often here, and I never had any issue).
> You have to do 2 things: in your windows startup file (boot.ini), add
> the option /3gb ... I suggest you not to modify the curent entry, but
> to add a new one so if your system doesn't like it, it is easy to use
> the normal configuration. Your entry should look somewhat like this:
> multi(0)disk(0)rdisk(0)partition(2)\WINDOWS="Microsoft Windows XP
> Professional 3G" /noexecute=optin /fastdetect /3GB
It is also worth noting that you can use the /USERVA switch to tune
the amount of RAM that is available to "large address aware"
applications. That extra GB comes at the expense of RAM available for
the kernel and some applications can run into issues when the kernel
is squeezed. I specifically had issues with the LANDesk remote
management tools on a machine with 4GB and I had issues with
enumerating files on network shares on a machine with 3GB of RAM.
Giving back a 100 MB or so by setting /USERVA=2900 solved the issues
in both cases.
By default I run my machine with the /3GB /USERVA=2900 switches and
have had no stability or compatibility issues to speak of.
-Rick
|
|
|
|
| Re: allocating memory [message #56351 is a reply to message #20680] |
Thu, 18 October 2007 08:41 |
Jean H.
Messages: 472
Registered: July 2006
|
Senior Member |
|
|
krastoboy@gmail.com wrote:
> I've the same problem!
> I'm working with an image of 1.8 G. on a machine with 4GB of memory.
> I've tried to use an half of the image but the problem still remains.
>
> Any suggestion?
>
> Thanks
> Lo
Hi,
You can use the procedure memtest from ITTVIS in order to find out the
size of the biggest array you could create.
Now, with windows, you can only have up to 2Gb of memory for any
process... even if your computer has 42 Gb available!. There is a way to
allocate up to 3 Gb, but this solution might make your system instable
(I use it quite often here, and I never had any issue).
You have to do 2 things: in your windows startup file (boot.ini), add
the option /3gb ... I suggest you not to modify the curent entry, but
to add a new one so if your system doesn't like it, it is easy to use
the normal configuration. Your entry should look somewhat like this:
multi(0)disk(0)rdisk(0)partition(2)\WINDOWS="Microsoft Windows XP
Professional 3G" /noexecute=optin /fastdetect /3GB
Restart your computer in the 3Gb mode.
Now you also have to modify the Idl executable... I did it with v6.2 and
v6.3 and it works perfectly. Again, make a copy of it, so you can choose
to launch IDL with up to 3gb of memory or with up to 2gb.
So, using visual studio (the free version from Microsoft works well),
you need to specify /LARGEADDRESSAWARE to the IDL executable (google
it, I don't remember what to do). Now run this new exe, and run memtest
again... you should see a big difference!
Personally, each time I need a lot of memory, I do it under Linux as
there is no silly memory limitations!
Jean
|
|
|
|
| Re: allocating memory [message #56354 is a reply to message #20680] |
Thu, 18 October 2007 08:37 |
R.G.Stockwell
Messages: 163
Registered: October 2004
|
Senior Member |
|
|
<jtmcahill@gmail.com> wrote in message
news:1192673434.544923.139390@i38g2000prf.googlegroups.com.. .
> So, I have this large model file that I need to open. Although it is
> 650MB I should be able to open it in IDL on my pc computer which has
> ~4GB of Memory. However, it keeps telling me insufficient memory.
> However, if I try to open it in IDL on a linux machine with ~2GB
> memory I can open it no problem. Is there a way to make my windows
> based pc cooperate and allow me to open this file that should be no
> problem to open?
>
> Thanks,
> Hawaiianite
I've attached a short program memtest.pro below. I grabbed this
off the newsgroup.
It shows you the memory sizes you can allocate.
The problem may be fragmentation of your ram by the many
dlls loaded by windows and other programs. I'd remove everything
you can from the startup (and other automatically loading programs)
and reboot (remove spyware, antivirus, firewalls, mail programs, but
be careful not to forget to turn them back on). That may help.
pro memtest
compile_opt idl2 ; set default integers to 32-bit and enforce [] for
indexing
MB = long64(2)^20
currentBlockSize = MB * 2047 ; 2 GB
print,'current block size = ',currentblocksize
maxIterations = 10 ; Max loop iterations
memPtrs = ptrarr(maxIterations)
memBlockSizes = ulonarr(maxIterations)
for i=0, maxIterations-1 do begin
; Error handler
catch, err
; Sepcifically designed for "Failure to allocate memory..." error
if (err ne 0) then begin
currentBlockSize = currentBlockSize - MB ; ...try 1 MB smaller
allocation
if (currentBlockSize lt MB) then break ; Give up, if currentBlockSize
< 1 MB
endif
; This 'wait' enables Ctrl-Break to interrupt if necessary (Windows).
wait, 0.0001
; Allocate memory (if possible)
memPtrs[i] = ptr_new(bytarr(currentBlockSize, /nozero), /no_copy)
memBlockSizes[i] = currentBlockSize ; Store the latest successful
allocation size
; Print the current allocated block size and the running total, in Mb
print, format='(%"Memory block #%2d: %6d Mb (total: %4d Mb)")', $
i + 1, ishft(currentBlockSize, -20),
ishft(ulong(total(memBlockSizes)), -20)
endfor
ptr_free,memPtrs
end
|
|
|
|
|
|
|
|
| Re: allocating memory [message #56422 is a reply to message #56354] |
Sat, 20 October 2007 01:14 |
Andrew Cool
Messages: 219
Registered: January 1996
|
Senior Member |
|
|
On Oct 19, 12:37 am, "R.G. Stockwell" <noem...@please.com> wrote:
> <jtmcah...@gmail.com> wrote in message
>
> news:1192673434.544923.139390@i38g2000prf.googlegroups.com.. .
>
>> So, I have this large model file that I need to open. Although it is
>> 650MB I should be able to open it in IDL on my pc computer which has
>> ~4GB of Memory. However, it keeps telling me insufficient memory.
>> However, if I try to open it in IDL on a linux machine with ~2GB
>> memory I can open it no problem. Is there a way to make my windows
>> based pc cooperate and allow me to open this file that should be no
>> problem to open?
>
>> Thanks,
>> Hawaiianite
>
> I've attached a short program memtest.pro below. I grabbed this
> off the newsgroup.
>
> It shows you the memory sizes you can allocate.
>
> The problem may be fragmentation of your ram by the many
> dlls loaded by windows and other programs. I'd remove everything
> you can from the startup (and other automatically loading programs)
> and reboot (remove spyware, antivirus, firewalls, mail programs, but
> be careful not to forget to turn them back on). That may help.
>
> pro memtest
> compile_opt idl2 ; set default integers to 32-bit and enforce [] for
> indexing
>
> MB = long64(2)^20
> currentBlockSize = MB * 2047 ; 2 GB
>
> print,'current block size = ',currentblocksize
> maxIterations = 10 ; Max loop iterations
> memPtrs = ptrarr(maxIterations)
> memBlockSizes = ulonarr(maxIterations)
>
> for i=0, maxIterations-1 do begin
> ; Error handler
> catch, err
>
> ; Sepcifically designed for "Failure to allocate memory..." error
> if (err ne 0) then begin
> currentBlockSize = currentBlockSize - MB ; ...try 1 MB smaller
> allocation
> if (currentBlockSize lt MB) then break ; Give up, if currentBlockSize
> < 1 MB
> endif
>
> ; This 'wait' enables Ctrl-Break to interrupt if necessary (Windows).
> wait, 0.0001
>
> ; Allocate memory (if possible)
> memPtrs[i] = ptr_new(bytarr(currentBlockSize, /nozero), /no_copy)
> memBlockSizes[i] = currentBlockSize ; Store the latest successful
> allocation size
>
> ; Print the current allocated block size and the running total, in Mb
> print, format='(%"Memory block #%2d: %6d Mb (total: %4d Mb)")', $
> i + 1, ishft(currentBlockSize, -20),
> ishft(ulong(total(memBlockSizes)), -20)
> endfor
>
> ptr_free,memPtrs
> end
Hmm, Here's what I get on my 4GB of RAM Quad core system running 64
bit IDL (v6.4) under 64 bit Vista :-
IDL> memtest
current block size = 2146435072
Memory block # 1: 2047 Mb (total: 2047 Mb)
Memory block # 2: 2047 Mb (total: 4094 Mb)
Memory block # 3: 2047 Mb (total: 2045 Mb)
Memory block # 4: 2045 Mb (total: 4090 Mb)
Memory block # 5: 2043 Mb (total: 2037 Mb)
Memory block # 6: 2041 Mb (total: 4078 Mb)
Memory block # 7: 1803 Mb (total: 1785 Mb)
Can't say I know how to interpret that at all !!
Andrew C.
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
