| Re: How to allocate memory for an array of more than 2G [message #71675] |
Tue, 13 July 2010 20:45  |
R.G.Stockwell
Messages: 163
Registered: October 2004
|
Senior Member |
|
|
"NEW-IDL-USER" <mchen911@gmail.com> wrote in message
news:c5a17126-f507-41bf-84d9-6260b9f0ec33@i18g2000pro.google groups.com...
> Hi,
> I have a trouble to read a .h5 file which is about 3G.
> I am using {x86_64 linux unix linux 7.1.1 Aug 21 2009 64 64} IDL. So
> you see, my system is 64 bit and IDL is also 64 bit. The problem I
> cannot allocate a memory for an array more than 2G. I do not know why?
> Can someone help me?
> Thanks a lot!
>
> IDL> help,/memory
> heap memory used: 818394, max: 1074560678, gets: 1069,
> frees: 258
> IDL> a=bytarr(1024,1024,1024)
> IDL> help,/memory
> heap memory used: 1074560268, max: 1074560367, gets: 1081,
> frees: 269
> IDL> a=bytarr(1024,1024,1024)
> % Unable to allocate memory: to make array.
> Cannot allocate memory
Hello New,
I have no idea why you cannot allocate that memory.
In fact, you call a = bytarr() twice, so it looks like you can
only allocate one gig. How much memory is on your machine?
And are there quotas involved on the IDL process (perhaps you should
ask your sys admin).
for instance, you should be able to do this.
IDL> a=bytarr(1024,1024,1024)
IDL> b=bytarr(1024,1024,1024)
IDL> c=bytarr(1024,1024,1024)
IDL> d=bytarr(1024,1024,1024)
IDL> help,/mem
heap memory used: 4303438407, max: 4303438540, gets: 107997, frees:
98790
IDL> e=bytarr(1024,1024,1024)
IDL> f=bytarr(1024,1024,1024)
IDL> help,/mem
heap memory used: 6450922396, max: 6450922529, gets: 108015, frees:
98806
cheers,
bob
|
|
|
|
| Re: How to allocate memory for an array of more than 2G [message #71754 is a reply to message #71675] |
Wed, 14 July 2010 15:14  |
minchen
Messages: 8
Registered: February 2010
|
Junior Member |
|
|
Hi Bob,
Thanks a lot!
I cannot do this:
IDL> a=bytarr(1024,1024,1024)
IDL> b=bytarr(1024,1024,1024)
IDL> c=bytarr(1024,1024,1024)
IDL> d=bytarr(1024,1024,1024)
IDL> help,/mem
on my system. It is a 64 bit linux system and the memory is 64G.
Is there a limitation of the virtual memory which is related to this
one? If so, how can I do it?
Thanks a lot!
On Jul 13, 8:45 pm, "R.G. Stockwell" <noem...@please.com> wrote:
> "NEW-IDL-USER" <mchen...@gmail.com> wrote in message
>
> news:c5a17126-f507-41bf-84d9-6260b9f0ec33@i18g2000pro.google groups.com...
>
>
>
>
>
>> Hi,
>> I have a trouble to read a .h5 file which is about 3G.
>> I am using {x86_64 linux unix linux 7.1.1 Aug 21 2009 64 64} IDL. So
>> you see, my system is 64 bit and IDL is also 64 bit. The problem I
>> cannot allocate a memory for an array more than 2G. I do not know why?
>> Can someone help me?
>> Thanks a lot!
>
>> IDL> help,/memory
>> heap memory used: 818394, max: 1074560678, gets: 1069,
>> frees: 258
>> IDL> a=bytarr(1024,1024,1024)
>> IDL> help,/memory
>> heap memory used: 1074560268, max: 1074560367, gets: 1081,
>> frees: 269
>> IDL> a=bytarr(1024,1024,1024)
>> % Unable to allocate memory: to make array.
>> Cannot allocate memory
>
> Hello New,
> I have no idea why you cannot allocate that memory.
> In fact, you call a = bytarr() twice, so it looks like you can
> only allocate one gig. How much memory is on your machine?
>
> And are there quotas involved on the IDL process (perhaps you should
> ask your sys admin).
>
> for instance, you should be able to do this.
>
> IDL> a=bytarr(1024,1024,1024)
> IDL> b=bytarr(1024,1024,1024)
> IDL> c=bytarr(1024,1024,1024)
> IDL> d=bytarr(1024,1024,1024)
> IDL> help,/mem
> heap memory used: 4303438407, max: 4303438540, gets: 107997, frees:
> 98790
> IDL> e=bytarr(1024,1024,1024)
> IDL> f=bytarr(1024,1024,1024)
> IDL> help,/mem
> heap memory used: 6450922396, max: 6450922529, gets: 108015, frees:
> 98806
>
> cheers,
> bob- Hide quoted text -
>
> - Show quoted text -
|
|
|
|
| Re: How to allocate memory for an array of more than 2G [message #71756 is a reply to message #71675] |
Wed, 14 July 2010 12:14 |
Karl[1]
Messages: 79
Registered: October 2005
|
Member |
|
|
On Jul 13, 9:45 pm, "R.G. Stockwell" <noem...@please.com> wrote:
> "NEW-IDL-USER" <mchen...@gmail.com> wrote in message
>
> news:c5a17126-f507-41bf-84d9-6260b9f0ec33@i18g2000pro.google groups.com...
>
>
>
>> Hi,
>> I have a trouble to read a .h5 file which is about 3G.
>> I am using {x86_64 linux unix linux 7.1.1 Aug 21 2009 64 64} IDL. So
>> you see, my system is 64 bit and IDL is also 64 bit. The problem I
>> cannot allocate a memory for an array more than 2G. I do not know why?
>> Can someone help me?
>> Thanks a lot!
>
>> IDL> help,/memory
>> heap memory used: 818394, max: 1074560678, gets: 1069,
>> frees: 258
>> IDL> a=bytarr(1024,1024,1024)
>> IDL> help,/memory
>> heap memory used: 1074560268, max: 1074560367, gets: 1081,
>> frees: 269
>> IDL> a=bytarr(1024,1024,1024)
>> % Unable to allocate memory: to make array.
>> Cannot allocate memory
>
> Hello New,
> I have no idea why you cannot allocate that memory.
> In fact, you call a = bytarr() twice, so it looks like you can
> only allocate one gig. How much memory is on your machine?
>
> And are there quotas involved on the IDL process (perhaps you should
> ask your sys admin).
>
> for instance, you should be able to do this.
>
> IDL> a=bytarr(1024,1024,1024)
> IDL> b=bytarr(1024,1024,1024)
> IDL> c=bytarr(1024,1024,1024)
> IDL> d=bytarr(1024,1024,1024)
> IDL> help,/mem
> heap memory used: 4303438407, max: 4303438540, gets: 107997, frees:
> 98790
> IDL> e=bytarr(1024,1024,1024)
> IDL> f=bytarr(1024,1024,1024)
> IDL> help,/mem
> heap memory used: 6450922396, max: 6450922529, gets: 108015, frees:
> 98806
>
> cheers,
> bob
Quotas are a good thing to check. Also, check the page file size. A
small RAM size and small page file size will restrict the amount of
virtual memory your process can allocate. Increasing RAM would give
you the best-performing improvement. But you could theoretically add
a page file or expand an existing page file and still (slowly) solve
the problem.
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
