| Re: largest array, most memory accessible [message #41206] |
Mon, 18 October 2004 21:58 |
netnews.comcast.net
Messages: 10
Registered: October 2004
|
Junior Member |
|
|
Karl Schultz wrote:
> "Karl Schultz" wrote...
>
>> "Rick Towler" wrote ...
>>
>>> Karl,
>>>
>>> How will all this change with the introduction of WinXP-64? While I
>>> very much hope you guys are working on a 64-bit version of IDL for
>>> windows, I won't ask you to comment on that. But, how will opening up
>>> the address space affect the 32bit version of IDL running on windows?
>>
>> I don't know exactly how a 64-bit Windows would run 32-bit apps. But at
>
> the
>
>> moment, I do not think it would be that much different than the 3GB
>> situation today. There would probably still have to be some kernel space
>> mapped below 4G.
>
>
> Actually, I just found a blog saying that a 32-bit app compiled with the
> large address flag can access 4G in Win 64. I don't know how true that
> statement is, however.
FWIW, I just read the same thing. With a 64bit OS running a 32-bit exe
the full 4GB will be available to the 32-bit app as long as there is
sufficient memory in the system to allow for the OS+extras to be loaded
elsewhere.
-r
|
|
|
|
| Re: largest array, most memory accessible [message #41245 is a reply to message #41206] |
Thu, 14 October 2004 16:59  |
Karl Schultz
Messages: 341
Registered: October 1999
|
Senior Member |
|
|
"Karl Schultz" <kschultz_no_spam@rsinc.com> wrote in message
news:10mtqcs4btgsoc1@corp.supernews.com...
>
> "Rick Towler" <rtowler@xxx.u.washington.edu> wrote in message
> news:ckmcut$uti$1@gnus01.u.washington.edu...
>> Karl,
>>
>> How will all this change with the introduction of WinXP-64? While I
>> very much hope you guys are working on a 64-bit version of IDL for
>> windows, I won't ask you to comment on that. But, how will opening up
>> the address space affect the 32bit version of IDL running on windows?
>
> I don't know exactly how a 64-bit Windows would run 32-bit apps. But at
the
> moment, I do not think it would be that much different than the 3GB
> situation today. There would probably still have to be some kernel space
> mapped below 4G.
Actually, I just found a blog saying that a 32-bit app compiled with the
large address flag can access 4G in Win 64. I don't know how true that
statement is, however.
Karl
|
|
|
|
| Re: largest array, most memory accessible [message #41301 is a reply to message #41245] |
Thu, 14 October 2004 15:28 |
R.G. Stockwell
Messages: 363
Registered: July 1999
|
Senior Member |
|
|
"Karl Schultz" <kschultz_no_spam@rsinc.com> wrote in message news:10mtqcs4btgsoc1@corp.supernews.com...
>
> "Rick Towler" <rtowler@xxx.u.washington.edu> wrote in message
> news:ckmcut$uti$1@gnus01.u.washington.edu...
>> Karl Schultz wrote:
Rick, Karl,
thanks for the responses! Good info.
Yes I am looking to get a solution without paging, if I had to
page then the problem with size would kick over into a problem with
speed (and the size problem is more amenable to solution).
I'm limping along on a mere 512mb now, so having 3 or 4 gigs
will be quite nice!
Cheers,
bob
PS For some reason I didn't see these posts, so I had to pop over to google to
read them. hmmmm
|
|
|
|
| Re: largest array, most memory accessible [message #41302 is a reply to message #41301] |
Wed, 13 October 2004 17:22 |
Karl Schultz
Messages: 341
Registered: October 1999
|
Senior Member |
|
|
"R.G. Stockwell" <noemail@please.com> wrote in message
news:2t5g66F1ghte9U1@uni-berlin.de...
> I was following some of the other threads about memory size,
> and wanted to see what the options are for a new computer I am
considering.
> (I am looking at getting a dell dimension or something in that price range
and wondering
> how much ram is useful. The main thing to spend money on is ram, even if
I get a slower
> processor)
>
> My dream would be to have an array of complex variables of size [1024,
1024, 1024,512],
> (four terrabytes of ram?) but I guess I can scrape by with arrays of
[128,128,128,64] perhaps.
> I cannot test out these values on my current computers.
>
>
> 1) What is the largest array in IDL 6.1 on a winXP machine?
> Is this 2 Gb?
>
> 2) What is the largest amount of ram IDL can access, i.e. can I
> make multiple copies of the array in question 1?
> Is this limit also 2Gb or can IDL grab the full 4gbs?
One place I can point you to is:
http://tinyurl.com/5uhom
aka:
http://groups.google.com/groups?hl=en&lr=&newwindow= 1&safe=off&threadm=e5SG4w7dDHA.2312%40TK2MSFTNGP12.p hx.gbl&rnum=3&prev=/groups%3Fhl%3Den%26lr%3D%26newwi ndow%3D1%26safe%3Doff%26q%3DwinXP%2Bcontiguous%2Baddress%2Bs pace
Other notes:
1) The amount of RAM in your machine doesn't really limit the size of the
problem you are trying to solve. More RAM means less paging and you will
get your problem solved faster. If you have a small amount of RAM and a lot
of virtual storage (large page file, a big enough disk, and an OS that
supports a large enough virtual address space) you can still finish, but it
will take a long time. If the entire problem fits into RAM, then it will
cruise, and that's why people get a lot of RAM.
2) Win XP can be configured to have up to 3GB of virtual storage. Usually,
it is 2GB. The 32-bit processors can address up to 4GB, but the Windows
(NT) architecture usually reserves the upper 2GB for the kernel. There is
plenty of documentation on the net about setting the user space to 3GB,
hence setting the kernel space to 1GB. So, IDL can never grab the full 4GB.
At best, it can get only part of the 2GB or 3GB that is the user virtual
address space.
3) And here is the most important part, which is covered by the URL I noted
above. The limiting factor on the largest array you can allocate in IDL is
the largest piece of *contiguous virtual* address space that is available.
So, for example, if Windows, IDL, and a lot of its required libraries occupy
the first 0.25GB of virtual address space, and Windows also grabs some of
the upper user virtual address space, say another 0.25 GB, then, at best,
you will be able to allocate a 1.5GB array (assuming the std 2GB available).
If a small DLL or some other small thing gets loaded in the middle of the
virtual address space, say at the 1GB mark, then the virtual address space
is fragmented and now you can only get two 750 MB arrays. This can happen,
for example, if your IDL program allocates a few large arrays and a small
array that happened to fall in between two large arrays. If you free the
two large arrays, thinking that you will be able to allocate a single array
that is the combined size of the two arrays you just freed, then the attempt
will fail unless there is a big enough chunk of virtual address space
someplace else.
Anyway, I seem to remember some discussion here awhile ago saying that
people could get 1.2 or 1.3 GB arrays on Windows machines. I doubt that the
numbers today are much different, unless you throw the 3GB switch. Perhaps
trying the 3GB configuration is worthwhile for you, or use a 64-bit OS.
Hope this helps,
Karl
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
