| Maximum memory under Windows NT [message #14587] |
Fri, 12 March 1999 00:00  |
rivers
Messages: 228
Registered: March 1991
|
Senior Member |
|
|
I have a question about maximum memory allocation under Windows NT. I have a
high-end Windows NT workstation configured with
- Dual 450 MHz Pentium CPUs
- 1GB of RAM
- Windows NT Workstation 4.0, SP3
I have allocated 2GB of paging file space for virtual memory. The Task
Manager/Performance screen shows a Commit Charge Limit of about 3200000, which
makes sense, since it is the sum of the paging file and the physical memory.
However, when I try to allocate large arrays in IDL I find that it fails at
a = bytarr(1024, 1024, 1024)
i.e. it cannot allocate a 1 GB array, but it succeeds at
a = bytarr(1024, 1024, 1000)
i.e. just less than 1 GB.
When it fails I see that the Task Manager/Performance/Commit Charge Total is
what I expect, and that it is not close to the Commit Charge Limit by a factor
of 3.
Does anyone know what the maximum memory available to application programs is
under Windows NT, and if there is a way to increase it? I realize that
Windows NT is a 32-bit operating system, so the maximum can be no more than 4
GB, but I was not aware that it was limited to 1 GB, if indeed it is.
____________________________________________________________
Mark Rivers (773) 702-2279 (office)
CARS (773) 702-9951 (secretary)
Univ. of Chicago (773) 702-5454 (FAX)
5640 S. Ellis Ave.
Chicago, IL 60637 rivers@cars.uchicago.edu (e-mail)
or:
Argonne National Laboratory (630) 252-0422 (office)
Building 434A (630) 252-0405 (lab)
9700 South Cass Avenue (630) 252-1713 (beamline)
Argonne, IL 60439 (630) 252-0443 (FAX)
|
|
|
|
| Re: Maximum memory under Windows NT [message #14636 is a reply to message #14587] |
Wed, 17 March 1999 00:00  |
menakkis
Messages: 37
Registered: June 1998
|
Member |
|
|
Mark Rivers writes
.. about hitting a 1GB memory allocation limit in IDL on WinNT.
Hi Mark, My initial thought was "1GB is quite some allocation", but then not
that long ago I used to think that 1GB was a whole lotta disk. Pretty soon
it won't be that rare at all. I guess you've noticed that the standard MS
docs are typically vague on the subject, apart from saying that "each
application on NT has most of the lower 2GB of its virtual address space at
its disposal". By chance I spotted a MS visual C linker option that governs
the heap size. BY DEFAULT THIS IS 1GB. Evidently this is a seldom-used
option, because there isn't even a field for it in the project settings
property sheet. If IDL is using malloc() from the MSVC C runtime library (or
is using some memory- management s/w that uses it) then I would guess that
this is the problem. If so, then perhaps RSI would respond to a feature
request to link specifying a larger heap.
"Martin Downing" <m.downing@abdn.ac.uk> wrote:
> As an aside:
> I am considering purchase of a similar spec machine for our workgroup, for
> use with memory hungry images and processing routines in IDL and C++.
> Would you possibly have time to drop me details of your spec and what you
> might change now that you have tried the system.
> 1.I was also thinking dual CPUs. As far as I know IDL 5.2 can not handle
> multithreading, but i anticipate there would be a performance benefit since
> one cpu would get all the OS work (does that include paging?) and the other
> would then be dedicated to running IDL - have you found this?
Hi Martin, Mind if I butt in? :-) I hope I don't offend the group by talking
about non-IDL things... I have access to a (dated) dual-PII-300 PC running
NT. The 2 CPUS do help a bit. e.g., When running ENVI on an
image-processing job (say, doing an MNF transform of an AVIRIS scene), the
total CPU usage sometimes goes over 60%. Even higher (70ish I think) at
times. In general, having 2 CPUS is rather pleasant if your budget can take
it - everything cruises along that much smoother. If you have the
inclination, you can also write your own multithreaded routines to call from
IDL. If you work with large files then I'd strongly recommend that you
consider beefing up your I/O. I have access to an (also dated) ADAPTEC RAID
controller with a RAID0 disk (of 3 physical disks). It makes a substantial
difference. Hardware RAID is not all that expensive to set up these days. It
might also be worth your while to check out Linux instead of NT. I have no
experience with Linux, but I'm really starting to wonder about NT. ("Fifty
billion dollars can't be right" and so on.) I have heard it said that some
number-crunching Fortran programs favoured by geophysicists run 3 to 5 times
faster on Linux! It's a pity that there's a timer problem in IDL for Linux -
it would be interesting to see some comparisons in J.D. Smith's time-test
archive.
Peter Mason
-----------== Posted via Deja News, The Discussion Network ==----------
http://www.dejanews.com/ Search, Read, Discuss, or Start Your Own
|
|
|
|
| Re: Maximum memory under Windows NT [message #14677 is a reply to message #14587] |
Sat, 13 March 1999 00:00 |
Martin Downing
Messages: 136
Registered: September 1998
|
Senior Member |
|
|
Hi Mark,
> I have a question about maximum memory allocation under Windows NT. I have
a
> high-end Windows NT workstation configured with
> - Dual 450 MHz Pentium CPUs
> - 1GB of RAM
> - Windows NT Workstation 4.0, SP3
>
Sorry - I have no real solution, but here is my tupence worth:
What I have found is that life becomes unbearable, at least on our single
PII450 with 124M Ram and a UDMA (ie non scsi) drive, if a routine exceeds
the available RAM for IDL and consequently forces paging.
On the above system I ran a simple test routine on a 64Mb 2D image. It
completed in 0.2-0.5 sec if the image was resident in ram, compared to 10-20
sec if it had been paged out to VM. This is much slower than the sum of the
theoretical time to transfer the 64Mb to RAM and then running the
routine -more like "gridlock"!
So to a new question: given a situation where you have IDL managing a large
amount of memory where paging to virtual memory is inevitable, is there an
efficient way to force/request the system to load required memory for a
routine (eg an image) into RAM at the start of the routine ? (or even before
if you have multiple processors!)
As an aside:
I am considering purchase of a similar spec machine for our workgroup, for
use with memory hungry images and processing routines in IDL and C++.
Would you possibly have time to drop me details of your spec and what you
might change now that you have tried the system.
1.I was also thinking dual CPUs. As far as I know IDL 5.2 can not handle
multithreading, but i anticipate there would be a performance benefit since
one cpu would get all the OS work (does that include paging?) and the other
would then be dedicated to running IDL - have you found this?
2. Do you advise the most expensive (ECC) memory
3. Do you think Dual Xeon (i know its pricey) offers significantly improved
memory handling (I/O speed) compared to Pentium III? Unless it does then the
current 500Mhz speed advantage of pentium 3 plus the possibility to code up
external routines in C which use the new SIMD extensions (4 floating points
calcs in one go) seems to be the way to go........if the system can feed the
cpus fast enough.
Martin Downing
Research Physicist,
Grampian Orthopaedic Clinical Research Unit,
Woodend Hospital, Grampian Healthcare Trust,
Eday Road, Aberdeen, Scotland. AB15 6LS
mailto:m.downing@abdn.ac.uk
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
