comp.lang.idl-pvwave archive - RDF feed
type: channel
description: Messages from Usenet group comp.lang.idl-pvwave, compiled by Paulo Penteado
items: Re: working with very large data sets

Re: working with very large data sets
type: item
description: Steve Carothers <onav1@flash.net> wrote: > I am working with 71 Mb data file on UNIX server that has 256 Mb RAM and 500 > Mb of virtual memory. I'm not doing much data manipulation before I plot > the data, but it doesn't take much manipulation to exceed the memory > allocation. Is it possible you are not allowed to use all this memory? Have you tried the csh command "limit"? -W. -- William M Connolley | wmc@bas.ac.uk | http://www.nbs.ac.uk/public/icd/wmc/ Climate Modeller, British Antarctic Survey | Disclaimer: I speak for myself
creator: wmc
date: 1999-03-18T08:00:00-00:00

Re: working with very large data sets
type: item
description: Steve Carothers wrote: > I am working with 71 Mb data file on UNIX server that has 256 Mb RAM and 500 > Mb of virtual memory. I'm not doing much data manipulation before I plot > the data, but it doesn't take much manipulation to exceed the memory > allocation. I understand the benefits of chunking up the data, but I would > really like to keep all the data together for plotting purposes. I think my > PV-Wave script could run properly if I can figure out how to minimize or > eliminate memory fragmentation. When I'm done with a variable I set it > equal to 0 to free up the memory. However, if I understand the manual > correctly, this will not free up contiguous memory, which is what I need. > Delstruct and delvar might help me but they can't be used inside a script, > only at the prompt. I have a feeling I'll be forced to chunk up the data. > > Also, is there a way to remove a set of records from an array of structures > if the records to delete are known without using the "where" command and > without creating a tempory variable in the memory? > > Any advice would be appreciated. > > Steve look at temporary a=bytarr(2e5) b=temporary(a)+1b R.Bauer
creator: R.Bauer
date: 1999-03-18T08:00:00-00:00