| Re: opening and display large file [message #21536] |
Thu, 31 August 2000 00:00  |
Martin Schultz
Messages: 515
Registered: August 1997
|
Senior Member |
|
|
Sylvain Carette wrote a lot of words but not too many facts:
>
You never gave us any specs of your system (not that I could do
anything with it, but I know of some people who might get
interested in helping you if you provided some more technical
infos). Before you blame IDL for not being able to do what you
want, you may want to assure yourself that it is not a hardware
problem or a problem with your window manager/X server or
whatever that you are running into. IDL is definitively capable
of handling huge arrays, but it can't display things that your
window manager doesn't like to display. Example: on my Linux PC
with 1GB memory, I am able to create four arrays as
test1=lindgen(10000L,10000L)
test2=lindgen(10000L,10000L)
etc.
this gives a total of 1.2 GB (and surely doesn't fly), but IDL
still works.
Please note: for some projects, selling a piano is not enough.
What you really seem to be in need of is a solid buisiness model.
Martin
--
[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[ [[[[[[[
[[ Dr. Martin Schultz Max-Planck-Institut fuer Meteorologie
[[
[[ Bundesstr. 55, 20146 Hamburg
[[
[[ phone: +49 40 41173-308
[[
[[ fax: +49 40 41173-298
[[
[[ martin.schultz@dkrz.de
[[
[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[ [[[[[[[
|
|
|
|
| Re: opening and display large file [message #21538 is a reply to message #21536] |
Wed, 30 August 2000 23:13  |
Sylvain Carette
Messages: 19
Registered: May 2000
|
Junior Member |
|
|
<!doctype html public "-//w3c//dtd html 4.0 transitional//en">
<html>
<tt></tt>
<br><tt>Craig Markwardt wrote:</tt>
<blockquote TYPE=CITE><tt>Sylvain Carette <sylvainc@total.net> writes:</tt>
<br><tt> > Give me an open file to work with, and I know what to do
after. I sell my</tt>
<br><tt> > piano to buy IDL naively thinking that it was handling
large file ( well,</tt>
<br><tt> > RSI claim they have "industrial strenght" tool to handle
very large stuff ).</tt>
<br><tt> > Where are those functionality?</tt>
<br><tt> > Do I need to write my own image manager? Should I have
buy ErMapper instead?</tt><tt></tt>
<p><tt>Your demands place you in the 0.1% group. The other 99.9%
people</tt>
<br><tt>don't need the features. IDL itself is pretty general purpose.
I</tt>
<br><tt>believe that ENVI is designed to do exactly what you want.</tt>
<br><tt></tt> </blockquote>
<tt>Story of my life; always been in the 0.001% group.....</tt>
<br><tt>As for ENVI, I could get my hand on it for a while but it was not
appropriate for my task and since most of the ENVI feature I need come
from idl itself I though it was better to get idl and build my own specific
tools (plus that I have only one piano to sell).</tt><tt></tt>
<p><tt>I think its kind of natural to think that a langage that is used
to produce a software like ENVI, will probably include functionality to
handle the management of memory for large file. Hence the deception; from
what I've read in this group it seem there is no such functionnality. So
thats mean I'll have to implement one and it also mean I'm a step back
instead of a step forward, nothing to make you feel like dancing samba...</tt><tt></tt>
<p><tt>I disagree though on your comment about "industrial price"; I've
seen too many expensive application that reveal to be dog... while finding
freeware outperforming high end software in the functionality they implement.
So industrial strenght is really not a matter of price but capability while
industrial price is a matter of marketing. Bentley system cost ave. 35k
fully equip and cannot match a $35 bucks japanese program to extract 3d
from photograph (</tt> <a href="http://www.photo3d.com/eindex.html">Photo3D:
3D modeling from photo.</a> <tt>- this one is really amazing; I show it
to an architect and his jaw simply drop on the floor - I was doing in 5min
what he use to take hours of tedious work on a $3000 software). Bentley's
peoples didnt even look at it... If Bentley system would build a module
implementing this, they will make you pay "industrial" price although it
will probably be more slow and less intuitive to use. So I dont beleive
in marketing and industrial price. About ENVI, there is a freeware application </tt>
<a href="http://dynamo.ecn.purdue.edu/~biehl/MultiSpec/">Purdue/LARS MultiSpec</a>
<tt>which duplicate all of ENVI multispectral capability and have documentation
that cover in depth multispectral theory with numerous examples. ENVI industrial
circular documentation dont give you that much for your bucks....</tt><tt></tt>
<p><tt>Actually, the tool that is the most appropriate for my need (integration
of images and data to produce 3d VRML) is TNT Mips </tt><a href="http://www.microimages.com/">MicroImages,
Inc.</a><tt>but I cant afford it yet. It handle (yes, very large file)
and edit raster, cad and vector to produce elaborated tin or gridded surfaces.
It have all the capability of ENVI plus cad, vector and 3D modeling and
it deliver it. Plus you can scan and digitize either raster and cad/vector
data and, ah yes, it have also a scripting langage that use "standard"
c syntax so here you dont loose anytime deciphering criptic and non-consistent
syntax and use that time to work.</tt>
<br><tt>And it even run from a x server so you can have the illusion of
working on something better than a PC...</tt><tt></tt>
<p><tt>Now you might think I hate idl; well admittedly, this is a very
bad langage: it doesnt have any design, just a pile of function added over
the years. Seem they have combined the worst feature of all langages to
implement idl... (well, what can you say about a langage that still have
GOTO statement and object with keyword......) But it work... So fine, I
want to *love* idl and if I came to idl (beside little deception for large
file) its for it power to manipulate array which I think is a unique feature
and is what someone could call "industrial strenght" feature.</tt>
<br><tt>So help me to take advantage of the power of idl array manipulation
and maybe I'll become an idl advocate. I'll begin right now thanks for
your reply and maybe later this night I'll begin to like idl...</tt>
<br><tt>Sorry for all that bla bla but I dont like to be tell I was not
willing to sell another piano... ;-)</tt>
<br><tt>And just to make sure it isnt have been buried in all those words,
thank you for your help. I'm grateful. It is just what I needed at this
time, really.</tt><tt></tt>
<p><tt>Sylvain Carette</tt>
<br><tt>VRML designer-composer</tt>
<br><tt></tt>
<blockquote TYPE=CITE><tt>If you want industrial strength professional
tools to work on</tt>
<br><tt>industrial sized images doing industrial tasks, and you *aren't*</tt>
<br><tt>willing to pay industrial prices, well what can I say?</tt><tt></tt>
<p><tt>Craig</tt><tt></tt>
<p><tt>--</tt>
<br><tt> ------------------------------------------------------------ -------------- </tt>
<br><tt>Craig B. Markwardt, Ph.D. & nbsp;
EMAIL: craigmnet@cow.physics.wisc.edu</tt>
<br><tt>Astrophysics, IDL, Finance, Derivatives | Remove "net" for better
response</tt>
<br><tt> ------------------------------------------------------------ -------------- </tt></blockquote>
<tt></tt></html>
|
|
|
|
| Re: opening and display large file [message #21548 is a reply to message #21538] |
Wed, 30 August 2000 00:00 |
Craig Markwardt
Messages: 1869
Registered: November 1996
|
Senior Member |
|
|
Sylvain Carette <sylvainc@total.net> writes:
> The only other difference is that instead of extracting a subset, I used
> congrid to resample a lightweigth display grid. Even though, nothing on tv
> and a small portion with slide_image which disapear immediately as soon I
> move a scroll bar (retain=0; retain 1 & 2 just make windows crash)...
> So this mean that 4800 x 6000 is too big for those two function? I still
> dont understand why it would be so big since according to the doc, you
> should be able to open a file as long you have enough memory and I have
> 250mb which should be enough for one gtopo tile no?
An image of 4800 x 6000 is getting pretty big. A test on my machine
(Solaris) has brought it to its knees. At the rendering stage with
SLIDE_IMAGE, it hung for a few minutes. I believe this is because the
entire image must be loaded into the window manager memory. However
IDL itself was able manipulate the arrays pretty easily (RAM=256mb).
The difference between taking a subset and using CONGRID could be
large. However I had no problem using it to downsample a 4800x6000
array. That's strange.
If you can't get away with looking at small tiles, then you may be
able to modify SLIDE_IMAGE. You'll have to study up on widgets, find
out about VIEWPORT_EVENTS, and then make an event handler that
re-renders the exposed portions when you scroll. Either that or make
a simple widget with directional buttons like mapquest.com, and render
your tiles like that.
> Give me an open file to work with, and I know what to do after. I sell my
> piano to buy IDL naively thinking that it was handling large file ( well,
> RSI claim they have "industrial strenght" tool to handle very large stuff ).
> Where are those functionality?
> Do I need to write my own image manager? Should I have buy ErMapper instead?
Your demands place you in the 0.1% group. The other 99.9% people
don't need the features. IDL itself is pretty general purpose. I
believe that ENVI is designed to do exactly what you want.
If you want industrial strength professional tools to work on
industrial sized images doing industrial tasks, and you *aren't*
willing to pay industrial prices, well what can I say?
Craig
--
------------------------------------------------------------ --------------
Craig B. Markwardt, Ph.D. EMAIL: craigmnet@cow.physics.wisc.edu
Astrophysics, IDL, Finance, Derivatives | Remove "net" for better response
------------------------------------------------------------ --------------
|
|
|
|
| Re: opening and display large file [message #21557 is a reply to message #21548] |
Wed, 30 August 2000 00:00 |
Sylvain Carette
Messages: 19
Registered: May 2000
|
Junior Member |
|
|
<!doctype html public "-//w3c//dtd html 4.0 transitional//en">
<html>
<tt>Thanks</tt>
<br><tt>That's look almost what I have tryed. I'll use it "as is" to see
the difference with mine but its mostly the fact I tryed to send the whole
array to tv and slide_image.</tt>
<br><tt>Also, I didnt close the file and free lun immediately after reading
as you did - is this could cause memory leakage (when freezing, I was wondering
after reset if I left big lump of anaccessible memory)?</tt>
<br><tt>The only other difference is that instead of extracting a subset,
I used congrid to resample a lightweigth display grid. Even though, nothing
on tv and a small portion with slide_image which disapear immediately as
soon I move a scroll bar (retain=0; retain 1 & 2 just make windows
crash)...</tt>
<br><tt>So this mean that 4800 x 6000 is too big for those two function?
I still dont understand why it would be so big since according to the doc,
you should be able to open a file as long you have enough memory and I
have 250mb which should be enough for one gtopo tile no?</tt><tt></tt>
<p><tt>I hope I dont sound like a bugger but beleive me, I tryed to do
my homework before requesting help here.</tt>
<br><tt>BYW, did you open usgs dem?</tt><tt></tt>
<p><tt>Sylvain Carette</tt>
<br><tt>VRML designer-composer</tt><tt></tt>
<p><tt>Kelly Dean wrote:</tt>
<blockquote TYPE=CITE><tt>This is what I use to access the GTOPO30 DEMs</tt><tt></tt>
<p><tt> Kelly Dean</tt>
<br><tt> CSU/CIRA</tt><tt></tt>
<p><tt>================================================</tt >
<br><tt>PRO ReadTopo</tt><tt></tt>
<p><tt>file = 'e:\w140n90\w140n90.dem</tt>
<br><tt>topo = INTarr(4800,6000)</tt>
<br><tt>;</tt>
<br><tt>; Open DEM, Swap_Endian if you are using a PC.</tt>
<br><tt>;</tt>
<br><tt>OpenR, lun, file, /Get_Lun, /swap_endian</tt>
<br><tt>ReadU, lun, topo</tt>
<br><tt>Close, lun</tt>
<br><tt>Free_Lun, lun</tt>
<br><tt>;</tt>
<br><tt>; Cut out a small area and scale it.</tt>
<br><tt>;</tt>
<br><tt>cut = BYTscl(topo(1500:2499,1500:2499), MIN=0 )</tt>
<br><tt>PRINT, ' MAX >',MAX( cut , MIN=MIN)</tt>
<br><tt>PRINT, ' MIN >',MIN</tt>
<br><tt>WINDOW, 0, XSize=1000, YSize=1000, TITLE='GTOP030'</tt>
<br><tt>TV, cut</tt><tt></tt>
<p><tt>END</tt>
<br><tt>=================================================</tt ><tt></tt>
<p><tt>Sylvain Carette wrote:</tt><tt></tt>
<p><tt>> Hi</tt>
<br><tt>> How do you manage to display very large images? What is the options?</tt>
<br><tt>> From the manual, it seem that using "assoc" is a winner but maybe
with</tt>
<br><tt>> one little example it would be more clear how can this be (It
seem to</tt>
<br><tt>> me that it is an important aspect of programming with IDL since
most</tt>
<br><tt>> of the time you'll deal with larger than memory files).</tt>
<br><tt>> Even with assoc, dont you still have to copy from the file to
an array</tt>
<br><tt>> before display?</tt>
<br><tt>> I dont decipher exactly why and when to use readu or read_binary,
the</tt>
<br><tt>> file pointer, assoc or writing to an offscreen buffer.</tt>
<br><tt>></tt>
<br><tt>> I tried to open and display a GTOPO30 tile (4800 x 6000 int).
tv could</tt>
<br><tt>> not display it - froze. I tried "slide_image" but it didnt scroll
with</tt>
<br><tt>> "retain=1" while "retain=2" and "retain=3" result in windows
error and</tt>
<br><tt>> crash.</tt>
<br><tt>></tt>
<br><tt>> Since most of my data fall in the 30mb to 610 mb range, I need</tt>
<br><tt>> absolutely to open and display very large file. What is available?</tt>
<br><tt>> Pointers, sample code, library, ideas, etc, I'm listening....
Just</tt>
<br><tt>> knowing that it have to be handled on your own or that IDL provide</tt>
<br><tt>> facility (that they forgot to explain in the manual) to deal
with this</tt>
<br><tt>> will already be invaluable</tt>
<br><tt>></tt>
<br><tt>> Thanks</tt>
<br><tt>></tt>
<br><tt>> Sylvain Carette</tt>
<br><tt>> VRML designer-composer</tt>
<br><tt>></tt></blockquote>
<tt></tt></html>
|
|
|
|
| Re: opening and display large file [message #21559 is a reply to message #21548] |
Wed, 30 August 2000 00:00 |
Sylvain Carette
Messages: 19
Registered: May 2000
|
Junior Member |
|
|
<!doctype html public "-//w3c//dtd html 4.0 transitional//en">
<html>
<tt></tt> <tt></tt>
<p><tt>"r.bauer" wrote:</tt>
<blockquote TYPE=CITE><tt>Sylvain Carette wrote:</tt>
<br><tt>></tt>
<br><tt>> Hi</tt>
<br><tt>> How do you manage to display very large images? What is the options?</tt>
<br><tt>> From the manual, it seem that using "assoc" is a winner but maybe
with</tt>
<br><tt>> one little example it would be more clear how can this be (It
seem to</tt>
<br><tt>> me that it is an important aspect of programming with IDL since
most</tt>
<br><tt>> of the time you'll deal with larger than memory files).</tt>
<br><tt>> Even with assoc, dont you still have to copy from the file to
an array</tt>
<br><tt>> before display?</tt>
<br><tt>> I dont decipher exactly why and when to use readu or read_binary,
the</tt>
<br><tt>> file pointer, assoc or writing to an offscreen buffer.</tt>
<br><tt>></tt>
<br><tt>> I tried to open and display a GTOPO30 tile (4800 x 6000 int).
tv could</tt>
<br><tt>> not display it - froze. I tried "slide_image" but it didnt scroll
with</tt>
<br><tt>> "retain=1" while "retain=2" and "retain=3" result in windows
error and</tt>
<br><tt>> crash.</tt>
<br><tt>></tt>
<br><tt>> Since most of my data fall in the 30mb to 610 mb range, I need</tt>
<br><tt>> absolutely to open and display very large file. What is available?</tt>
<br><tt>> Pointers, sample code, library, ideas, etc, I'm listening....
Just</tt>
<br><tt>> knowing that it have to be handled on your own or that IDL provide</tt>
<br><tt>> facility (that they forgot to explain in the manual) to deal
with this</tt>
<br><tt>> will already be invaluable</tt>
<br><tt>></tt>
<br><tt>> Thanks</tt>
<br><tt>></tt>
<br><tt>> Sylvain Carette</tt>
<br><tt>> VRML designer-composer</tt>
<br><tt>></tt><tt></tt>
<p><tt>Hi Sylvain,</tt><tt></tt>
<p><tt>Instead of OPEN / CLOSE you can use ASSOC too.</tt><tt></tt>
<p><tt>The idea is to display portions of the large image in a loop.</tt>
<br><tt>e.g. 480x600</tt><tt></tt>
<p><tt>I hope this helps a bit.</tt><tt></tt>
<p><tt>Reimar</tt>
<br><tt></tt> </blockquote>
<tt>Thanks for all your answer everybody</tt>
<br><tt>I hope this could help but I know this online help exemple since
I've pass more than a week trying to decipher it. Of course, giving exemple
with 512 x 512 images that have nothing to do with the usual size you'll
find in the remote sensing, astronomy and medical field doesnt really help.
I wont say what I think of the documentation just to stay polite... The
best way to learn a langage is trough looking at sample code as long that
sample code present some real life problem solving. How about handling
a 24,000 x 48,000 avhrr mosaic with more than 3 channels plus surface (GTOPO30
merged with USGS dem) plus vector "drapping"(vmap)(retesselating the surface
with constrain) to export in segmented multi-scale vrml IndexedFaceSet
- that's the real life job I have to do.</tt><tt></tt>
<p><tt>Give me an open file to work with, and I know what to do after.
I sell my piano to buy IDL naively thinking that it was handling large
file ( well, RSI claim they have "industrial strenght" tool to handle very
large stuff ). Where are those functionality?</tt>
<br><tt>Do I need to write my own image manager? Should I have buy ErMapper
instead?</tt>
<br><tt>Here I dont have much clue; opening the file in a kind of ram disk?
Along IDL procedure and functions, which one should be used to implement
this?</tt>
<br><tt>So please anybody, any cue especially pointer to sample code that
does a little something signifiant will be tremendiously appreciated.</tt><tt></tt>
<p><tt>I still need help to understand why my usgs dem code doest work
- see other post; why a statement which execute fine from the console,
doest work anymore inside a loop? I know, its me but where?</tt><tt></tt>
<p><tt>Sylvain Carette</tt>
<br><tt>VRML designer-composer</tt>
<br><tt></tt>
<blockquote TYPE=CITE><tt></tt>
<br><tt>The online help says.</tt><tt></tt>
<p><tt>Syntax</tt><tt></tt>
<p><tt>Result = ASSOC( Unit, Array_Structure [, Offset] [, /PACKED] )</tt><tt></tt>
<p><tt>Example</tt><tt></tt>
<p><tt>Suppose that the file images.dat holds 5 images as 256-element by</tt>
<br><tt>256-element arrays of bytes. Open the file for reading and create
an</tt>
<br><tt>associated variable by entering:</tt><tt></tt>
<p><tt>OPENR, 1, 'images.dat' ;Open the file as file unit 1.</tt><tt></tt>
<p><tt>A = ASSOC(1, BYTARR(256, 256)) ;Make an associated variable.</tt></blockquote>
<tt></tt></html>
|
|
|
|
| Re: opening and display large file [message #21561 is a reply to message #21548] |
Wed, 30 August 2000 00:00 |
Kelly Dean
Messages: 92
Registered: March 1997
|
Member |
|
|
This is what I use to access the GTOPO30 DEMs
Kelly Dean
CSU/CIRA
================================================
PRO ReadTopo
file = 'e:\w140n90\w140n90.dem
topo = INTarr(4800,6000)
;
; Open DEM, Swap_Endian if you are using a PC.
;
OpenR, lun, file, /Get_Lun, /swap_endian
ReadU, lun, topo
Close, lun
Free_Lun, lun
;
; Cut out a small area and scale it.
;
cut = BYTscl(topo(1500:2499,1500:2499), MIN=0 )
PRINT, ' MAX >',MAX( cut , MIN=MIN)
PRINT, ' MIN >',MIN
WINDOW, 0, XSize=1000, YSize=1000, TITLE='GTOP030'
TV, cut
END
=================================================
Sylvain Carette wrote:
> Hi
> How do you manage to display very large images? What is the options?
> From the manual, it seem that using "assoc" is a winner but maybe with
> one little example it would be more clear how can this be (It seem to
> me that it is an important aspect of programming with IDL since most
> of the time you'll deal with larger than memory files).
> Even with assoc, dont you still have to copy from the file to an array
> before display?
> I dont decipher exactly why and when to use readu or read_binary, the
> file pointer, assoc or writing to an offscreen buffer.
>
> I tried to open and display a GTOPO30 tile (4800 x 6000 int). tv could
> not display it - froze. I tried "slide_image" but it didnt scroll with
> "retain=1" while "retain=2" and "retain=3" result in windows error and
> crash.
>
> Since most of my data fall in the 30mb to 610 mb range, I need
> absolutely to open and display very large file. What is available?
> Pointers, sample code, library, ideas, etc, I'm listening.... Just
> knowing that it have to be handled on your own or that IDL provide
> facility (that they forgot to explain in the manual) to deal with this
> will already be invaluable
>
> Thanks
>
> Sylvain Carette
> VRML designer-composer
>
|
|
|
|
| Re: opening and display large file [message #21563 is a reply to message #21548] |
Wed, 30 August 2000 00:00 |
Craig Markwardt
Messages: 1869
Registered: November 1996
|
Senior Member |
|
|
Sylvain Carette <sylvainc@total.net> writes:
> Hi
> How do you manage to display very large images? What is the options?
> From the manual, it seem that using "assoc" is a winner but maybe with
> one little example it would be more clear how can this be (It seem to me
> that it is an important aspect of programming with IDL since most of the
> time you'll deal with larger than memory files).
> Even with assoc, dont you still have to copy from the file to an array
> before display?
> I dont decipher exactly why and when to use readu or read_binary, the
> file pointer, assoc or writing to an offscreen buffer.
What you are hearing from people is that you definitely need to do
some kind of chunking, banding, tiling, whatever you want to call it.
This involves only reading a subset of the image at a time, and
operating on it. For example, I know of no display that can handle
4800x6000 resolution, so you will probably only want to look at
subsets.
ENVI and/or FREELOOK is probably the commercial version of doing this,
since it's specifically designed for large images.
Doing it on your own is not necessarily hard, but it will take some
experimenting to do it right.
Craig
--
------------------------------------------------------------ --------------
Craig B. Markwardt, Ph.D. EMAIL: craigmnet@cow.physics.wisc.edu
Astrophysics, IDL, Finance, Derivatives | Remove "net" for better response
------------------------------------------------------------ --------------
|
|
|
|
| Re: opening and display large file [message #21565 is a reply to message #21548] |
Wed, 30 August 2000 00:00 |
R.Bauer
Messages: 1424
Registered: November 1998
|
Senior Member |
|
|
Sylvain Carette wrote:
>
> Hi
> How do you manage to display very large images? What is the options?
> From the manual, it seem that using "assoc" is a winner but maybe with
> one little example it would be more clear how can this be (It seem to
> me that it is an important aspect of programming with IDL since most
> of the time you'll deal with larger than memory files).
> Even with assoc, dont you still have to copy from the file to an array
> before display?
> I dont decipher exactly why and when to use readu or read_binary, the
> file pointer, assoc or writing to an offscreen buffer.
>
> I tried to open and display a GTOPO30 tile (4800 x 6000 int). tv could
> not display it - froze. I tried "slide_image" but it didnt scroll with
> "retain=1" while "retain=2" and "retain=3" result in windows error and
> crash.
>
> Since most of my data fall in the 30mb to 610 mb range, I need
> absolutely to open and display very large file. What is available?
> Pointers, sample code, library, ideas, etc, I'm listening.... Just
> knowing that it have to be handled on your own or that IDL provide
> facility (that they forgot to explain in the manual) to deal with this
> will already be invaluable
>
> Thanks
>
> Sylvain Carette
> VRML designer-composer
>
Hi Sylvain,
Instead of OPEN / CLOSE you can use ASSOC too.
The idea is to display portions of the large image in a loop.
e.g. 480x600
I hope this helps a bit.
Reimar
The online help says.
Syntax
Result = ASSOC( Unit, Array_Structure [, Offset] [, /PACKED] )
Example
Suppose that the file images.dat holds 5 images as 256-element by
256-element arrays of bytes. Open the file for reading and create an
associated variable by entering:
OPENR, 1, 'images.dat' ;Open the file as file unit 1.
A = ASSOC(1, BYTARR(256, 256)) ;Make an associated variable.
|
|
|
|
| Re: opening and display large file [message #21569 is a reply to message #21548] |
Wed, 30 August 2000 00:00 |
Martin Schultz
Messages: 515
Registered: August 1997
|
Senior Member |
|
|
Sylvain Carette wrote:
>
> Hi
> How do you manage to display very large images? What is the
> options?
> From the manual, it seem that using "assoc" is a winner but
> maybe with one little example it would be more clear how can
> this be (It seem to me that it is an important aspect of
> programming with IDL since most of the time you'll deal with
> larger than memory files).
> Even with assoc, dont you still have to copy from the file to
> an array before display?
> I dont decipher exactly why and when to use readu or
> read_binary, the file pointer, assoc or writing to an offscreen
> buffer.
>
> I tried to open and display a GTOPO30 tile (4800 x 6000 int).
> tv could not display it - froze. I tried "slide_image" but it
> didnt scroll with "retain=1" while "retain=2" and "retain=3"
> result in windows error and crash.
>
> Since most of my data fall in the 30mb to 610 mb range, I need
> absolutely to open and display very large file. What is
> available? Pointers, sample code, library, ideas, etc, I'm
> listening.... Just knowing that it have to be handled on your
> own or that IDL provide facility (that they forgot to explain
> in the manual) to deal with this will already be invaluable
>
> Thanks
>
> Sylvain Carette
> VRML designer-composer
>
One option is to look into scientific data formats like netcdf.
This won't decrease your file size, but you can directlt access
portions of the data without having to read the whole thing
first, and there are tools available (NCO) which allow you to do
some averaging etc. offline and thus create a smaller file that
will fit.
Martin
--
[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[ [[[[[[[
[[ Dr. Martin Schultz Max-Planck-Institut fuer Meteorologie
[[
[[ Bundesstr. 55, 20146 Hamburg
[[
[[ phone: +49 40 41173-308
[[
[[ fax: +49 40 41173-298
[[
[[ martin.schultz@dkrz.de
[[
[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[[ [[[[[[[
|
|
|
|
| Re: opening and display large file [message #21571 is a reply to message #21548] |
Wed, 30 August 2000 00:00 |
Nando Iavarone
Messages: 48
Registered: December 1998
|
Member |
|
|
<!doctype html public "-//w3c//dtd html 4.0 transitional//en">
<html>
Sylvain Carette wrote:
<blockquote TYPE=CITE><tt>Hi</tt>
<br><tt>How do you manage to display very large images? What is the options?</tt>
<br><tt>From the manual, it seem that using "assoc" is a winner but maybe
with one little example it would be more clear how can this be (It seem
to me that it is an important aspect of programming with IDL since most
of the time you'll deal with larger than memory files).</tt>
<br><tt>Even with assoc, dont you still have to copy from the file to an
array before display?</tt>
<br><tt>I dont decipher exactly why and when to use readu or read_binary,
the file pointer, assoc or writing to an offscreen buffer.</tt>
<p><tt>I tried to open and display a GTOPO30 tile (4800 x 6000 int). tv
could not display it - froze. I tried "slide_image" but it didnt scroll
with "retain=1" while "retain=2" and "retain=3" result in windows error
and crash.</tt>
<p><tt>Since most of my data fall in the 30mb to 610 mb range, I need absolutely
to open and display very large file. What is available? Pointers, sample
code, library, ideas, etc, I'm listening.... Just knowing that it have
to be handled on your own or that IDL provide facility (that they forgot
to explain in the manual) to deal with this will already be invaluable</tt>
<p><tt>Thanks</tt>
<p><tt>Sylvain Carette</tt>
<br><tt>VRML designer-composer</tt>
<br> </blockquote>
Have you tried FreeLook?
<br>I work with ENVI and I don't know what is missing in Freelook respect
to ENVI, but the visualization tools should work fine.
<br>www.rsinc.com
<p>This post is also valid for your previous message.
<p>Bye.
<pre>--
Nando Iavarone
Advanced Computer System - SPACE DIVISION
via Lazzaro Belli, 23
00040 Frascati - RM
Tel: +39-6-944091 (switchboard)
9440968 (direct)
E-mail:
f.iavarone@acsys.it
FrdndVrn@altavista.net</pre>
</html>
|
|
|
|
| Re: opening and display large file [message #21601 is a reply to message #21557] |
Thu, 31 August 2000 14:52 |
Kelly Dean
Messages: 92
Registered: March 1997
|
Member |
|
|
With RAM at 256 MB, there is no problem reading and working with large
data arrays. However, what I learn recently, is the monitor doesn't use
the RAM. If you don't have enough monitor memory, don't expect to
display and retain large files.
However, you can use the TV command to add tiles or lines on a big
4800x6000 window.
Try this....
Kelly
P.S. The same thing can be done in a Widget, so you can scroll around
the large display when the data array is bigger than your screen area.
============= Modify ReadTopo ======================
PRO TheWholeThing
file = 'e:\w140n90\w140n90.dem
topo = INTarr(4800,6000)
OpenR, lun, file, /Get_Lun, /swap_endian
ReadU, lun, topo
Close, lun
Free_Lun, lun
maxElev = MAX(topo)
WINDOW, 0, XSize=4800, YSize =6000, TITLE = " The Whole Thing "
FOR i = 0, 5999 DO BEGIN
TV, BYTscl(topo(*,i), MIN=0, MAX=maxElev ), 0, i
ENDFOR
END
==================================================
Sylvain Carette wrote:
> Thanks
> That's look almost what I have tryed. I'll use it "as is" to see the
> difference with mine but its mostly the fact I tryed to send the whole
> array to tv and slide_image.
> Also, I didnt close the file and free lun immediately after reading as
> you did - is this could cause memory leakage (when freezing, I was
> wondering after reset if I left big lump of anaccessible memory)?
> The only other difference is that instead of extracting a subset, I
> used congrid to resample a lightweigth display grid. Even though,
> nothing on tv and a small portion with slide_image which disapear
> immediately as soon I move a scroll bar (retain=0; retain 1 & 2 just
> make windows crash)...
> So this mean that 4800 x 6000 is too big for those two function? I
> still dont understand why it would be so big since according to the
> doc, you should be able to open a file as long you have enough memory
> and I have 250mb which should be enough for one gtopo tile no?
>
> I hope I dont sound like a bugger but beleive me, I tryed to do my
> homework before requesting help here.
> BYW, did you open usgs dem?
>
> Sylvain Carette
> VRML designer-composer
>
> Kelly Dean wrote:
>
>> This is what I use to access the GTOPO30 DEMs
>>
>> Kelly Dean
>> CSU/CIRA
>>
>> ================================================
>> PRO ReadTopo
>>
>> file = 'e:\w140n90\w140n90.dem
>> topo = INTarr(4800,6000)
>> ;
>> ; Open DEM, Swap_Endian if you are using a PC.
>> ;
>> OpenR, lun, file, /Get_Lun, /swap_endian
>> ReadU, lun, topo
>> Close, lun
>> Free_Lun, lun
>> ;
>> ; Cut out a small area and scale it.
>> ;
>> cut = BYTscl(topo(1500:2499,1500:2499), MIN=0 )
>> PRINT, ' MAX >',MAX( cut , MIN=MIN)
>> PRINT, ' MIN >',MIN
>> WINDOW, 0, XSize=1000, YSize=1000, TITLE='GTOP030'
>> TV, cut
>>
>> END
>> =================================================
>>
>> Sylvain Carette wrote:
>>
>>> Hi
>>> How do you manage to display very large images? What is the
>> options?
>>> From the manual, it seem that using "assoc" is a winner but maybe
>> with
>>> one little example it would be more clear how can this be (It seem
>> to
>>> me that it is an important aspect of programming with IDL since
>> most
>>> of the time you'll deal with larger than memory files).
>>> Even with assoc, dont you still have to copy from the file to an
>> array
>>> before display?
>>> I dont decipher exactly why and when to use readu or read_binary,
>> the
>>> file pointer, assoc or writing to an offscreen buffer.
>>>
>>> I tried to open and display a GTOPO30 tile (4800 x 6000 int). tv
>> could
>>> not display it - froze. I tried "slide_image" but it didnt scroll
>> with
>>> "retain=1" while "retain=2" and "retain=3" result in windows error
>> and
>>> crash.
>>>
>>> Since most of my data fall in the 30mb to 610 mb range, I need
>>> absolutely to open and display very large file. What is available?
>>
>>> Pointers, sample code, library, ideas, etc, I'm listening.... Just
>>
>>> knowing that it have to be handled on your own or that IDL provide
>>
>>> facility (that they forgot to explain in the manual) to deal with
>> this
>>> will already be invaluable
>>>
>>> Thanks
>>>
>>> Sylvain Carette
>>> VRML designer-composer
>>>
>
|
|
|
|
| Re: opening and display large file [message #21619 is a reply to message #21538] |
Thu, 31 August 2000 08:33 |
promashkin
Messages: 169
Registered: December 1999
|
Senior Member |
|
|
I would not sell anything to by a program when I can call the company
and arrange for a 30-day trial license to make sure the product fits my
needs. As far as I know, RSI is willning to do this.
Maybe you are right and other products are more suitable for you. I am
not sure if I'd buy ArcInfo if I needed to create VRML pages.
Cheers,
Pavel
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
