| connecting to a server to retrieve data (optimizing) [message #41806] |
Sat, 27 November 2004 20:57  |
nunoragil
Messages: 3
Registered: November 2004
|
Junior Member |
|
|
Hi listers,
I'm fetching some data from the web with the following routine. This
does the trick but it gets very slow, for it reads the data on the
server and writes it localy bit by bit in a loop.
Does anyone knows if it is possible to send the request and simply get
the entire data in one turn?
If not, can I read the data bit by bit and append it inside a variable
instead of first writing it all to the disk in a file and then reading
that same file afterwards?
Regards
Nuno
FUNCTION getCoverage, coverage, xMin, xMax, yMin, yMax, format
getRequest='GET /cgi-bin/wcs?REQUEST=GetCoverage&COVERAGE='+coverage+'&a mp;SERVICE=WCS&'+
$
'map=wcs.map&BBOX='+xMin+','+yMin+','+xMax+','+yMax+'&am p;FORMAT='+format+
$
'&CRS=EPSG:4326&RESX=0.00083333&RESY=0.00083333'
host = 'iceds.ge.ucl.ac.uk'
port = 80
SOCKET, unit, host, port, /GET_LUN
PRINTF, unit, getRequest
WIDGET_CONTROL, /HOURGLASS
OPENW,img, 'c:\temp.tif', /GET_LUN
byte_in = 0B
WHILE EOF(unit) EQ 0 DO BEGIN
READU, unit, byte_in
WRITEU,img,byte_in
ENDWHILE
FREE_LUN, unit, img
CLOSE, /ALL
RETURN, READ_TIFF ('c:\temp.tif', GEOTIFF = imgGeoTags)
END
|
|
|
|
| Re: connecting to a server to retrieve data (optimizing) [message #41941 is a reply to message #41806] |
Mon, 29 November 2004 15:16  |
Craig Markwardt
Messages: 1869
Registered: November 1996
|
Senior Member |
|
|
nunoragil@yahoo.com (Nuno Gil) writes:
> Craig,
> I'm having some problems writing the image.
> If you try this link it will download a 16-bit signed Integer tiff
> image with 2816 KB:
> http://iceds.ge.ucl.ac.uk/cgi-bin/wcs?REQUEST=GetCoverage&am p;COVERAGE=srtm&SERVICE=WCS&map=wcs.map&BBOX=46, -25,47,-24&FORMAT=GEOTIFFINT16&CRS=EPSG:4326&RES X=0.00083333&RESY=0.00083333
>
> However, if trying to get it via IDL with:
> img=INTARR(1200,1200)
> READU, in, img
> WRITEU,out,img
> I can only get a 2813 KB file and cannot open it properly.
> 1200x1200 are the number of pixels of the image.
> By the way, if I declare the array bigger than this (lets say
> 2001x2001) the READU issues an "End of file encountered" error.
What if you read the image in smaller chunks instead of reading it all
at once?
Craig
--
------------------------------------------------------------ --------------
Craig B. Markwardt, Ph.D. EMAIL: craigmnet@REMOVEcow.physics.wisc.edu
Astrophysics, IDL, Finance, Derivatives | Remove "net" for better response
------------------------------------------------------------ --------------
|
|
|
|
| Re: connecting to a server to retrieve data (optimizing) [message #41944 is a reply to message #41806] |
Mon, 29 November 2004 04:45 |
nunoragil
Messages: 3
Registered: November 2004
|
Junior Member |
|
|
andrew.cool@dsto.defence.gov.au (Andrew Cool) wrote in message news:<c6d70400.0411281810.7b804992@posting.google.com>...
> Craig Markwardt <craigmnet@REMOVEcow.physics.wisc.edu> wrote in message news:<on8y8mvzv1.fsf@cow.physics.wisc.edu>...
>> nunoragil@yahoo.com (Nuno Gil) writes:
>>
>>
>>> Hi listers,
>>>
>>> I'm fetching some data from the web with the following routine. This
>>> does the trick but it gets very slow, for it reads the data on the
>>> server and writes it localy bit by bit in a loop.
>>> Does anyone knows if it is possible to send the request and simply get
>>> the entire data in one turn?
>>
>> Yes, read a byte array instead of a scalar byte. You choose the size
>> of the array you want to read, and if there is less data available,
>> then IDL will only read that much. The number of bytes read is
>> returned in the TRANSFER_COUNT keyword. You can then use that same
>> number to write the data to your file.
>>
>> Craig
>
>
> Nuno and Craig,
>
> I hope you guys have emailed RSI to add your support to the request
> for RSI to intrpduce an http/ftp client into IDL?
>
> See my recent post under "Support for getting HTTP/FTP clients into IDL"
>
> Cheers,
>
> Andrew
> DSTO, Adelaide, South Oz
Craig,
I'm having some problems writing the image.
If you try this link it will download a 16-bit signed Integer tiff
image with 2816 KB:
http://iceds.ge.ucl.ac.uk/cgi-bin/wcs?REQUEST=GetCoverage&am p;COVERAGE=srtm&SERVICE=WCS&map=wcs.map&BBOX=46, -25,47,-24&FORMAT=GEOTIFFINT16&CRS=EPSG:4326&RES X=0.00083333&RESY=0.00083333
However, if trying to get it via IDL with:
img=INTARR(1200,1200)
READU, in, img
WRITEU,out,img
I can only get a 2813 KB file and cannot open it properly.
1200x1200 are the number of pixels of the image.
By the way, if I declare the array bigger than this (lets say
2001x2001) the READU issues an "End of file encountered" error.
Thank you and regards,
Nuno
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
