comp.lang.idl-pvwave archive: archive » IDLnetURL to test for files

Home » Public Forums » archive » IDLnetURL to test for files

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

IDLnetURL to test for files [message #88105]

Thu, 20 March 2014 10:27

Matthew Argall
Messages: 286
Registered: October 2011

Senior Member

I am trying to create a GetHttpDirList method for the IDLnetURL object (there exists a GetFtpDirList method already) by searching a webpage's content for <a href=""></a> tags and extracting the links. I treat the links as directories. Is there a way I can determine if a specific link is a file that can be downloaded? Perhaps with the response header or the response code when I set the URL_PATH to the file?

Thanks

Report message to a moderator

Re: IDLnetURL to test for files [message #88107 is a reply to message #88105]

Thu, 20 March 2014 14:14

Matthew Argall
Messages: 286
Registered: October 2011

Senior Member

> Is there a way I can determine if a specific link is a file that can be downloaded? Perhaps with the response header or the response code when I set the URL_PATH to the file?

I found a Content-Type field in the response header that tells me what the link is.... Oh, and there is a property named CONTENT_TYPE, too. Go figure... Unfortunately, the only way I seem to be able to trigger a response is by using the Get method.

Is there a way to trigger response without downloading the contents of a page first? I am currently using something like

!Null = Get(/String_Array)
self -> GetProperty, CONTENT_TYPE=ct
IF ct NE 'html' THEN !Null = Get(FILENAME=filename)

But this requires two calls to Get.

Report message to a moderator

Re: IDLnetURL to test for files [message #88113 is a reply to message #88107]

Thu, 20 March 2014 19:27

Matthew Argall
Messages: 286
Registered: October 2011

Senior Member

> Is there a way to trigger response without downloading the contents of a page first?

I decided to spawn curl and scan use its response for the content type. It is much faster when searching for data files.

spawn, 'curl -I -s' + url, response

To check for curl, I try

spawn, 'which curl', response
if response ne '' then hasCurl = 1 else hasCurl = 0

Report message to a moderator

Previous Topic:	IDL's BESELJ returns NAN for small argument and large order
Next Topic:	CV_Coord function

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

]

Current Time: Wed Oct 08 13:36:13 PDT 2025

Total time taken to generate the page: 0.00469 seconds