comp.lang.idl-pvwave archive: archive » Get directory list from HTTP Server

Home » Public Forums » archive » Get directory list from HTTP Server

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Get directory list from HTTP Server [message #84884]

Mon, 17 June 2013 06:54

corinnefrey
Messages: 18
Registered: November 2006

Junior Member

Hi all,

I was using an IDL script for batch-downloading MODIS satellite data. Unfortunately thFTP is not supported anymore. Only HTTP is supported.

So I have to modify my script. The OBJ_NEW and the GET function should also work for HTTP, however, what do I use in replacement for GetFTPDirList? I need to get the directory list, as I just need selected scenes.

Cheers,
Corinne

Below the script that needs to be modified:

PRO download_modis_http

targetDir = 'MOLT/MOD11A1.005'

oUrl = OBJ_NEW('IDLnetUrl', URL_SCHEME='HTTP', $
URL_HOST='e4ftl01.cr.usgs.gov', $
URL_USERNAME='anonymous', URL_PASSWORD='my email', $
URL_PATH=targetDir)

dirArray = oURL->GetFTPDirList()
dirArray = dirArray[1:N_ELEMENTS(dirArray)-1]

dirArray = STRMID(dirArray,37,10)

year = FIX(STRMID(dirArray, 0,4))
month = FIX(STRMID(dirArray, 5,2))
valid = WHERE(year EQ 2012 AND month GE 10)
dirArray = dirArray[valid]

FOR i = 0,N_ELEMENTS(dirArray)-1 DO BEGIN

oURL->SetProperty, URL_PATH=targetDir+'/'+dirArray[i]
fileArray = oURL->GetFTPDirList()

IF fileArray[0] EQ 'total 0' THEN BEGIN
;dont do anything, this file is missing on the ftp server
stop
ENDIF ELSE BEGIN
fileArray = fileArray[1:N_ELEMENTS(fileArray)-1]

;Exclude all jpgs from the list
jpg = STRPOS(fileArray, 'jpg') EQ -1
valid = WHERE(jpg EQ 1)

;Make a list with all hdf files
hdf = STRPOS(fileArray, 'hdf.xml') NE - 1
valid = WHERE(hdf EQ 1)
fileArray = fileArray[valid]
fileArray = STRMID(fileArray,38,45)

FOR j = 0, N_ELEMENTS(fileArray)-1 DO BEGIN
oURL->SetProperty, URL_PATH=targetDir+'/'+dirArray[i]+'/'+fileArray[j]

tile = STRMID(fileArray[j],17, 6)
year = STRMID(fileArray[j],9, 4)

IF tile EQ 'h25v05' OR tile EQ 'h26v05' OR tile EQ 'h26v06' OR tile EQ 'h27v06' OR tile EQ 'h27v07' OR tile EQ 'h28v07' OR tile EQ 'h28v08' THEN BEGIN

downloadPath = filepath(fileArray[j], ROOT_DIR='d:/CMF/Mekong Data/MODIS/MOD11A1v5/'+tile+'/'+year)
print, oURL->Get(FILENAME=downloadPath)
ENDIF
ENDFOR
ENDELSE
ENDFOR
END

Report message to a moderator