| Re: IDL and OPENDAP [message #71617] |
Fri, 09 July 2010 14:43 |
Kenneth P. Bowman
Messages: 585
Registered: May 2000
|
Senior Member |
|
|
In article <MPG.26a1365ffad4c23d9896f2@news.giganews.com>,
David Fanning <news@dfanning.com> wrote:
> Kenneth P. Bowman writes:
>
>> To be more political about this, OPENDAP sounds like a good
>> way for the data aristocrats to distribute data to the
>> yearning masses.
>
> I think this is *exactly* the point of it. But, and I
> think this is the best part (at least as it was described
> to me a couple of days ago), the data user doesn't have to
> download the entire netCDF file. They can actually go into
> a file and get just that portion of the file they are
> particularly interested in. For example, you could get
> just a particular time slice in an image cube.
I'm not saying OPENDAP is a bad thing, just that it falls
short of solving the 'data distribution problem'.
Cheers, Ken
|
|
|
|
| Re: IDL and OPENDAP [message #71618 is a reply to message #71617] |
Fri, 09 July 2010 14:04  |
Michael Galloy
Messages: 1114
Registered: April 2006
|
Senior Member |
|
|
On 7/9/10 2:37 PM, David Fanning wrote:
> Kenneth P. Bowman writes:
>
>> To be more political about this, OPENDAP sounds like a good
>> way for the data aristocrats to distribute data to the
>> yearning masses.
>
> I think this is *exactly* the point of it. But, and I
> think this is the best part (at least as it was described
> to me a couple of days ago), the data user doesn't have to
> download the entire netCDF file. They can actually go into
> a file and get just that portion of the file they are
> particularly interested in. For example, you could get
> just a particular time slice in an image cube.
Exactly, a file may have many variables, each of which could be quite
large. Clients can grab individual variables or even slice a given
variable with IDL-like array notation of the type start:stop:stride.
This is geared towards a scenario where a data distribution center is
providing data to many users, but I have seen locations which provide
many users a staging area which they can copy files over to and then the
users have access to this data via DAP from anywhere inside that network
(of course this could be made available to the Internet at large, if you
were OK with that).
Mike
--
www.michaelgalloy.com
Research Mathematician
Tech-X Corporation
|
|
|
|
| Re: IDL and OPENDAP [message #71619 is a reply to message #71618] |
Fri, 09 July 2010 13:37 |
David Fanning
Messages: 11724
Registered: August 2001
|
Senior Member |
|
|
Kenneth P. Bowman writes:
> To be more political about this, OPENDAP sounds like a good
> way for the data aristocrats to distribute data to the
> yearning masses.
I think this is *exactly* the point of it. But, and I
think this is the best part (at least as it was described
to me a couple of days ago), the data user doesn't have to
download the entire netCDF file. They can actually go into
a file and get just that portion of the file they are
particularly interested in. For example, you could get
just a particular time slice in an image cube.
Cheers,
David
--
David Fanning, Ph.D.
Fanning Software Consulting, Inc.
Coyote's Guide to IDL Programming: http://www.dfanning.com/
Sepore ma de ni thue. ("Perhaps thos speakest truth.")
|
|
|
|
| Re: IDL and OPENDAP [message #71620 is a reply to message #71619] |
Fri, 09 July 2010 13:13 |
Kenneth P. Bowman
Messages: 585
Registered: May 2000
|
Senior Member |
|
|
In article <i17t9t$79p$1@news.eternal-september.org>,
mgalloy <mgalloy@gmail.com> wrote:
> On 7/9/10 12:48 PM, Kenneth P. Bowman wrote:
>> Is anyone using IDL and OPENDAP to access netCDF and HDF files?
>> Can you write as well as read files?
>>
>> I see that there is an OPENDAP IDL client, but I can't find any
>> actual examples of how it is used. I am concerned that server
>> installation and configuration may be complex. It appears
>> that the server software might have complex dependencies.
>
> There is an DAP IDL client from OPeNDAP which is a DLM around a C
> OPeNDAP implementation. It requires building both the C code and the
> DLM. Alternatively, I have a pure IDL DAP client that does not require
> anything else (its just a .sav file to drop into your IDL_PATH). It
> comes with IDLdoc documentation with examples. Let me know if you are
> interested in trying this.
>
> Clients can not write files, only read them.
>
> The server is another story: there are many servers available. Because
> the clients and servers agree on a common standard (DAP), you should be
> able to pick your clients and servers independently. Some of the servers
> are (fairly) easy to install and configure, some more difficult. The
> PyDAP server (pydap.org) is well supported and fairly easy to install
> (it helps if you have some experience installing Python packages). It is
> written in Python, but that only matters when installing, modifying, or
> creating a plugin for it. OPeNDAP has their own server, Hyrax, which
> runs under Tomcat. It is written in Java, but has dependencies written
> in C++ and C.
>
> Mike
Thanks, Mike. That is very helpful.
The OPENDAP web site seems to be by programmers and for
programmers, which leaves scientists like me (who happen to
program a lot) at a loss.
OPENDAP functionality would be useful, but it does seem to
me to be rather complex for something that only does half
of what I need to do. (We also write a lot of netCDF files.)
I thought that it might be a way to simplify some of our
NFS filesystem cross-mounting complexity, but I see that is
not the case.
To be more political about this, OPENDAP sounds like a good
way for the data aristocrats to distribute data to the
yearning masses.
Ken
|
|
|
|
| Re: IDL and OPENDAP [message #71621 is a reply to message #71620] |
Fri, 09 July 2010 12:29 |
Michael Galloy
Messages: 1114
Registered: April 2006
|
Senior Member |
|
|
On 7/9/10 12:48 PM, Kenneth P. Bowman wrote:
> Is anyone using IDL and OPENDAP to access netCDF and HDF files?
> Can you write as well as read files?
>
> I see that there is an OPENDAP IDL client, but I can't find any
> actual examples of how it is used. I am concerned that server
> installation and configuration may be complex. It appears
> that the server software might have complex dependencies.
There is an DAP IDL client from OPeNDAP which is a DLM around a C
OPeNDAP implementation. It requires building both the C code and the
DLM. Alternatively, I have a pure IDL DAP client that does not require
anything else (its just a .sav file to drop into your IDL_PATH). It
comes with IDLdoc documentation with examples. Let me know if you are
interested in trying this.
Clients can not write files, only read them.
The server is another story: there are many servers available. Because
the clients and servers agree on a common standard (DAP), you should be
able to pick your clients and servers independently. Some of the servers
are (fairly) easy to install and configure, some more difficult. The
PyDAP server (pydap.org) is well supported and fairly easy to install
(it helps if you have some experience installing Python packages). It is
written in Python, but that only matters when installing, modifying, or
creating a plugin for it. OPeNDAP has their own server, Hyrax, which
runs under Tomcat. It is written in Java, but has dependencies written
in C++ and C.
Mike
--
www.michaelgalloy.com
Research Mathematician
Tech-X Corporation
|
|
|
|
comp.lang.idl-pvwave archive
Members
Search
Help
Login
Home
