comp.lang.idl-pvwave archive
Messages from Usenet group comp.lang.idl-pvwave, compiled by Paulo Penteado

Home » Public Forums » archive » Some histogram magic help required - gridding/counting large dataset
Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend 
Switch to threaded view of this topic Create a new topic Submit Reply
Some histogram magic help required - gridding/counting large dataset [message #88153] Tue, 25 March 2014 03:11 Go to next message
rjp23 is currently offline  rjp23
Messages: 97
Registered: June 2010
Member
I have 2 large datasets that I want to plot as the x and y data on a scatter plot.

However I'd also like to grid this data in the x and y direction and count how many datapoints fall into each grid cell.

I know I should be able to do this with hist_nd but I just can't figure out exactly how to do it.

Cheers

Rob
Re: Some histogram magic help required - gridding/counting large dataset [message #88156 is a reply to message #88153] Tue, 25 March 2014 05:56 Go to previous messageGo to next message
Andy Sayer is currently offline  Andy Sayer
Messages: 127
Registered: February 2009
Senior Member
If it is a 2D (x/y) dataset you could use the built-in function hist_2d: http://www.exelisvis.com/docs/HIST_2D.html

What exactly is the issue you are having? You can specify bin sizes and min/max values with keywords min1, max1, bin1, min2, max2, bin2.

Andy

On Tuesday, March 25, 2014 6:11:03 AM UTC-4, rj...@le.ac.uk wrote:
> I have 2 large datasets that I want to plot as the x and y data on a scatter plot.
>
>
>
> However I'd also like to grid this data in the x and y direction and count how many datapoints fall into each grid cell.
>
>
>
> I know I should be able to do this with hist_nd but I just can't figure out exactly how to do it.
>
>
>
> Cheers
>
>
>
> Rob
Re: Some histogram magic help required - gridding/counting large dataset [message #88171 is a reply to message #88156] Tue, 25 March 2014 07:59 Go to previous messageGo to next message
rjp23 is currently offline  rjp23
Messages: 97
Registered: June 2010
Member
It's two 1D datasets with e.g. 20,000 values.

How I've used hist_nd before was to put these into a 2D dataset [20,000, 2] and then run hist_nd on that data but that doesn't seem to work and is only returning 1 bin, even though I'm specifying e.g. [100,2] bins.



On Tuesday, March 25, 2014 12:56:48 PM UTC, AMS wrote:
> If it is a 2D (x/y) dataset you could use the built-in function hist_2d: http://www.exelisvis.com/docs/HIST_2D.html
>
>
>
> What exactly is the issue you are having? You can specify bin sizes and min/max values with keywords min1, max1, bin1, min2, max2, bin2.
>
>
>
> Andy
>
>
Re: Some histogram magic help required - gridding/counting large dataset [message #88172 is a reply to message #88153] Tue, 25 March 2014 08:28 Go to previous messageGo to next message
David Fanning is currently offline  David Fanning
Messages: 11724
Registered: August 2001
Senior Member
rjp23@le.ac.uk writes:

> I have 2 large datasets that I want to plot as the x and y data on a scatter plot.
>
> However I'd also like to grid this data in the x and y direction and count how many datapoints fall into each grid cell.
>
> I know I should be able to do this with hist_nd but I just can't figure out exactly how to do it

I believe your thinking about this is All Wrong. :-)

Just histogram each 1D data set into the same number of bins.

h1 = cgHistogram(data1, NBins=100, Reverse_Indices=ri1)
h2 = cgHistogram(data2, NBins=100, Reverse_Indices=ri2)

When you want to find out, for example how many "hits" you have in bin
25 in the first data set and bin 45 in the second data set, you do this:

b25indices = cgReverseIndices(ri1, 24, COUNT=c1)
b45indices = cgReverseIndices(ri2, 44, COUNT=c2)

IF (c1) GT 0) && (c2 GT 0) THEN BEGIN
indices = cgSetIntersection(b25indices, b45indices, COUNT=count)
IF count GT 0 THEN BEGIN
Print, 'Matches: ', count
ENDIF ELSE Print, 'No matches.
ELSE Print, 'No matches'

If you want to plot the points in this intersection of bins:

cgPlot, data1[indices], data2[indices], PSYM=1

Cheers,

David
--
David Fanning, Ph.D.
Fanning Software Consulting, Inc.
Coyote's Guide to IDL Programming: http://www.idlcoyote.com/
Sepore ma de ni thue. ("Perhaps thou speakest truth.")
Re: Some histogram magic help required - gridding/counting large dataset [message #88173 is a reply to message #88153] Tue, 25 March 2014 09:10 Go to previous message
Phillip Bitzer is currently offline  Phillip Bitzer
Messages: 223
Registered: June 2006
Senior Member
On Tuesday, March 25, 2014 5:11:03 AM UTC-5, rj...@le.ac.uk wrote:
> I have 2 large datasets that I want to plot as the x and y data on a scatter plot.
>
>
>
> However I'd also like to grid this data in the x and y direction and count how many datapoints fall into each grid cell.
>
>
>

Here's a snippet for how to do what (I think) you're asking:

https://www.dropbox.com/s/j9ffh1ns3wcguq4/histo_notes.pdf

In the notes, h2d will contain the counts per bin.
  Switch to threaded view of this topic Create a new topic Submit Reply
Previous Topic: a GDL Workshop in June 2014
Next Topic: cgwindow unable to create PDF file in IDL 8.3

-=] Back to Top [=-
[ Syndicate this forum (XML) ] [ RSS ] [ PDF ]

Current Time: Wed Oct 08 15:08:00 PDT 2025

Total time taken to generate the page: 0.00562 seconds