Messages from Usenet group comp.lang.idl-pvwave, compiled by Paulo Penteado
On Monday, March 11, 2013 11:32:16 PM UTC+1, mark...@gmail.com wrote: > Hi All > > > > I have a dataset which is noisy, but contains a small number (< 5) broad peaks, much like the data given in http://carlwillis.files.wordpress.com/2011/03/wellrich_spect rum.jpg (just an pictorial example, not the actual data). You'll notice that there is both small amplitude noise throughout the data together with a small number of broad peaks. > > > > The majority of peak finding routines I've found work on the basis of derivatives/gradients which find hundreds of "local" peaks in the small amplitude noise. I'm interested in tracking the broader, large peaks only. Ideally I'm after a peak finding routine which filters on the basis of peak width and allows some filtering based around amplitude so certain peaks can be selected over others. > > > > Just to add to make things even more complicated ( :-) ) the data is often quite sparse meaning that while the broad peaks are always present, they may be not be as well formed as in the example image above. > > > > Can anyone suggest a robust method or existing IDL routine that would help pick out the broad peaks only? > > > > Any help or advice would be massively appreciated! > > > > Mark Not an expert, but normally you smooth the data to get rid of the local peaks and then look for changes in sign of the first deriv (zeros or). Then you may use a slope detection/thresholding to further reduce the number of wrong peaks. You can iteratively reduce the smoothing and monitor the positions of the peaks that appear... This should show the "correct" peaks first. Clearly though, if you know what elements you're expecting, you will know the approximate energy and that is normally of great help. If you also can guess the width, that is another helper. Can't help more. Sorry. Cheers, Helder
First thing I would try is remove the high frequency noise. If your data is regularly sampled, Fourier transforming it, truncating the result (perhaps applying a suitable low-pass filter and then zerofilling?), and then transforming it back to your original domain would do that. The result at that point would preserve the "spectral integrity" of the lower frequency peaks you seek to isolate. You could then use the derivative peak-finding methodology you mention. The big issue with using Fourier transform is your mention of data sparsity. If the sampling of your data is below a critical threshold for the various peak features, the Fourier approach won't help. But, it's pretty cheap to give it a shot. :o) cheers, paulv On 03/11/13 18:32, markjamie@gmail.com wrote: > Hi All > > I have a dataset which is noisy, but contains a small number (< 5) > broad peaks, much like the data given in > http://carlwillis.files.wordpress.com/2011/03/wellrich_spect rum.jpg > (just an pictorial example, not the actual data). You'll notice that > there is both small amplitude noise throughout the data together with > a small number of broad peaks. > > The majority of peak finding routines I've found work on the basis of > derivatives/gradients which find hundreds of "local" peaks in the > small amplitude noise. I'm interested in tracking the broader, large > peaks only. Ideally I'm after a peak finding routine which filters on > the basis of peak width and allows some filtering based around > amplitude so certain peaks can be selected over others. > > Just to add to make things even more complicated ( :-) ) the data is > often quite sparse meaning that while the broad peaks are always > present, they may be not be as well formed as in the example image > above. > > Can anyone suggest a robust method or existing IDL routine that would > help pick out the broad peaks only? > > Any help or advice would be massively appreciated! > > Mark
On Monday, March 11, 2013 6:32:16 PM UTC-4, mark...@gmail.com wrote: > Hi All > > > > I have a dataset which is noisy, but contains a small number (< 5) broad peaks, much like the data given in http://carlwillis.files.wordpress.com/2011/03/wellrich_spect rum.jpg (just an pictorial example, not the actual data). You'll notice that there is both small amplitude noise throughout the data together with a small number of broad peaks. > > > > The majority of peak finding routines I've found work on the basis of derivatives/gradients which find hundreds of "local" peaks in the small amplitude noise. I'm interested in tracking the broader, large peaks only. Ideally I'm after a peak finding routine which filters on the basis of peak width and allows some filtering based around amplitude so certain peaks can be selected over others. > > > > Just to add to make things even more complicated ( :-) ) the data is often quite sparse meaning that while the broad peaks are always present, they may be not be as well formed as in the example image above. > > > > Can anyone suggest a robust method or existing IDL routine that would help pick out the broad peaks only? > > > > Any help or advice would be massively appreciated! I agree with the other posters that smoothing is your first stop. I'm not sure about FFT though. Your data have a large dynamic range - it's plotted on log scale! - so you are going to get strange aliases. (The FFT assumes your data has circular symmetry, which it obviously doesn't). You seem to know the approximate width of each of your peaks, about 40 keV. In principle you can make a simple contrast enhancer using that knowledge. In pseudocode it might me, SMOOTH_SIGNAL = SMOOTH(Y,(40 keV)) - SMOOTH(Y,(160 keV)) Edge effects will need to be dealt with. It looks like the peaks become broader at higher energies, so you might need to write a fancier version of SMOOTH() that is adaptive to energy. If you know very approximately where the peaks must be found, then you can take the approach of fitting the peak amplitude. If the position is only known approximately, then it's wise to do it iteratively: first fit amplitude only as a rough cut, then allow centroid to vary, then width. Of course this also relies on having a semi-accurate baseline=continuum model. Craig
On Monday, 11 March 2013 23:32:16 UTC+1, mark...@gmail.com wrote: > Hi All > > > > I have a dataset which is noisy, but contains a small number (< 5) broad peaks, much like the data given in http://carlwillis.files.wordpress.com/2011/03/wellrich_spect rum.jpg (just an pictorial example, not the actual data). You'll notice that there is both small amplitude noise throughout the data together with a small number of broad peaks. [...] > Can anyone suggest a robust method or existing IDL routine that would help pick out the broad peaks only? For this kind of application, you might want to have look at the concept "significance in scale space", pioneered by Steve Marron. Examples from 1D and 2D here: http://www.unc.edu/~marron/Movies/SSS_movies.html Papers here (quite technical): http://www.unc.edu/~marron/marron_papers.html No IDL code that I know of, but there should be some Matlab code to be found (SiZer by Marron and Chaudhuri), and if I remember correcty also a Java port. -- Yngvar
On Tuesday, 12 March 2013 10:44:07 UTC+1, Yngvar Larsen wrote: > On Monday, 11 March 2013 23:32:16 UTC+1, mark...@gmail.com wrote: >> Can anyone suggest a robust method or existing IDL routine that would help pick out the broad peaks only? > > For this kind of application, you might want to have look at the concept "significance in scale space", pioneered by Steve Marron. Examples from 1D and 2D here: > > http://www.unc.edu/~marron/Movies/SSS_movies.html > > Papers here (quite technical): > http://www.unc.edu/~marron/marron_papers.html And a more accessible intro here: http://www.unc.edu/~marron/DataAnalyses/SiZer_Intro.html -- Yngvar