comp.lang.idl-pvwave archive: archive » TRICKY TASK USING AWK

Home » Public Forums » archive » TRICKY TASK USING AWK

Show: Today's Messages :: Show Polls :: Message Navigator
E-mail to friend

Switch to threaded view of this topic

Create a new topic

Submit Reply

TRICKY TASK USING AWK [message #92769]

Fri, 26 February 2016 01:20

Sapna Mishra is currently offline

Sapna Mishra
Messages: 66
Registered: December 2015

Member

Hello all,

What I have is:

IDL> i=1
IDL> $ awk -v var="$i" '{print $var}' tmp1.dat> tmp3.dat

Upto this it is working ok. now what I want:

IDL> fname[i]='QB0332_CHIP1_GRIS_600V_SCIENCE_SKY_SPECTRUM_FORS1. 2007_11_11T06_43_02.032redwave.dat'
IDL> $ awk -v var="$i" '{print $var}' fname[i]> fnameout[i]

Can I do it some how??? I found it most tricky. Basically can we use variables defined inside idl into shell commands.

Anyone know how to solve this tricky job???

Report message to a moderator

Re: TRICKY TASK USING AWK [message #92772 is a reply to message #92769]

Fri, 26 February 2016 04:03

dg86 is currently offline

dg86
Messages: 118
Registered: September 2012

Senior Member

On Friday, February 26, 2016 at 4:20:20 AM UTC-5, Sapna Mishra wrote:
> Hello all,
>
> What I have is:
>
> IDL> i=1
> IDL> $ awk -v var="$i" '{print $var}' tmp1.dat> tmp3.dat
>
> Upto this it is working ok. now what I want:
>
> IDL> fname[i]='QB0332_CHIP1_GRIS_600V_SCIENCE_SKY_SPECTRUM_FORS1. 2007_11_11T06_43_02.032redwave.dat'
> IDL> $ awk -v var="$i" '{print $var}' fname[i]> fnameout[i]
>
> Can I do it some how??? I found it most tricky. Basically can we use variables defined inside idl into shell commands.
>
> Anyone know how to solve this tricky job???

Without delving too deeply into the details of your specific application, it sounds like
you want to build up a string containing the desired command, and then to use SPAWN to
execute that command. The general idea is

IDL> cmd = 'awk my_awk_command ' + fname[i] + ' > ' + fnameout[i]
IDL> spawn, cmd

Note that you'll have to be careful about the quotation marks in the IDL string
that I've called 'cmd'. To get this right, you'll have to read the documentation on
IDL strings, specifically how to escape special characters (such as quotation marks).

You can double-check that you've built up the command properly by printing the string:

IDL> print, cmd

This should print exactly the command that you'd like to execute.

Finally, you'll have to be careful to insert spaces between the substrings that make up
your command so that the result is syntactically correct for execution by the shell.
In my sketched solution, I've deliberately placed a space after the placeholder
pseudocode 'my_awk_command'.

All the best,

David

Report message to a moderator

Re: TRICKY TASK USING AWK [message #92774 is a reply to message #92769]

Fri, 26 February 2016 06:55

Sapna Mishra is currently offline

Sapna Mishra
Messages: 66
Registered: December 2015

Member

if my_awk command is : -v var="$i" '{print $var} then how i will proceed??

I am doing:

cmd ='awk -v var="$i" '{print $var}' ' + fname[i] + ' > ' + fnameout[i]
and on:
print,cmd I am getting:
IDL> print,cmd
% PRINT: Variable is undefined: CMD.
% Execution halted at: $MAIN$

Report message to a moderator

Re: TRICKY TASK USING AWK [message #92776 is a reply to message #92774]

Fri, 26 February 2016 07:09

Mats Löfdahl is currently offline

Mats Löfdahl
Messages: 263
Registered: January 2012

Senior Member

Den fredag 26 februari 2016 kl. 15:55:43 UTC+1 skrev Sapna Mishra:
> if my_awk command is : -v var="$i" '{print $var} then how i will proceed??
>
> I am doing:
>
> cmd ='awk -v var="$i" '{print $var}' ' + fname[i] + ' > ' + fnameout[i]
> and on:
> print,cmd I am getting:
> IDL> print,cmd
> % PRINT: Variable is undefined: CMD.
> % Execution halted at: $MAIN$

You need to escape the single quotes inside the single-quoted string:

IDL> cmd ='awk -v var="$i" ''{print $var}'' '
IDL> print,cmd
awk -v var="$i" '{print $var}'

Report message to a moderator

Re: TRICKY TASK USING AWK [message #92789 is a reply to message #92769]

Mon, 29 February 2016 07:57

Sapna Mishra is currently offline

Sapna Mishra
Messages: 66
Registered: December 2015

Member

Hello,

I am using this part of IDL

for i=1,c do begin
fnameo[i]='tmp'
cmd ='awk -v var="$i" ''{print $var}'' ' + fname[i] + ' > ' + fnameo[i]
spawn, cmd
endfor

where tmp is a new file i want to create, it is showing error fnameo[i] not exist.
Where I am wrong???

Report message to a moderator

Re: TRICKY TASK USING AWK [message #92790 is a reply to message #92789]

Mon, 29 February 2016 08:07

Paul Van Delst[1] is currently offline

Paul Van Delst[1]
Messages: 1157
Registered: April 2002

Senior Member

Hello,

On 02/29/16 10:57, Sapna Mishra wrote:
> Hello,
>
> I am using this part of IDL
>
> for i=1,c do begin
> fnameo[i]='tmp'
> cmd ='awk -v var="$i" ''{print $var}'' ' + fname[i] + ' > ' + fnameo[i]
> spawn, cmd
> endfor
>
> where tmp is a new file i want to create, it is showing error fnameo[i] not exist.
> Where I am wrong???

I'm not an awk user so forgive the silly question, but what is the awk
command above supposed to do?

It seems pretty simple (despite my lack of awknowledge) - isn't there an
IDL way of doing what you want and skip using SPAWN altogether?

cheers,

paulv

Report message to a moderator

Re: TRICKY TASK USING AWK [message #92791 is a reply to message #92769]

Mon, 29 February 2016 08:58

Sapna Mishra is currently offline

Sapna Mishra
Messages: 66
Registered: December 2015

Member

Actually I want to grep a particular column say column 2 from fname[i] and want to save it to fnameo[i] as an output. In IDL I don't know how to read any file having unknown numbers of columns because in READCOL to read file we need to mention a variable for each column in which particular column will get stored.

So this is why i wanted to grep value of unknown numbers of column into another file.

Awk command grep columns.

Report message to a moderator

Re: TRICKY TASK USING AWK [message #92792 is a reply to message #92789]

Mon, 29 February 2016 09:33

Heinz Stege is currently offline

Heinz Stege
Messages: 189
Registered: January 2003

Senior Member

Does the error message really contain the string "fnameo[i]"? If yes,
then this the error message must stem from IDL, not from the OS.

What happens if you replace the line
| cmd ='awk -v var="$i" ''{print $var}'' ' + fname[i] + ' > ' + fnameo[i]
by
| print,'awk -v var="$i" ''{print $var}'' ' + fname[i] + ' > ' + fnameo[i]
?

Heinz

Report message to a moderator

Re: TRICKY TASK USING AWK [message #92793 is a reply to message #92791]

Mon, 29 February 2016 09:44

wlandsman is currently offline

wlandsman
Messages: 743
Registered: June 2000

Senior Member

On Monday, February 29, 2016 at 11:58:32 AM UTC-5, Sapna Mishra wrote:
> Actually I want to grep a particular column say column 2 from fname[i] and want to save it to fnameo[i] as an output. In IDL I don't know how to read any file having unknown numbers of columns because in READCOL to read file we need to mention a variable for each column in which particular column will get stored.

Here is a short IDL program to read the number of columns in a file, e.g.
ncol = findnumcol('myfile.dat')

function findnumcol, file
; Return the number of columns of data in a file
; Just reads the first line and assume rest of the file
; is the same. Could add code to skip header lines
openr,lun,file,/get_lun
st = ''
readf,lun,st
free_lun,lun
tmp = strsplit(st,count=ncol)
return,ncol
end

Report message to a moderator

Re: TRICKY TASK USING AWK [message #92794 is a reply to message #92769]

Mon, 29 February 2016 22:08

Sapna Mishra is currently offline

Sapna Mishra
Messages: 66
Registered: December 2015

Member

Hi wlandsman ,
Is this code reads the contents of columns or it returns the no. of columns to be read???
If it is the case then my problem remains unsolved because i want to READ columns from a file without knowing how much column does this file has. Say If I have infinite number of columns And I want to read 100th column how would I do that, shall I need to give 100 parameters in Readcol?? This seems tough to me.

Report message to a moderator

Re: TRICKY TASK USING AWK [message #92795 is a reply to message #92769]

Mon, 29 February 2016 22:12

Sapna Mishra is currently offline

Sapna Mishra
Messages: 66
Registered: December 2015

Member

Hi Heinz Stege ,
Error is as following:

IDL> cmd ='awk -v var="$i" ''{print $var}'' ' + fname[i] + ' > ' + fnameo[i]
% Variable is undefined: I.
% Execution halted at: $MAIN$
IDL> print,'awk -v var="$i" ''{print $var}'' ' + fname[i] + ' > ' + fnameo[i]
% Variable is undefined: I.
% Execution halted at: $MAIN$
IDL>

Report message to a moderator

Re: TRICKY TASK USING AWK [message #92796 is a reply to message #92795]

Tue, 01 March 2016 01:37

Mats Löfdahl is currently offline

Mats Löfdahl
Messages: 263
Registered: January 2012

Senior Member

Den tisdag 1 mars 2016 kl. 07:12:13 UTC+1 skrev Sapna Mishra:
> Hi Heinz Stege ,
> Error is as following:
>
> IDL> cmd ='awk -v var="$i" ''{print $var}'' ' + fname[i] + ' > ' + fnameo[i]
> % Variable is undefined: I.
> % Execution halted at: $MAIN$
> IDL> print,'awk -v var="$i" ''{print $var}'' ' + fname[i] + ' > ' + fnameo[i]
> % Variable is undefined: I.
> % Execution halted at: $MAIN$
> IDL>

I think you need to do that inside the loop to get a useful error message. Or just set i to something between 0 and c.

Report message to a moderator

Re: TRICKY TASK USING AWK [message #92797 is a reply to message #92796]

Tue, 01 March 2016 01:50

Mats Löfdahl is currently offline

Mats Löfdahl
Messages: 263
Registered: January 2012

Senior Member

Den tisdag 1 mars 2016 kl. 10:37:36 UTC+1 skrev Mats Löfdahl:
> Den tisdag 1 mars 2016 kl. 07:12:13 UTC+1 skrev Sapna Mishra:
>> Hi Heinz Stege ,
>> Error is as following:
>>
>> IDL> cmd ='awk -v var="$i" ''{print $var}'' ' + fname[i] + ' > ' + fnameo[i]
>> % Variable is undefined: I.
>> % Execution halted at: $MAIN$
>> IDL> print,'awk -v var="$i" ''{print $var}'' ' + fname[i] + ' > ' + fnameo[i]
>> % Variable is undefined: I.
>> % Execution halted at: $MAIN$
>> IDL>
>
> I think you need to do that inside the loop to get a useful error message. Or just set i to something between 0 and c.

Just guessing here, since we don't have the actual error message where IDL complains about the non-existence of fnameo[i]: is fnameo an array with c elements? In that case, your loop should go from 0 to c-1 rather than from 1 to c.

Or, since the complaint was only about fnameo and not fname, and you set fnameo[i] to 'tmp' inside the loop: is perhaps only fname an array while fnameo is undefined before you enter the loop? In that case, just substitute "fnameo" for "fnamao[i]" in the loop. Twice, of course.

Report message to a moderator

Re: TRICKY TASK USING AWK [message #92798 is a reply to message #92794]

Tue, 01 March 2016 07:39

wlandsman is currently offline

wlandsman
Messages: 743
Registered: June 2000

Senior Member

On Tuesday, March 1, 2016 at 1:08:03 AM UTC-5, Sapna Mishra wrote:
> Hi wlandsman ,
> Is this code reads the contents of columns or it returns the no. of columns to be read???

Why don't you try it and find out yourself? (It does the same thing as your spawn to "awk".)

> If it is the case then my problem remains unsolved because i want to READ columns from a file without knowing how much column does this file has. Say If I have infinite number of columns And I want to read 100th column how would I do that, shall I need to give 100 parameters in Readcol?? This seems tough to me.

If your data is all numeric then the easiest thing is to use READ_ASCII

output = read_ascii('myfile.dat')

the output array will contain all the data. Then just select the column of the array you want.

Report message to a moderator

Re: TRICKY TASK USING AWK [message #92801 is a reply to message #92794]

Tue, 01 March 2016 19:36

wlandsman is currently offline

wlandsman
Messages: 743
Registered: June 2000

Senior Member

READCOL does not care how many columns are in a file. Say a file "bigtable.dat' has 100 columns and we just want to read the 5th column. This would work.

IDL> readcol,'bigtable.dat',v5,f='x,x,x,x,f'

Unfortunately, one can't abbreviate the format as f = '4x,f' though this feature wouldn't be too hard to add.

On Tuesday, March 1, 2016 at 1:08:03 AM UTC-5, Sapna Mishra wrote:
Say If I have infinite number of columns And I want to read 100th column how would I do that, shall I need to give 100 parameters in Readcol?? This seems tough to me.

Report message to a moderator

Switch to threaded view of this topic

Create a new topic

Submit Reply

Previous Topic:	Looping through structures
Next Topic:	Segmentation fault (core dumped)

-=] Back to Top [=-

[ Syndicate this forum (XML) ] [

] [

PDF

]

Current Time: Wed Oct 08 09:17:18 PDT 2025

Total time taken to generate the page: 0.00520 seconds