comp.lang.idl-pvwave archive - RDF feed
type: channel
description: Messages from Usenet group comp.lang.idl-pvwave, compiled by Paulo Penteado
items: Processing about 10,000 tif files
items: Re: Processing about 10,000 tif files

Processing about 10,000 tif files
type: item
description: Dear Gurus, I have RGB images in tif format and want to do simple arithmetic calculations. A big obstacle to me is number of files to be processed, about 10,000 images. Name of the image files has sequential id such as img_1.tif, img_2.tif, ....., img_10000.tif. And I have a text file containing image names, coefficients, and offsets. I think reading the text file as an array with three columns and lines matching number of files seems to be successful. I've tried to read image files with dialog_pickfile, but I've learned that the function limits byte length and can't seem to go beyond more than 2,000 files or so. And I've tried to read image files with file_search, but it didn't do calculation in sequential order. Would you give some suggestion how to read image files in order, so that following calculation should be in order from 1 to 10,000? Or a way to do calculation with image name corresponding lines in the text file? Thanks in advance, Kim
creator: livingwater000
date: 2015-05-19T21:07:38-00:00

Re: Processing about 10,000 tif files
type: item
description: On Tuesday, May 19, 2015 at 11:07:39 PM UTC+2, livingw...@gmail.com wrote: > Dear Gurus, > > I have RGB images in tif format and want to do simple arithmetic calculations. > A big obstacle to me is number of files to be processed, about 10,000 images. > Name of the image files has sequential id such as img_1.tif, img_2.tif, ....., img_10000.tif. > And I have a text file containing image names, coefficients, and offsets. > I think reading the text file as an array with three columns and lines matching number of files seems to be successful. > > I've tried to read image files with dialog_pickfile, but I've learned that the function limits byte length and can't seem to go beyond more than 2,000 files or so. > And I've tried to read image files with file_search, but it didn't do calculation in sequential order. > > Would you give some suggestion how to read image files in order, so that following calculation should be in order from 1 to 10,000? > Or a way to do calculation with image name corresponding lines in the text file? > > Thanks in advance, > Kim I've been there before and I see two ways out of this: 1) construct the file array with something like 'img_'+strtrim(indgen(10000)+1,2)+'.tif' 2) use file_search and then sort the file array to have the order you wish 3) read the text file sequentially and process sequentially Option 2) is not easy because the files don't have a format like img_00001.tif, otherwise sorting would be easier. Hope it helps. Helder
creator: Helder Marchetto
date: 2015-05-19T22:03:21-00:00

Re: Processing about 10,000 tif files
type: item
description: On Wednesday, May 20, 2015 at 12:03:23 AM UTC+2, Helder wrote: > On Tuesday, May 19, 2015 at 11:07:39 PM UTC+2, livingw...@gmail.com wrote: >> Dear Gurus, >> >> I have RGB images in tif format and want to do simple arithmetic calculations. >> A big obstacle to me is number of files to be processed, about 10,000 images. >> Name of the image files has sequential id such as img_1.tif, img_2.tif, ....., img_10000.tif. >> And I have a text file containing image names, coefficients, and offsets. >> I think reading the text file as an array with three columns and lines matching number of files seems to be successful. >> >> I've tried to read image files with dialog_pickfile, but I've learned that the function limits byte length and can't seem to go beyond more than 2,000 files or so. >> And I've tried to read image files with file_search, but it didn't do calculation in sequential order. >> >> Would you give some suggestion how to read image files in order, so that following calculation should be in order from 1 to 10,000? >> Or a way to do calculation with image name corresponding lines in the text file? >> >> Thanks in advance, >> Kim > > I've been there before and I see two ways out of this: > 1) construct the file array with something like 'img_'+strtrim(indgen(10000)+1,2)+'.tif' > 2) use file_search and then sort the file array to have the order you wish > 3) read the text file sequentially and process sequentially > > Option 2) is not easy because the files don't have a format like img_00001.tif, otherwise sorting would be easier. > > Hope it helps. > Helder Brilliant... two ways out in three points. Don't drink and type. Helder
creator: Helder Marchetto
date: 2015-05-19T22:12:52-00:00

Re: Processing about 10,000 tif files
type: item
description: On Tuesday, May 19, 2015 at 6:12:53 PM UTC-4, Helder wrote: > On Wednesday, May 20, 2015 at 12:03:23 AM UTC+2, Helder wrote: >> On Tuesday, May 19, 2015 at 11:07:39 PM UTC+2, livingw...@gmail.com wrote: >>> Dear Gurus, >>> >>> I have RGB images in tif format and want to do simple arithmetic calculations. >>> A big obstacle to me is number of files to be processed, about 10,000 images. >>> Name of the image files has sequential id such as img_1.tif, img_2.tif, ....., img_10000.tif. >>> And I have a text file containing image names, coefficients, and offsets. >>> I think reading the text file as an array with three columns and lines matching number of files seems to be successful. >>> >>> I've tried to read image files with dialog_pickfile, but I've learned that the function limits byte length and can't seem to go beyond more than 2,000 files or so. >>> And I've tried to read image files with file_search, but it didn't do calculation in sequential order. >>> >>> Would you give some suggestion how to read image files in order, so that following calculation should be in order from 1 to 10,000? >>> Or a way to do calculation with image name corresponding lines in the text file? >>> >>> Thanks in advance, >>> Kim >> >> I've been there before and I see two ways out of this: >> 1) construct the file array with something like 'img_'+strtrim(indgen(10000)+1,2)+'.tif' >> 2) use file_search and then sort the file array to have the order you wish >> 3) read the text file sequentially and process sequentially >> >> Option 2) is not easy because the files don't have a format like img_00001.tif, otherwise sorting would be easier. >> >> Hope it helps. >> Helder > > Brilliant... two ways out in three points. Don't drink and type. > Helder Thanks much. will try it shortly and update. Kim
creator: livingwater000
date: 2015-05-20T01:01:43-00:00

Re: Processing about 10,000 tif files
type: item
description: On Wednesday, 20 May 2015 06:37:39 UTC+9:30, livingw...@gmail.com wrote: > Dear Gurus, > > I have RGB images in tif format and want to do simple arithmetic calculations. > A big obstacle to me is number of files to be processed, about 10,000 images. > Name of the image files has sequential id such as img_1.tif, img_2.tif, ....., img_10000.tif. > And I have a text file containing image names, coefficients, and offsets. > I think reading the text file as an array with three columns and lines matching number of files seems to be successful. > > I've tried to read image files with dialog_pickfile, but I've learned that the function limits byte length and can't seem to go beyond more than 2,000 files or so. > And I've tried to read image files with file_search, but it didn't do calculation in sequential order. > > Would you give some suggestion how to read image files in order, so that following calculation should be in order from 1 to 10,000? > Or a way to do calculation with image name corresponding lines in the text file? > > Thanks in advance, > Kim How about just rename the files img_00001.tif to img_09999.tif? Solve many problems in a very few lines of code. And while you're at it, insert some useful coefficients and offsets ino the filenames too. Andrew
creator: andrewcool777
date: 2015-05-25T01:23:11-00:00

Re: Processing about 10,000 tif files
type: item
description: On Sunday, May 24, 2015 at 8:23:14 PM UTC-5, andrew...@gmail.com wrote: > > How about just rename the files img_00001.tif to img_09999.tif? Solve many problems in a very few lines of code. > > And while you're at it, insert some useful coefficients and offsets ino the filenames too. > > Andrew Code below will rename all files so that they can be found in order: f=file_search('img_*.tif', count=n) max=20000 ;<--- this should be larger than your max nzeros = ceil(alog10(max)) > 0 myformat = '(i0'+strcompress(nzeros, /re)+')' for i=0L,n-1 do print, f[i],' -> ', 'img_'+string((strsplit(file_basename(f[i], '.tif'), '_', /ex))[1], format = myformat)+'.tif' If you like the above, run it like so: for i=0L,n-1 do file_move, f[i], 'img_'+string((strsplit(file_basename(f[i], '.tif'), '_', /ex))[1], format = myformat)+'.tif' G
creator: cgguido
date: 2015-05-27T02:05:12-00:00