Just wanted to say many thanks. Nice program, works fine, saves me a lot of time!

2. Generate both Local time and GM time and compare the file’s time with GM time.

3. If the file’s timestamp is lower than the GM time (meaning 3 days older), then we move it to a new directory from the list and pick another file. Else, process it as an command line argument to program (/root/home/source/programs/abc.c)

4. Parallelly, append that file, which is processed from the directory and store it in a log file

I wrote the following script:

#!/bin/sh -x
checkduration=259200 # 3 days in seconds
cur_timestamp=`date +%s` #current time in seconds

cd /mnt/ldm/data/public/RAOB/200801/ #  change dir
for i in /mnt/ldm/data/public/RAOB/200801/*; do
 if [ -f $i ]; then   # if the file is there
  filename=${i#/mnt/ldm/data/public/RAOB/200801/};# get file name
  last_mod=$(stat -c "%Y" ${i#/mnt/ldm/data/public/RAOB/200801/});
  # get last mod time of file in seconds last_mod=$(stat -c "%Y" filename)
  #above line gathers file modified time in seconds from epoch
 if [ $last_mod ]
  then
   time_diff=`expr $cur_timestamp - $last_mod`; #cal time difference 3 days or more
   if [ $time_diff -lt $checkduration] # file modified three days or less back
         then
    mv ${i#/mnt/ldm/data/public/RAOB/200801/} /mnt/ldm/data/public/RAOB/old_data     #if more then move to a new folder
    cat $filename>>log #append this file name to log sheet
     fi
  fi
 fi
done

I do not get any errors, but there seem to be a problem with the comparison of times and to move the file read to another directory.

Could you help me please?

rgds,
Kalyan

I put both scripts in my ~/bin dir, and then have this in my .bash_profile:

# set PATH so it includes user's private bin if it exists
if [[ -d ~/bin && ! $PATH =~ ~/bin ]] ; then
    export PATH=~/bin:"${PATH}"
fi

If you use something like that, you won’t have to hard code the path to copy_it.sh, and you can have all your scripts readily available.

(It might be helpful also to rewrite it with the advice from Mario above so that it is only a single script, but since it’s working this way I haven’t had that much motivation to do so.)

My bash version is 3.2.13(1)-release. I just searched and found http://tldp.org/LDP/abs/html/bashver3.html: “The version 3.1 update of Bash introduces a number of bugfixes and a few minor changes. The += operator is now permitted in places where previously only the = assignment operator was recognized.” So that might explain this situation.

I had an error

find: copy_it.sh: No such file or directory

I found the problem on cpafter.sh line 101-108. After setting cd to the /source/ directory. copy_it.sh is called however it’s not on the /source/ directory. I rewrote your script hard linking this to my local root location.

find . -type f -daystart -mtime $date_dif -exec bash ~/copy_it.sh $verbose -s {} -t $to_dir ;
find . -type l -daystart -mtime $date_dif -exec bash ~/copy_it.sh $verbose -s {} -t $to_dir ;

On a tangent, I’m new to bash. And was wondering why the append command += doesn’t work in my version.

e.g.

pluto ~ # bash cpafter.sh -v -a 20071120 -s "/source/" -t "/target/"
cpafter.sh: line 24: usage+=to target dir, creating any subdirectories as needed.nn: command not found
cpafter.sh: line 25: usage+=tusage: cpafter.sh [-vf] -a after_date_YYYYMMDD -s source_dir -t target_dirn: command not found
cpafter.sh: line 26: usage+=tt-v verbosen: command not found
cpafter.sh: line 27: usage+=tt-f force target dir creation or copying to non-empty target dirn: command not found

copying from /source
        to /target/root/copy_it.sh: line 24: usage+=build a path from the given dir root if necessary. Meant to be usedn: command not found
/root/copy_it.sh: line 25: usage+=with cpafter.sh.nn: command not found
/root/copy_it.sh: line 26: usage+=tusage: copy_it.sh [-v] -s source_file -t target_directory_rootn: command not found
/root/copy_it.sh: line 27: usage+=tt-v verbose (just lists the source file)n: command not found
./my_test_file

My version: GNU bash, version 3.00.16(1)-release (i686-pc-linux-gnu)

Drew

You’ll want to look at rsync. It copies files that have changed and that are new from one place to another. To do what you want, you’d do something like this from machine B:


rsync -a A:/mydir /foo


This will copy all the new files from A:/mydir to /foo on B. rsync requires rsh or ssh to copy over a network, or you can set up the rsync protocol.

I gave an explanation below:

Script scenario:
I have two machines “A”, “B”.

for every 5mnts some files are written into directory “DIR1″ on
Machine “A” with naming convention
and the count
increases in day. Now I have to run the script on machine “B” to
get the files from machine “A”. Here I have to read the files in
FIFO manner ( as they reach DIR1 on “A”) and each files resides
in DIR1 on “A” upto 7days. So I have to ensure that each file is
copied only once from DIR1 on “A” to macine “B”, though we run
the script several times. I mean, files which are copied once
from “A” to “B” to be ignored in future. I remind that we can’t
remove the files after reading from A:/DIR1.

For that I developed the below script. But this is simply
copying all files from Machine A-to-Machine B.

#!/bin/ksh

clear

echo “Start of the script ”
HOST=hostname
USER=user1
PASSWD=userpw1
scrdir=/users/user1/TEST
ftp -n $HOST