Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Categories

Welcome to the new platform of Programmer's Heaven! We apologize for the inconvenience caused, if you visited us from a broken link of the previous version. The main reason to move to a new platform is to provide more effective and collaborative experience to you all. Please feel free to experience the new platform and use its exciting features. Contact us for any issue that you need to get clarified. We are more than happy to help you.

Concatenating files in a directory

marcelafinkmarcelafink Posts: 1Member
I am looking to find an eloquent way to concatenate all the files in a specific directory, basically making them into one. I created a script that loops through and appends everything to the new file name by way of cat $pathname/$i >> $pathname/$newfile.txt. As each item gets appended, I remove it. The $newfile.txt is actually in a different directory.

Depending on the UNIX/LINUS OS, this works great or does not work correctly. I am actually building a rather large file that will eventually get FTP'd to a website. In some cases, the first line gets chopped off when the concatenation is occurring. To fix this a appended a blank line at the end of each item.

On one particular box, I see the following messages in the jobstack log, my routine runs early in the morning:

cat: cannot open $pathname/$i
rm: $pathname/$i: No such file or directory

The $pathname is actually the fullpath and $i is something as follows:

06192003_c999999_100_1_vehicles_00001.txt

Each piece of information in this id has substantial relevance but the bottom line is that it is vehicle information. The ultimate filename is 06192003_c999999_100_1_vehicles.txt. Sometimes, the name gets chopped up as well where instead of vehicles, we may have veh.

If anyone has any suggestions or comments, please do not hesitate. The fact that it works well in one UNIX implemenation and not in others is frustrating.

Thank you in advance.

Comments

  • FirestormFirestorm Posts: 162Member
    : I am looking to find an eloquent way to concatenate all the files in a specific directory, basically making them into one. I created a script that loops through and appends everything to the new file name by way of cat $pathname/$i >> $pathname/$newfile.txt. As each item gets appended, I remove it. The $newfile.txt is actually in a different directory.
    :
    : Depending on the UNIX/LINUS OS, this works great or does not work correctly. I am actually building a rather large file that will eventually get FTP'd to a website. In some cases, the first line gets chopped off when the concatenation is occurring. To fix this a appended a blank line at the end of each item.
    :
    : On one particular box, I see the following messages in the jobstack log, my routine runs early in the morning:
    :
    : cat: cannot open $pathname/$i
    : rm: $pathname/$i: No such file or directory
    :
    : The $pathname is actually the fullpath and $i is something as follows:
    :
    : 06192003_c999999_100_1_vehicles_00001.txt
    :
    : Each piece of information in this id has substantial relevance but the bottom line is that it is vehicle information. The ultimate filename is 06192003_c999999_100_1_vehicles.txt. Sometimes, the name gets chopped up as well where instead of vehicles, we may have veh.
    :
    : If anyone has any suggestions or comments, please do not hesitate. The fact that it works well in one UNIX implemenation and not in others is frustrating.
    :
    : Thank you in advance.
    :
    :

    sorry I didn't quite understand whether you want to put all your files from one directory into one archive or if you want to write all filenames into a plain/text file.....
    ________________________________
    [red]http://www.againsttcpa.com[/red]
    [blue]Don't give big brother a chance![/blue]

  • BASIC FriendBASIC Friend Posts: 354Member
    : : I am looking to find an eloquent way to concatenate all the files in a specific directory, basically making them into one. I created a script that loops through and appends everything to the new file name by way of cat $pathname/$i >> $pathname/$newfile.txt. As each item gets appended, I remove it. The $newfile.txt is actually in a different directory.
    : :
    : : Depending on the UNIX/LINUS OS, this works great or does not work correctly. I am actually building a rather large file that will eventually get FTP'd to a website. In some cases, the first line gets chopped off when the concatenation is occurring. To fix this a appended a blank line at the end of each item.
    : :
    : : On one particular box, I see the following messages in the jobstack log, my routine runs early in the morning:
    : :
    : : cat: cannot open $pathname/$i
    : : rm: $pathname/$i: No such file or directory
    : :
    : : The $pathname is actually the fullpath and $i is something as follows:
    : :
    : : 06192003_c999999_100_1_vehicles_00001.txt
    : :
    : : Each piece of information in this id has substantial relevance but the bottom line is that it is vehicle information. The ultimate filename is 06192003_c999999_100_1_vehicles.txt. Sometimes, the name gets chopped up as well where instead of vehicles, we may have veh.
    : :
    : : If anyone has any suggestions or comments, please do not hesitate. The fact that it works well in one UNIX implemenation and not in others is frustrating.
    : :
    : : Thank you in advance.
    : :
    : :
    :
    : sorry I didn't quite understand whether you want to put all your files from one directory into one archive or if you want to write all filenames into a plain/text file.....
    : ________________________________
    : [red]http://www.againsttcpa.com[/red]
    : [blue]Don't give big brother a chance![/blue]
    :
    :
    I don't know if this is what you want, but I have used split many mnay times to slit up a large file into floppy sized bits. Once I have all the files transfer, I then concatenate them by one simple command cat file.* >> file.tar.gz or simialar. I can remove the old files easily too. If your filenames are in order alphabetically then cat * >> newfile will work. Another way would to be in the parent directory and do this: cat ./folder/* >> ./file && rm -r ./folder
    no script neccasary.

Sign In or Register to comment.