Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Sign In with Facebook Sign In with Google Sign In with OpenID

Categories

We have migrated to a new platform! Please note that you will need to reset your password to log in (your credentials are still in-tact though). Please contact lee@programmersheaven.com if you have questions.
Welcome to the new platform of Programmer's Heaven! We apologize for the inconvenience caused, if you visited us from a broken link of the previous version. The main reason to move to a new platform is to provide more effective and collaborative experience to you all. Please feel free to experience the new platform and use its exciting features. Contact us for any issue that you need to get clarified. We are more than happy to help you.

FTP File streaming

chaosownschaosowns Posts: 1Member
Hello,

I have been asked to try to program a ftp file stream in python. Here's the complete story:

There's a FTP server with a comma seperated file (csv) on it. A device writes log information in to the csv file whenever something log-worthy occures. This csv file is around 22 MB and keeps on growing. I want to read the latest csv entry that's gotten in to the file. So my question is:

Is it possible to connect to the FTP server, open up a stream to the csv file and retrieve the latest data? I have to able to get a constant stream and always get the latest csv entry.

If this is possible wont there be a conflict when the device is writing to the .csv file and i'm reading from it?

If this isn't possible in python, do you have any idea if it is in another programming language?

Hope i explained everything well enough.

Thanks in advance.

Comments

  • iblisiblis Posts: 3Member
    : Hello,
    :
    : I have been asked to try to program a ftp file stream in python.
    : Here's the complete story:
    :
    : There's a FTP server with a comma seperated file (csv) on it. A
    : device writes log information in to the csv file whenever something
    : log-worthy occures. This csv file is around 22 MB and keeps on
    : growing. I want to read the latest csv entry that's gotten in to the
    : file. So my question is:
    :
    : Is it possible to connect to the FTP server, open up a stream to the
    : csv file and retrieve the latest data? I have to able to get a
    : constant stream and always get the latest csv entry.
    :
    : If this is possible wont there be a conflict when the device is
    : writing to the .csv file and i'm reading from it?
    :
    : If this isn't possible in python, do you have any idea if it is in
    : another programming language?
    :
    : Hope i explained everything well enough.
    :
    : Thanks in advance.
    :

    Should be possible. Quick question, though: Do you, or perhaps the person asking for this program, have control over the process that creates this csv?

    If so, you and/or they could always change the process to create an empty filename.csv.lock file while it's writing, and remove it when it's finished; then code your client to respect it (i.e., not download the file) if it finds it.

    I don't have any example code immediately available for parsing csv's, though there's a nice csv module including with python called, amazingly enough, "csv". :D

    On most systems a 'pydoc csv' will give you oodles of information on the module, or you can just do [code]import csv;help(csv)[/code] from an interpreter.

    But I do use something like this to pull down data over my lan - since I have gigabit and the files aren't big I don't worry about pulling down the whole thing each time, but if that's not the case for you you'll likely want to look at [link=http://www.faqts.com/knowledge_base/view.phtml/aid/4360]resuming[/link] to save bandwidth.

    [code]
    from ftplib import FTP
    import sys

    ftp = FTP('hostname')
    ftp.login('username', 'password')

    filename = 'filename.dat'

    ls = ftp.retrlines('LIST %s' % filename).split('
    ')[-1]

    if ls.startswith('226 0'):
    sys.exit(1)

    ls = ftp.retrlines('LIST %s.lock' % filename).split('
    ')[-1]

    if not ls.startswith('226 0'):
    ftp.quit()
    sys.exit(1)
    else:
    ftp.retrbinary('RETR %s' % filename, open(filename , 'wb').write)
    ftp.quit()
    sys.exit(0)

    [/code]

    Hope this helps.
Sign In or Register to comment.