how to read and write file faster? - Programmers Heaven

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Categories

Welcome to the new platform of Programmer's Heaven! We apologize for the inconvenience caused, if you visited us from a broken link of the previous version. The main reason to move to a new platform is to provide more effective and collaborative experience to you all. Please feel free to experience the new platform and use its exciting features. Contact us for any issue that you need to get clarified. We are more than happy to help you.

how to read and write file faster?

weixj2003ldweixj2003ld Posts: 11Member
how to read and write file on the disk faster?

Comments

  • melanie14melanie14 Posts: 4Member
    This post has been deleted.
  • weixj2003ldweixj2003ld Posts: 11Member
    I want to use multi-thread,but I have some questions.
    1. If read one large file(500G) with multi-thread,can it accelerate the speed?
    2.If read many files and each thread read one,can it accelerate the speed?
    3.If the multi-thread program would accelerate the speed,Must it run on multi-CPU? what about on multi-kernel?
  • SundayForeverSundayForever Posts: 4Member
    This post has been deleted.
  • nebgastnebgast Posts: 54Member
    : I want to use multi-thread,but I have some questions.
    : 1. If read one large file(500G) with multi-thread,can it accelerate
    : the speed?

    Assuming that you are working with a multi-core CPU and lock the critical location of what is being read, but if you are reading one file in a concurrent manner are you really able to process the file? Unless the file is created to have X size bytes read concurrently and processed reading a single file on multiple threads might be pointless.

    : 2.If read many files and each thread read one,can it accelerate the
    : speed?

    Yes, reading one file per thread could speed up the processing of the files. Given either a single-core or multi-core threading system.

    : 3.If the multi-thread program would accelerate the speed,Must it run
    : on multi-CPU? what about on multi-kernel?

    Multi-threading on a single core uses the concept of time-slicing to fake multiple-threads, so it can potentially speed up the program depends on what's going on.

    Multi-kernal - I don't know how to answer that question.
  • AsmGuru62AsmGuru62 Posts: 6,519Member
    [color=Blue]When dealing with a single hard drive - multithreading may be slower. No matter how many CPU or cores you have - a single physical drive may cause excessive wear on the drive and slow processing.

    Consider this: two threads read two files, separated physically by some amount of sectors. The hard drive heads must move to the 1st file and read the part of it, then next thread comes and heads move to 2nd file sectors, then process repeats - probably a lot of times - depends on a size of the part of the file being loaded. Since there is only one set of drive heads and it cannot be really shared at run-time (like CPU cores) - some thread always will wait until other thread is moving these heads.[/color]
  • alinora11alinora11 Posts: 1Member
    This post has been deleted.
  • weixj2003ldweixj2003ld Posts: 11Member
  • fuatgencfuatgenc Posts: 17Member
    This post has been deleted.
Sign In or Register to comment.