Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Categories

Welcome to the new platform of Programmer's Heaven! We apologize for the inconvenience caused, if you visited us from a broken link of the previous version. The main reason to move to a new platform is to provide more effective and collaborative experience to you all. Please feel free to experience the new platform and use its exciting features. Contact us for any issue that you need to get clarified. We are more than happy to help you.

winAPI programming-How two threads process data from in parallel?

How two thread process data from same input file in parallel?
Can anyone enlighten how to begin with? There is one textfile, that contains lines of phrases. One thread counts the number of line, and the other thread counts the number or words.

Comments

  • AsmGuru62AsmGuru62 Posts: 6,519Member
    [color=Blue]That depends on how the algorithm works. I see a few ways to do it:

    CASE A)
    1. Open file and load ALL text into a buffer
    2. Create threads for counting and pass buffer pointer to each thread

    This case does not require a synchronization, because the data in buffer only being read and not written to. However, it will fail for very large files (few Gb), because there will be not enough memory to load the whole file.

    CASE B)
    1. Open file
    2. Create both threads in suspended state
    3. In a loop do the following:
    - load a part of file
    - resume both threads from suspended state
    - wait for both threads to finish processing data using events,
    do not ends threads themselves - just the data processing loops
    - when both threads process all their data - suspend them again
    and return back to beginning of the loop (load next piece
    of data, etc.)

    In this case you can process file of ANY size, because data being loaded by pieces. You also have to synchronize in a way that BOTH threads are suspended until buffer is being loaded. The buffer is a common data for both threads.

    CASE C)
    1. Simply create both threads and each thread has its own buffer loading code (you can make it into a function) and data counting loop.

    In this case you may need to synchronize the file loading. I am not sure if OS allows to read same file on both threads - you can try, but to be safe I would suggest to synchronize it. Meaning, that if one thread loads file - 2nd waits for 1st to finish it. Also, if each thread has its own data buffer - it will increase memory consumption for the code, but Windows has a lot of memory - you can allocate 4Mb for each thread easily and it will not even be noticeable.

    For some information go here:

    [link=http://msdn.microsoft.com/en-us/library/aa383686(v=vs.85).aspx]http://msdn.microsoft.com/en-us/library/aa383686(v=vs.85).aspx[/link]

    Then go to [italic][color=Green]Synchronization[/color][/italic] and then go to [italic][color=Green]Critical-section function[/color][/italic] table. You need to use critical section to make one thread wait until other doing its work on a shared resource. You also can use events and wait for the events to happen, but critical sections are the faster way.
    [/color]
  • candyTreecandyTree Posts: 2Member
    hi,

    can you please give a clearer example? i can't still get it. i am new to OS. infact i have just started studying about the subject.
    Questions:
    1. What should we use to open file? Is it CreateFile() or the normal FILE *fp, fp= fopen("text.txt", "r") from C; where the file should be opened? is it in the main function?

    i did write my own code but still it fails to give the expected result.
    in my main function, i created two threads. i created also two functions
    DWORD WINAPI CountWordsThread() and DWORD WINAPI CountLinesThread().
    In both functions i opened the input file.

    thanks
  • AsmGuru62AsmGuru62 Posts: 6,519Member
    [color=Blue]So, you have some code written? Very well.
    Can you post the code here?
    Or if you wish, you can send me the code you wrote and I'll help.
    I promise not to share the code.

    Zip the code and send it to me:
    asmguru62@hotmail.com

    You can open file with both of these functions. fopen() will simply call CreateFile() somewhere inside the C Run-Time Library code.

    Also, tell me what option did you choose to do the counting?
    Do you have one buffer for both threads or each thread has its own buffer?
    [/color]
Sign In or Register to comment.