Speed of Text Reader - Programmers Heaven

Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Categories

Welcome to the new platform of Programmer's Heaven! We apologize for the inconvenience caused, if you visited us from a broken link of the previous version. The main reason to move to a new platform is to provide more effective and collaborative experience to you all. Please feel free to experience the new platform and use its exciting features. Contact us for any issue that you need to get clarified. We are more than happy to help you.

Speed of Text Reader

mjsmikeymjsmikey Posts: 1Member
I have been trying to speed up my process of reading a text document in my RichTextBox1 (Document.Text) and preforming some tasks on it then dumping the text into RichTextBox2.

I have tried a few different ways to make the 26,000 line documents that I have move a little faster, Currently one document takes about 1.15 hours to complete. It does the job... but I cannot figure out what it is taking so long.

I think there is a way to read text from Document.Text and Store it into Permanent/Temp Memory so that It doesn't have to read line one then line one again to get to Line two, and then line one and two again to get to line three.

The amount of time to process each file gets extreamly longer and longer depending on the amount of Lines of Text that can fill the memory. When I get to line 1,000 or so the system slows way down. Any textfile smaller then this runs very fast and very efficiently. Unfortunatly I don't have files this small to process.

The Code (below) will search for a string of text in Document.Text and Copy that line and a certain number of lines before it and after it. It will store these to another file durring the process.

I belive that using the Find Feature (don't know how), I could locate the String and note the Line numbers in a list box. Then I could Write only the lines required saving me a ton of time in the Line by Line search.

Or I could just dump the buffer (don't know how), to reduce the amount of data in the system so that it can process more efficiently and forget about the old text.
----------------------------------------
RichTextBox1.Text = String.Empty
Dim StreamReaderIO As IO.StreamReader
MsgBox("THIS MAY TAKE SOME TIME - MAY EVEN SHOW NOT RESPONDING - JUST WAIT - YOU CAN CONTINUE TO WORK WHILE THIS IS RUNNING", vbOKCancel)
RichTextBox1.Text &= "(THIS DOCUMENT HAS BEEN MODIFIED WITH STRIP-N-SAVE)" & vbNewLine
'Correcting Blank Values on Form
If TextBox5.Text = "" Then
TextBox5.Text = "75"
End If
If TextBox4.Text = "" Then
TextBox4.Text = "20"
End If
If TextBox3.Text = "" Then
TextBox3.Text = "20"
End If
If TextBox1.Text = "" Then
TextBox1.Text = "M6"
End If
If TextBox2.Text = "" Then
TextBox2.Text = TextBox1.Text
End If

'Variables
Dim endLine = CInt(TextBox5.Text)
Dim LineCount As Integer = Document.Lines.Length
Dim randarray(0 To LineCount) As Long
ProgressBar1.Maximum = UBound(randarray)
Dim StartLine = CInt(TextBox3.Text)
Dim Ending = CInt(TextBox4.Text)
Dim J, L, kk As Integer
'Correcting Input if Document too Short
If LineCount < endLine - 1 Then
endLine = LineCount
End If
'Copying First Lines At all times
For x As Integer = 0 To endLine - 1
RichTextBox1.Text &= Document.Lines(x) & vbNewLine
J = x
Next
'Search For Text and Copy if Exists
For x As Integer = endLine To LineCount - 1
If Document.Lines(x).Contains(TextBox1.Text) Or Document.Lines(x).Contains(TextBox2.Text) Then
If LineCount <= x + Ending Then
Ending = 10
End If
If LineCount <= x + Ending Then
Ending = 5
End If
If LineCount <= x + Ending Then
Ending = 1
End If
If LineCount <= x + Ending Then
Ending = 0
End If
If LineCount <= x + Ending Then
Exit For
End If
L = x - StartLine
kk = L
If kk <= J + StartLine Then
L = J + 1
End If
For k As Integer = L To x + Ending
RichTextBox1.Text &= Document.Lines(k) & vbNewLine
If LineCount <= k Then
Exit For
End If
Next
x = x + Ending
J = x
ProgressBar1.Value = x
'Stop if Document is Too Short
If x >= LineCount - 1 Then
Exit For
End If
Else
If x >= LineCount - 1 Then
Exit For
End If
End If
'Stop if Document is Too Short
If x >= LineCount - 1 Then
Exit For
End If
Next
ProgressBar1.Value = LineCount
Try
My.Computer.Audio.Play("C:WINDOWSMedia
otify.wav")
Catch ex As Exception
End Try
MsgBox("Document Complete - Press Save", vbOKCancel)
-----------------------

Comments

  • bardillbardill Posts: 2Member
    Hi
    When you will scan in a Text File for some Words i do this with The logparser 2.2 from Microsoft
    For search 100 different Names like (Hans,Peter etz) for a file with 135MB Text you need 1 Minute

    You can very easy search with a select statement

    this small example search word by word and line by line for Hans and Peter

    You can experimet wit the strquery

    Downoad Logparser
    http://www.microsoft.com/downloads/details.aspx?FamilyID=890cd06b-abf8-4c25-91b2-f8d975cf8c07&displaylang=en




    Imports MSUtil

    Module Module1



    Sub Main()
    Dim strQuery As String
    Dim iisLogWord As New COMTextWordInputContextClass
    Dim iisLogLine As New COMTextLineInputContextClass

    'search for all words like Peter and Hans
    strQuery = "Select Text from mytest.txt where text in ('Peter';'Hans')"
    parseLogWord(strQuery)

    strQuery = "Select Text from mytest.txt where text like '%Peter%' or text like '%Hans%'"
    parseLogLine(strQuery)
    Console.ReadLine()
    End Sub

    Public Function parseLogWord(ByVal strQuery As String)
    Dim parser As New LogQueryClass
    Dim iisLog As New COMTextWordInputContextClass
    Dim rsLp As ILogRecordset
    Dim rowLp As ILogRecord

    'Word input format
    rsLp = parser.Execute(strQuery, iisLog)


    While Not rsLp.atEnd
    rowLp = rsLp.getRecord

    Console.WriteLine(rowLp.getValue(0))

    rsLp.moveNext()
    End While

    End Function
    Public Function parseLogLine(ByVal strQuery As String)
    Dim parser As New LogQueryClass
    Dim iisLog As New COMTextLineInputContextClass
    Dim rsLp As ILogRecordset
    Dim rowLp As ILogRecord

    'Word input format
    rsLp = parser.Execute(strQuery, iisLog)


    While Not rsLp.atEnd
    rowLp = rsLp.getRecord

    Console.WriteLine(rowLp.getValue(0))

    rsLp.moveNext()
    End While

    End Function
    End Module


Sign In or Register to comment.