Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Categories

Welcome to the new platform of Programmer's Heaven! We apologize for the inconvenience caused, if you visited us from a broken link of the previous version. The main reason to move to a new platform is to provide more effective and collaborative experience to you all. Please feel free to experience the new platform and use its exciting features. Contact us for any issue that you need to get clarified. We are more than happy to help you.

Dealing BIG texts

ampoampo Posts: 63Member
Hello.
I have large text (can be 60,000 chars or more) in a string and I have TEXT type field in my table.
I need to see if the text in the string is not already in the DB.

What is the best way to do this ?

What I do now is to take 10 pieces of the string:
from char 2 to 102, from char 350 to 450, from char 460 to 560 and so on, and connecting 10 SQL's to check that field.

The problem is that it takes 7 seconds...

Can I do it better ?

Thanks

Comments

  • infidelinfidel Posts: 2,900Member
    : Hello.
    : I have large text (can be 60,000 chars or more) in a string and I have TEXT type field in my table.
    : I need to see if the text in the string is not already in the DB.
    :
    : What is the best way to do this ?
    :
    : What I do now is to take 10 pieces of the string:
    : from char 2 to 102, from char 350 to 450, from char 460 to 560 and so on, and connecting 10 SQL's to check that field.
    :
    : The problem is that it takes 7 seconds...
    :
    : Can I do it better ?

    It depends on whether your database has, or you can find, some kind of hashing function like MD5 or SHA. These algorithms take a string and produce a "hash" which is a 128-bit or 160-bit (or whatever, depends on the algorithm) value. If you find a way to use the same hashing algorithm within the database and in your code, then you can pass the hash of your string into the database and let it compare that value to hashes of the values in the table. If you find a match then you already have the string.

    Or, if it's possible to put a unique index on that column in your database, you could just try an INSERT statement. If it raises an exception then the value is not unique, otherwise it was unique.


    [size=5][italic][blue][RED]i[/RED]nfidel[/blue][/italic][/size]

    [code]
    $ select * from users where clue > 0
    no rows returned
    [/code]

Sign In or Register to comment.