Howdy, Stranger!

It looks like you're new here. If you want to get involved, click one of these buttons!

Categories

Database Structure for Web Search Engine

TNT_TNT_ Member Posts: 3
I'm trying to build a web search engine to index around 2000 web sites. I've chosen mysql as the database and came up with 3 table definitions to store the data. table words that contains the list of words along with a id-key. table webpages that contains the list of URL's with each with an id, and a table called wordlinks that contains wordid and webpageid pairs denoting that the url whose id is webpageid contains the word whose id is wordid. This sounded like a good solution in theory but when implemented, I tried to index a medium-sized web site and it resulted into 17mb of data in the wordlinks table which obviously is far from space-efficient. Could anyone suggest an alternative structure?

Comments

  • Shawn CarterShawn Carter Member Posts: 0

    < http://forcoder.org /> free video tutorials and ebooks about / C Python MATLAB C++ R Ruby PL/SQL Perl Scratch Delphi Visual Basic Go Objective-C Visual Basic .NET Swift JavaScript C# PHP Assembly Java Scheme Bash Erlang COBOL D Kotlin VBScript Julia Fortran FoxPro Lua ABAP Apex Crystal Rust Alice Scala Awk F# Transact-SQL Lisp ML Dart Hack LabVIEW Ada Clojure Prolog SAS Logo / __________

Sign In or Register to comment.