[dvLAN]

Hello,
I'd like to know if it possible to build search engine that will search all over the net, searching by meta tags, just like google. But, do I need cache servers? In the system, I want my own Pay-Per-Click and Webmasters program for the Pay-Per-Click ads. Does anyone know how to do it?
Without using API's just my own system.
I'm ready to pay until 10000$ for my own search engine.
Thanks,
Nissim.

[Eclipse]

Well it is possible but you would need a giant amount of bandwidth and disk space, you could set up that it searchs using fsocket() and randomly generating ip address and such. I belive that most spyders {what you seem to be asking for} are writen with java or C.

[infinitylimit]

Randomly generating ip addresses isn't exactly the most effciencent way to do it but it can be done with php.

I've done this in a smaller fashion with php as a image grabber. You enter in a few sites and it would run through the sites looking for images.

The search engine would do basically the same, just using metatags. Use fsocketopen then look for links and follow all the links (like a spider) caching pages. You could seed it with a random ip I suppose but it should do fine spidering on it's own. This is also how you page rank is through spidering links to follow.

Eclipse is right though most web engines I've seen are written in C or java.

You can always buy google's search engine.
http://www.google.com/appliance/products.html

If after all this you still think you want to do this shoot me an email and we can explore the idea more.

[hardcoded]

AFAIK, google ignores meta tag, or at least, it is not exclusively what it uses to build it's database.

[dvLAN]

Bandwidth isnt problem, I have 50MBIT bandwidth each server.
And if I want, I can make it 100MBIT or 150MBIT.
So, what do I need to do?
The fsocketopen is caching the pages to my servers?
But how does it find the documents on the meta tags?
Can anyone give me an exemple of the code?

Thanks VERY VERY VERY VERY MACH,
and sorry about my bad english.

[dvLAN]

Originally Posted by hardcoded

AFAIK, google ignores meta tag, or at least, it is not exclusively what it uses to build it's database.

hardcoded, I didnt talked about google,
I talked about my private search engine for my site .
Thanks for the help.

[hardcoded]

Originally Posted by dvLAN

Hello,
that will search all over the net, searching by meta tags, just like google.

There, that's what I meant.

[mikaelf]

i ever tried to make a very advanced webbot which can spider, crawl, and even read a very long query string (like http://www.blah-blah.com/page.html?g...fgafgafdga=gga...). anyway, i don't have much time to finish it and later decided to use built up one. mnogosearch is one of the best for free. if you want to build a customized one. you can contact people from eastern europe. they are great in deploying algorithm and their cost is cheap.
here is the contact:

>> Oksana Svetlova.
>> oksa [at] rt03.imxee.com
>> Website : www.onxee.com

[Eclipse]

Why are you so set on doing this in PHP Java or C/C++ or even C# would be a lot easier?