Current location: Hot Scripts Forums » Programming Languages » Perl » URL Harvestor or Web Robot


URL Harvestor or Web Robot

Reply
  #1 (permalink)  
Old 04-30-04, 10:13 PM
stingraze stingraze is offline
Newbie Coder
 
Join Date: Aug 2003
Posts: 8
Thanks: 0
Thanked 0 Times in 0 Posts
URL Harvestor or Web Robot

Hello. I made a search engine with Perl, and it is all pretty good. But the problem I have is that I don't have a URL harvestor (or Web Robot) to get the keywords and sites to the database. This means that I have to do this manually. If some one could teach me or make me a URL Harvestor, it will be greatly appreciated. The search engine is: http://www.stingraze.net/search.htm

My database is like this: keyword;URL;description all separated by semicolon. Please reply.

Best regards,
Tsubasa Kato
__________________
visit http://www.stingraze.net/search.htm for Mohawk Search Engine
Reply With Quote
  #2 (permalink)  
Old 05-02-04, 05:06 AM
Skeleton Man Skeleton Man is offline
Community Liaison
 
Join Date: Jun 2003
Location: Australia
Posts: 406
Thanks: 0
Thanked 0 Times in 0 Posts
Making such a thing wouldn't be too difficult in theory.. look into the LWP::Simple module for easy http connection to URLs..

http://search.cpan.org/~gaas/libwww-.../LWP/Simple.pm
Reply With Quote
  #3 (permalink)  
Old 05-04-04, 08:10 PM
stingraze stingraze is offline
Newbie Coder
 
Join Date: Aug 2003
Posts: 8
Thanks: 0
Thanked 0 Times in 0 Posts
Thanks for the reply.

Quote:
Originally Posted by Skeleton Man
Making such a thing wouldn't be too difficult in theory.. look into the LWP::Simple module for easy http connection to URLs..

http://search.cpan.org/~gaas/libwww-.../LWP/Simple.pm
Thankyou for the reply. I think I'll think about ideas from this module. Other people please post if you have any ideas.
__________________
visit http://www.stingraze.net/search.htm for Mohawk Search Engine
Reply With Quote
Reply

Bookmarks


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
send e-mail through mailto protocol, url length rajeshnaidu JavaScript 1 04-28-04 09:36 AM
change IE search URL or change window registry sma JavaScript 1 04-26-04 02:40 PM
change IE search URL or change window registry sma PHP 0 04-26-04 05:38 AM
Banned URL Array optix PHP 14 04-16-04 03:26 AM
URL masking and redirection Scion HTML/XHTML/XML 4 04-14-04 04:51 AM


All times are GMT -5. The time now is 04:58 AM.
vBulletin® Copyright ©2000 - 2012, Jelsoft Enterprises Ltd.