MelH Posted May 30, 2006 Posted May 30, 2006 hello all! I run MySoccerLinks.com It's a soccer specific directory. I'm adding websites manually, as well as encouraging soccer websites to submit themselves. I'm looking for a crawler to add websites to a 'pending' sql database, so then I can review, modify and categorize, and approve for inclusion. I want to be able to set the starting URL, and then allow the crawler to do it's thing. I'm not looking for millions of URLs, not looking to be the next DMOZ - I'm just looking for an easier way to add websites without a lot of the finding, cutting, and pasting that I'm doing now. Anyone have any suggestions? Mel Quote
j2k4b Posted May 31, 2006 Posted May 31, 2006 The only thing I can think of at the moment is the PHPLinks that is located in your cPanel. Quote
j2k4b Posted May 31, 2006 Posted May 31, 2006 The only thing I can think of at the moment is the PHPLinks that is located in your cPanel. PHPlinks is not in TCH's cpanel. It comes with Fantastico but I guess TCH found it as a treat and removed it. I could be wrong! I was the first time. Quote
Deverill Posted May 31, 2006 Posted May 31, 2006 I've never used any of this kind of thing but I did find several interesting looking possibilities on a Google search. Quote
MelH Posted May 31, 2006 Author Posted May 31, 2006 I found the crawler portion of http://www.cs.ioc.ee/~ando/sphider/ to do exactly what I wanted. Very very interesting...... Mel Quote
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.