surefire Posted June 3, 2003 Share Posted June 3, 2003 With all of the craziness surrounding Google right now, I was thrilled to see that Google finally visited several hundred of my pages. Although the 513 page visit is less than 10% of the site... I'm not compaining. Maybe in a month or two I'll see some solid ranking results... who knows. Other good SEO news... my wife's website is #1 or #2 on Google for about 9-10 of her best search terms! Maybe I know what I'm doing after all... Nah. That can't be it. PS - At first glance, this post would seem to have zero relevance to anyone... except for the fact that bizarro things have been going on with Google lately testing new ranking algorithms. So maybe this indicates that the new month long dance or whatever they're doing is over. Should be interesting. Quote Link to comment Share on other sites More sharing options...
TCH-Andy Posted June 3, 2003 Share Posted June 3, 2003 Google hit the bulk of my site over the last couple of days to, so maybe it's waking up again I don't know enough about google to know if this is a "fresh" or a "deep" trawl through the site though. Andy Quote Link to comment Share on other sites More sharing options...
surefire Posted June 3, 2003 Author Share Posted June 3, 2003 My understanding is that freshbot can swing by a site several times a month to get updates on pages already in Google's index. Deepcrawl happens once a month (or at least used to) and it searches for new pages by following links within your site. I don't quite understand why it wouldn't follow every single link the first time... but apparently Google's spider grabs more pages with each successive deep crawl. DSDemmin has written several tutorials on this topic in the forum and in the main TCH website. Since Google checked out 513 of my pages, I'd guess that's a deepcrawl. Quote Link to comment Share on other sites More sharing options...
SEO Posted June 4, 2003 Share Posted June 4, 2003 Since Google checked out 513 of my pages, I'd guess that's a deepcrawl.No, I do not think so. The fresh bot is crawling deeper these days and acting a little differnt. If you would like this confirmed, give me the IP of the bot. So maybe this indicates that the new month long dance or whatever they're doing is over. No month long dance. This past dance began on May 2 and was completed on May 20, which was a substantially longer 'dance' then usual. As I have stated in other threads this dance has given very unuasual results (i.e. they had some problems). I belive (as others) that there will be a quicker turn-around to the next dance... they have to get things back in order. Deepcrawl happens once a month (or at least used to) Accurate statement. There was an extended time between the last two dances. Now that things are messed up, the next duration will not reveal what pattern may follow. It was nice when things were relatively consistent (i.e. monthly). We shall see .... Quote Link to comment Share on other sites More sharing options...
surefire Posted June 5, 2003 Author Share Posted June 5, 2003 Well, Google came back and looked at 1041 more pages. So, maybe this is freshbot... I can't tell because I don't know how to use Awstats to view a list of all IP's. I tried to download the raw log file, but it wasn't in a format I could decipher. Dsdemmin, or anyone else, maybe you could give me some much needed help in deciphering which IP Google was using. If memory serves... Freshbot starts with 64. and deepcrawl is 203 or 212 or something like that. Thanks. PS - Even if this isn't deepcrawl, still about 1041 more pages worth of activity then I've ever received before. So I'm pretty pumped. PPS - Dsdemmin, how long would you estimate would be the time (typically... I know it's a guess) from Freshbot to Deepcrawl to those deepcrawl results showing up in results. I would guess one month from Freshbot to Deepcrawl, and then one more month after that for results to show up. But I'm guessing. Quote Link to comment Share on other sites More sharing options...
SEO Posted June 5, 2003 Share Posted June 5, 2003 If memory serves... Freshbot starts with 64. and deepcrawl is 203 or 212 or something like that.Recently 61.* = fresh and 216.* and 218.* = deep. Dsdemmin, how long would you estimate would be the time (typically... I know it's a guess) from Freshbot to Deepcrawl to those deepcrawl results showing up in results. I would guess one month from Freshbot to Deepcrawl, and then one more month after that for results to show up. But I'm guessing. No telling right now... things are just 'not what they used to be'. I have a feeling that the index will be changed shortly (weeks). Your logic (one month plus one month) is sound but if my 'feeling' is correct, you may get in quicker then the two months. Regarding your site's rank: If after you get these pages indexed and you are not pleased with your rankings, let me know. I will have some suggestions on your internal links structure. Large sites usually have an overall disadvantage. However, you can take this disadvantage and turn it into a big advantage with proper structure. Quote Link to comment Share on other sites More sharing options...
surefire Posted June 5, 2003 Author Share Posted June 5, 2003 Thanks for the prompt reply... what was that, five seconds? And thanks for the offer, can I PM or IM you? Or would you prefer thread? My site is quite large and it is consistently devoted to only one theme... so I was thinking this would be good. Please point me in the direction of making the most of all these pages I have. How do I use the cpanel, or any other tool, to find out which IP was used by Google? Thanks for all your help. Quote Link to comment Share on other sites More sharing options...
KevinW Posted June 5, 2003 Share Posted June 5, 2003 OK, so it's my birthday and I'm old --- so someone tell me this -- how does one know that Google just scanned your site, or how many pages they scanned? -kw Quote Link to comment Share on other sites More sharing options...
TCH-Andy Posted June 5, 2003 Share Posted June 5, 2003 KW, There are a number of ways, but all basically involve looking at the log file - usually with a friendly program If you use AWstats, scroll down to Robots/Spiders visitors (Top 25) This should hopefully have "Googlebot (Google)" somewhere (if it hasn't, then you haven't been visited by google), followed by "Hits" (how many pages it has visited), and the time of the last visit. I download the log file to my hard disk and use a log analyser on it to identify which pages have been hit on which day. Andy Quote Link to comment Share on other sites More sharing options...
surefire Posted June 5, 2003 Author Share Posted June 5, 2003 What are you doing analyzing your logs... that's gross. Quote Link to comment Share on other sites More sharing options...
TCH-Andy Posted June 5, 2003 Share Posted June 5, 2003 Two countries seperated by a common language I had to think about that. Reminds me of the many times I've been over to the good old US and got myself into trouble using the wrong words... Andy Quote Link to comment Share on other sites More sharing options...
surefire Posted June 5, 2003 Author Share Posted June 5, 2003 (Blushing...) As stupid as it is... I could never resist potty humor. It's cheap humor... but it never lets you down. On to a serious question... do you have a log analyzer program you'd recommend for TCH log files? I'd like to know which of the Google bots is checking out my site. Thanks Quote Link to comment Share on other sites More sharing options...
TCH-Andy Posted June 5, 2003 Share Posted June 5, 2003 At the moment I'm playing with absolute log analyser - you can download a free trial copy from ; http://www.absolutelog.com/ it reads TCH log files without any difficulty at all, and seems to give me most of the information I'm after. On the summary though, it tells you which pages have been hit when by google (and other search engines), what search words were used, which are the most common etc. and it will turn the IP addresses into domain names for you. On the negative side, it doesn't tell you which IP address google hit you - it just says google. I just opened the log in a text editor and searched for "googlebot", the IP address is at the beginning of that line. All mine were 64.68.xx.xx - which doesn't fit with 61.* but it's closer that 21x.* so I guess it's a fresh bot Andy Quote Link to comment Share on other sites More sharing options...
SEO Posted June 5, 2003 Share Posted June 5, 2003 For log analysis check this old thread Quote Link to comment Share on other sites More sharing options...
surefire Posted June 5, 2003 Author Share Posted June 5, 2003 Darn, I just realized that I wasn't archiving my raw log files... and so unless I'm missing something, I can't look back through previous days... which is where the Google information is. Shucks. Quote Link to comment Share on other sites More sharing options...
SEO Posted June 6, 2003 Share Posted June 6, 2003 This post and some other threads prompted me to write: Looking For Googlebot Hope this helps. Quote Link to comment Share on other sites More sharing options...
natimage Posted June 6, 2003 Share Posted June 6, 2003 Morning...I tried to get the error logs from Cpanel with no success. When I clicked either the bottom or top link, I get a box asking to either open from current location or save to disk. I did both. When I opened from current location, a DOS window popped up and then a "illegal operation" error immediately popped up. The same thing happened when I saved and then double clicked it. I'm sure I'm doing something stupid just because I don't know anything about doing this sort of stuff!!! So, feel free to enlighten me... Tracy Quote Link to comment Share on other sites More sharing options...
surefire Posted June 6, 2003 Author Share Posted June 6, 2003 Open the file within Wordpad. Either right click on the file and choose Open With... or launch Wordpad first and choose the file. Quote Link to comment Share on other sites More sharing options...
SEO Posted June 6, 2003 Share Posted June 6, 2003 Just a quick correction in your wording I tried to get the error logs... So we all understand, these are not error logs, they are referred to as your 'raw logs' or 'access logs'. These provide the raw data for any statistics program (e.g. AwStats) to use and analyze. An error log exists too; it is used for diagnosing problems relating to the web site. Any errors the server encounters will be logged into the error log. Quote Link to comment Share on other sites More sharing options...
KevinW Posted June 6, 2003 Share Posted June 6, 2003 I've added the Looking for Googlebot info to the TCH Help web site, as well - click here. Thanks for taking the time to write it! -kw Quote Link to comment Share on other sites More sharing options...
SEO Posted June 6, 2003 Share Posted June 6, 2003 Looks good KW... thanks. Quote Link to comment Share on other sites More sharing options...
natimage Posted June 7, 2003 Share Posted June 7, 2003 Thanks Jack...got it open. Sorry Scott...about the verbage. Such a newbie... NOW...another question regarding this quote: Further comment: Logs are reset at the end of each month, so the logs of today began on May 31. The log I opened only included the 5th and 6th of June. What am I missing on this one??? "The Big Bot" visited me on the fourth...of course! Quote Link to comment Share on other sites More sharing options...
TCH-Rick Posted June 7, 2003 Share Posted June 7, 2003 One fairly new feature on cPanel is the Raw Log Manager. You can have it save your log each day to a file. It will append the current day's log to the end so you have a complete monthly log. You need to enable it since it is off by default. It will even delete it at the end of the month if you wish although I plan to just download mine to keep locally. Quote Link to comment Share on other sites More sharing options...
surefire Posted June 9, 2003 Author Share Posted June 9, 2003 by the way... it was freshbot that visited me. Maybe it would be a good idea to start a thread the next time a TCH user or must dsdemmin sees Google deepcrawl come through. Freshbot has looked at 1500+ pages of my site. I'm thinking this is a VERY good thing. Quote Link to comment Share on other sites More sharing options...
theladybug Posted June 23, 2003 Share Posted June 23, 2003 How do you know if they added your site??? Do they let you know??? Quote Link to comment Share on other sites More sharing options...
SEO Posted June 23, 2003 Share Posted June 23, 2003 No they do not, I think they might be too busy to notify everyone Go to Google and search... this will give you an idea if you are there. This does not necessarily mean you are in the index though. I would suggest you begin reading all the pinned threads in this topic (Web Site Promotion). Start with 'SEO What is it' and be sure to follow all the links. This will give you a better background to approach the subject. Hope this helps. Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.