{"id":425,"date":"2004-12-12T01:53:36-05:00","date_gmt":"2004-12-12T09:53:36+00:00","guid":{"rendered":"http:\/\/peterjanes.ca\/wordpress\/?p=425"},"modified":"2004-12-12T01:53:36-05:00","modified_gmt":"2004-12-12T09:53:36+00:00","slug":"strange-robot-patterns","status":"publish","type":"post","link":"https:\/\/peterjanes.ca\/blog\/2004\/12\/12\/strange-robot-patterns\/","title":{"rendered":"Strange robot&nbsp;patterns"},"content":{"rendered":"<div class='e-content'><p>If I didn&#8217;t know any better, I&#8217;d say Lycos and Google were in cahoots.  From my webserver logs yesterday:<\/p>\n\n<pre>209.202.221.139 - - [10\/Dec\/2004:02:41:35 -0800] \"HEAD \/LenniFan\/Discography\/Audio\/tensongslive\/Perhaps%20Perhaps%20Perhaps.mp3 HTTP\/1.0\" 200 0 \"-\" <strong>\"Lycos-News-Xml-Fetcher\"<\/strong>\n66.249.66.100 - - [10\/Dec\/2004:02:41:38 -0800] \"GET \/LenniFan\/Discography\/Audio\/tensongslive\/Perhaps%20Perhaps%20Perhaps.mp3 HTTP\/1.1\" 200 91810 \"-\" <strong>\"Mediapartners-Google\/2.1\"<\/strong>\n66.249.66.100 - - [10\/Dec\/2004:02:41:39 -0800] \"GET \/LenniFan\/Discography\/Audio\/tensongslive\/Perhaps%20Perhaps%20Perhaps.mp3 HTTP\/1.1\" 200 491649 \"-\" <strong>\"Mediapartners-Google\/2.1\"<\/strong>\n61.11.93.167 - - [10\/Dec\/2004:02:41:52 -0800] \"GET \/LenniFan\/Discography\/Audio\/tensongslive\/Perhaps%20Perhaps%20Perhaps.mp3 HTTP\/1.1\" 200 445050 \"-\" \"Mozilla\/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)\"<\/pre>\n\n<p>And a little later:<\/p>\n\n<pre>209.202.221.139 - - [10\/Dec\/2004:10:00:15 -0800] \"HEAD \/LenniFan\/Discography\/Audio\/tensongslive\/La%20Chanson%20Triste.mp3 HTTP\/1.0\" 200 0 \"-\" <strong>\"Lycos-News-Xml-Fetcher\"<\/strong>\n66.249.66.100 - - [10\/Dec\/2004:10:00:23 -0800] \"GET \/LenniFan\/Discography\/Audio\/tensongslive\/La%20Chanson%20Triste.mp3 HTTP\/1.1\" 200 2568777 \"-\" <strong>\"Mediapartners-Google\/2.1\"<\/strong>\n195.68.12.162 - - [10\/Dec\/2004:10:00:43 -0800] \"GET \/LenniFan\/Discography\/Audio\/tensongslive\/La%20Chanson%20Triste.mp3 HTTP\/1.0\" 200 623058 \"-\" \"NSPlayer\/8.0.0.4487\"<\/pre>\n\n<p>Strangest of all is that every time they appear, those paired hits are followed closely by a single request from an unrelated IP address using a different browser.  I&#8217;m certain that they&#8217;re related, because there are no surrounding page hits and those files aren&#8217;t something that one would just stumble upon.<\/p>\n\n<p>This has been going on for at least a week, with each crawler retrieving only MP3 files.  If nothing else, Lycos&#8217;s so-called &#8220;News XML Fetcher&#8221; is way out of its league.<\/p><\/div><div class=\"syndication-links\"><\/div>","protected":false},"excerpt":{"rendered":"If I didn&#8217;t know any better, I&#8217;d say Lycos and Google were in cahoots. From my webserver logs yesterday: 209.202.221.139 &#8211; &#8211; [10\/Dec\/2004:02:41:35 -0800] &#8220;HEAD \/LenniFan\/Discography\/Audio\/tensongslive\/Perhaps%20Perhaps%20Perhaps.mp3 HTTP\/1.0&#8221; 200 0 &#8220;-&#8221; &#8220;Lycos-News-Xml-Fetcher&#8221; 66.249.66.100 &#8211; &#8211; [10\/Dec\/2004:02:41:38 -0800] &#8220;GET \/LenniFan\/Discography\/Audio\/tensongslive\/Perhaps%20Perhaps%20Perhaps.mp3 HTTP\/1.1&#8221; 200 91810 &#8220;-&#8221; &#8220;Mediapartners-Google\/2.1&#8221; 66.249.66.100 &#8211; &#8211; [10\/Dec\/2004:02:41:39 -0800] &#8220;GET \/LenniFan\/Discography\/Audio\/tensongslive\/Perhaps%20Perhaps%20Perhaps.mp3 HTTP\/1.1&#8221; 200 491649 &#8220;-&#8221; &#8220;Mediapartners-Google\/2.1&#8243;&hellip;","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"","sticky":false,"template":"","format":"standard","meta":{"mf2_syndication":[],"venue_id":0},"categories":[3],"tags":[],"kind":false,"_links":{"self":[{"href":"https:\/\/peterjanes.ca\/blog\/wp-json\/wp\/v2\/posts\/425"}],"collection":[{"href":"https:\/\/peterjanes.ca\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/peterjanes.ca\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/peterjanes.ca\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/peterjanes.ca\/blog\/wp-json\/wp\/v2\/comments?post=425"}],"version-history":[{"count":0,"href":"https:\/\/peterjanes.ca\/blog\/wp-json\/wp\/v2\/posts\/425\/revisions"}],"wp:attachment":[{"href":"https:\/\/peterjanes.ca\/blog\/wp-json\/wp\/v2\/media?parent=425"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/peterjanes.ca\/blog\/wp-json\/wp\/v2\/categories?post=425"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/peterjanes.ca\/blog\/wp-json\/wp\/v2\/tags?post=425"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}