{"id":815,"date":"2008-04-06T13:04:05-04:00","date_gmt":"2008-04-06T17:04:05+00:00","guid":{"rendered":"http:\/\/peterjanes.ca\/blog\/?p=815"},"modified":"2008-04-06T13:20:26-04:00","modified_gmt":"2008-04-06T17:20:26+00:00","slug":"history-and-future-of-robots-exclusion","status":"publish","type":"post","link":"https:\/\/peterjanes.ca\/blog\/2008\/04\/06\/history-and-future-of-robots-exclusion\/","title":{"rendered":"History and future of&nbsp;robots-exclusion"},"content":{"rendered":"<div class='e-content'><p>About three and a half years ago I <a href=\"http:\/\/peterjanes.ca\/blog\/2004\/08\/24\/xmdp-style-robot-profile\/\">published<\/a> a first cut at what I called an <cite class=\"title\"><a href=\"http:\/\/gmpg.org\/xmdp\/\"><abbr title=\"XHTML MetaData Profile\">XMDP<\/abbr><\/a>-style Robot Profile<\/cite>.  XMDP forms <a href=\"http:\/\/microformats.org\/wiki\/profile-uris\">the basis<\/a> of what are now known as <a href=\"http:\/\/microformats.org\/\">microformats<\/a>, so a few months after that post I created an <a href=\"http:\/\/peterjanes.ca\/2005\/robots\/example.xhtml\">example<\/a> and a page on the Technorati developers wiki.  Shortly after <em>that<\/em> the wiki was moved to <a href=\"http:\/\/microformats.org\/\">its own domain<\/a> and <a href=\"http:\/\/microformats.org\/wiki\/robots-exclusion\">my <cite class=\"title\">robots-exclusion<\/cite> page<\/a> with it.<\/p>\r\n\r\n<p>Since then I&#8217;ve been sort of an absentee landlord; several people, including microformat maven <a href=\"http:\/\/tantek.com\/\">Tantek &Ccedil;elik<\/a>, have <a href=\"http:\/\/microformats.org\/wiki?title=robots-exclusion&amp;action=history\">contributed comments<\/a>, but the profile itself hasn&#8217;t changed, or really <a href=\"http:\/\/www.google.com\/search?q=link%3Ahttp%3A%2F%2Fmicroformats.org%2Fwiki%2Frobots-exclusion\">garnered much attention<\/a>.  Regardless of that&#8212;and perhaps because of it&#8212;I recently released the document and all of my (few) contributions to the wiki in general into the public domain (it was previously under <a href=\"http:\/\/creativecommons.org\/licenses\/by\/2.0\">CC-BY<\/a>), prompted by <a title=\"Making open standards as open as possible &mdash; required public domain licensing of all microformats wiki contributions\" href=\"http:\/\/microformats.org\/blog\/2007\/12\/29\/making-open-standards-as-open-as-possible\/\">this post<\/a>.<\/p>\r\n\r\n<p>I hope that the proviso that <q cite=\"http:\/\/microformats.org\/blog\/2007\/12\/29\/making-open-standards-as-open-as-possible\/\">any content added to the microformats wiki from here forward is placed into the public domain for maximum possible reuse<\/q> means there will be more of a look at this <a title=\"We need to discuss whether or not this is a suitable microformat.\" href=\"http:\/\/microformats.org\/wiki\/robots-exclusion-issues#Suitability_as_a_microformat\">pseudo-microformat<\/a>.  I haven&#8217;t seen other ideas (<a href=\"http:\/\/peterjanes.ca\/blog\/2005\/02\/21\/google-off\/\">vendor-specific ones<\/a> aside) take off, so I think it&#8217;s still an itch that needs to be scratched.<\/p><\/div><div class=\"syndication-links\"><\/div>","protected":false},"excerpt":{"rendered":"About three and a half years ago I published a first cut at what I called an XMDP-style Robot Profile. XMDP forms the basis of what are now known as microformats, so a few months after that post I created an example and a page on the Technorati developers wiki. Shortly after that the wiki&hellip;","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"mf2_syndication":[],"venue_id":0},"categories":[3],"tags":[],"kind":false,"_links":{"self":[{"href":"https:\/\/peterjanes.ca\/blog\/wp-json\/wp\/v2\/posts\/815"}],"collection":[{"href":"https:\/\/peterjanes.ca\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/peterjanes.ca\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/peterjanes.ca\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/peterjanes.ca\/blog\/wp-json\/wp\/v2\/comments?post=815"}],"version-history":[{"count":0,"href":"https:\/\/peterjanes.ca\/blog\/wp-json\/wp\/v2\/posts\/815\/revisions"}],"wp:attachment":[{"href":"https:\/\/peterjanes.ca\/blog\/wp-json\/wp\/v2\/media?parent=815"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/peterjanes.ca\/blog\/wp-json\/wp\/v2\/categories?post=815"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/peterjanes.ca\/blog\/wp-json\/wp\/v2\/tags?post=815"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}