Optimisations ‐ Managing Search Engines - DNA-styx/HLStatsX-Windmill-Theme GitHub Wiki

HLStatsx does not appear to be optimised for search engine crawling. Example: a single game on a low population server with ~400 players resulted in Google indexing 207,000 pages due to query strings used to sort tables. Some meta tags have been added to this theme been added to help reduce this load ( ref #62 & commit, but I recommend using a robots.txt both to reduce the load on your SQL server when a search engine crawls your site and also to increase the 'usefulness' of search engine results. Feedback and improvements welcome on this subject.

User-agent: *
    Disallow: /*?*maps_sort=*
    Disallow: /*?*maps_sortorder=*
    Disallow: /*?*obj_sort=*
    Disallow: /*?*obj_sortorder=*
    Disallow: /*?*ppa_sort=*
    Disallow: /*?*ppa_sortorder=*
    Disallow: /*?*playerkills_page=*
    Disallow: /*?*playerkills_sort=*
    Disallow: /*?*playerkills_sortorder=*
    Disallow: /*?*roles_sort=*
    Disallow: /*?*roles_sortorder=*
    Disallow: /*?*server_sort=*
    Disallow: /*?*server_sortorder=*
    Disallow: /*?*teams_sort=*
    Disallow: /*?*teams_sortorder=*
    Disallow: /*?*weap_sort=*
    Disallow: /*?*weap_sortorder=* 
	
    Disallow: /ingame.php 
    Disallow: /show_graph.php 
    Disallow: /sig.php
    Disallow: /trend_graph.php
    Disallow: /assets/ 
    Disallow: /css/
    Disallow: /hlstatsimg/
    Disallow: /includes/ 
    Disallow: /styles/
    Disallow: /updater/