11 Commits

Author	SHA1	Message	Date
wilsonge	a1158ee07f	Merge branch '3.10-dev' into 4.0-dev	2020-12-15 18:17:51 +00:00
Brian Teeman	767ec05fe5	Update robots.txt.dist (#31327) ... Update the information link as it is now available on https Remove the checker link as it is no longer available. (I couldn't find another vendor neutral checker to replace it with)	2020-11-07 13:32:09 +01:00
Phil Taylor	67b52bc87f	[4.0] For the same reasons /cli is there - add /api to robots.txt (#29963) ... * Add /api from robots.txt * 1-z	2020-07-04 15:31:49 +02:00
mmustermann628483	78ffb8050d	Update robots.txt.dist (#23916)	2019-02-16 12:05:05 +01:00
Brian Teeman	aacfe55109	Update grammar (#10778)	2016-06-16 00:17:29 +01:00
Gruber	d6acff007b	Update robots.txt.dist ... as per https://github.com/joomla/joomla-cms/pull/3965 removed the commented folders (templates and media)	2014-07-25 18:14:40 +02:00
Gruber	089df85eab	Update robots.txt.dist ... I've revised a bit the code again, giving total access to both `templates` and `media`, since there could be images and other useful stuff apart `CSS` and `js` files. Should be evaluated if totally delete the filters or leave them commented.	2014-07-23 03:14:20 +02:00
Gruber	5272105be8	SEO improvement to robot.txt ... SEO improvement to robot.txt allowing .css and .js file to be accessed by bots in their crawlings. See https://github.com/joomla/joomla-cms/issues/3937	2014-07-22 21:36:32 +02:00
Marco Richter	0aa6759a27	Update robots.txt.dist ... Remove /images from robots.txt	2014-02-13 15:12:47 +01:00
Matt Thomas	4c04e51a0a	# [#32393] /layouts not included in robots.txt Thanks Matt Thomas ... (Fix #2306)	2013-10-23 09:39:28 +02:00
Mark Dexter	1f37e4c7ac	[#29677] robots.txt in updates	2013-03-17 12:44:04 -04:00