{"id":1301,"date":"2023-05-31T17:07:42","date_gmt":"2023-05-31T15:07:42","guid":{"rendered":"https:\/\/iconsult.agency\/blog\/?p=1301"},"modified":"2024-12-13T13:34:39","modified_gmt":"2024-12-13T11:34:39","slug":"kak-zablokirovat-nezhelatelnikh-robotov-v-apache-i-nginx","status":"publish","type":"post","link":"https:\/\/iconsult.agency\/blog\/kak-zablokirovat-nezhelatelnikh-robotov-v-apache-i-nginx\/","title":{"rendered":"\u041a\u0430\u043a \u0437\u0430\u0431\u043b\u043e\u043a\u0438\u0440\u043e\u0432\u0430\u0442\u044c \u043d\u0435\u0436\u0435\u043b\u0430\u0442\u0435\u043b\u044c\u043d\u044b\u0445 \u0440\u043e\u0431\u043e\u0442\u043e\u0432 \u0432 Apache \u0438 Nginx, \u0430 \u0442\u0430\u043a\u0436\u0435 Robots.txt"},"content":{"rendered":"\n<p>\u042d\u0442\u043e \u043f\u0435\u0440\u0435\u0432\u043e\u0434 \u0441\u0442\u0430\u0442\u044c\u0438 \u0441 Builder Society. \u0421\u0441\u044b\u043b\u043a\u0430 \u043d\u0430 \u0438\u0441\u0442\u043e\u0447\u043d\u0438\u043a \u0432 \u043a\u043e\u043d\u0446\u0435.<\/p>\n\n\n\n<p><\/p>\n\n\n\n<p>\u0427\u0442\u043e\u0431\u044b \u043d\u0435 \u043e\u0442\u043f\u0440\u0430\u0432\u043b\u044f\u0442\u044c \u043b\u044e\u0434\u0435\u0439 \u043d\u0430 \u043c\u043e\u0439 \u0441\u0430\u0439\u0442 moneyoverethics.com \u0437\u0430 \u043f\u043e\u0441\u043b\u0435\u0434\u043d\u0438\u043c\u0438 \u0432\u0435\u0440\u0441\u0438\u044f\u043c\u0438 \u0440\u043e\u0431\u043e\u0442\u043e\u0432, \u044f \u0440\u0435\u0448\u0438\u043b \u043f\u0440\u043e\u0441\u0442\u043e \u0441\u043e\u0437\u0434\u0430\u0442\u044c \u0440\u0430\u0431\u043e\u0442\u0430\u044e\u0449\u0443\u044e\/\u043e\u0431\u043d\u043e\u0432\u043b\u044f\u0435\u043c\u0443\u044e \u0432\u0435\u0440\u0441\u0438\u044e \u0437\u0434\u0435\u0441\u044c. \u042f \u0440\u0435\u043a\u043e\u043c\u0435\u043d\u0434\u0443\u044e \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u0442\u044c \u0441\u043b\u0435\u0434\u0443\u044e\u0449\u0438\u0435 \u0434\u043e\u043f\u043e\u043b\u043d\u0435\u043d\u0438\u044f \u0434\u043b\u044f \u0431\u0440\u0430\u0443\u0437\u0435\u0440\u0430, \u0447\u0442\u043e\u0431\u044b \u043f\u0440\u043e\u0432\u0435\u0440\u0438\u0442\u044c, \u0434\u0435\u0439\u0441\u0442\u0432\u0438\u0442\u0435\u043b\u044c\u043d\u043e \u043b\u0438 \u0432\u044b \u0431\u043b\u043e\u043a\u0438\u0440\u0443\u0435\u0442\u0435 \u0442\u0435\u0445 \u0431\u043e\u0442\u043e\u0432, \u043a\u043e\u0442\u043e\u0440\u044b\u0445 \u0445\u043e\u0442\u0438\u0442\u0435 \u0437\u0430\u0431\u043b\u043e\u043a\u0438\u0440\u043e\u0432\u0430\u0442\u044c. \ud83d\udc47<br><\/p>\n\n\n\n<!--more-->\n\n\n\n<p>\u0412\u044b \u043c\u043e\u0436\u0435\u0442\u0435 \u0441\u0434\u0435\u043b\u0430\u0442\u044c \u044d\u0442\u043e \u0441 \u043f\u043e\u043c\u043e\u0449\u044c\u044e \u0440\u0430\u0441\u0448\u0438\u0440\u0435\u043d\u0438\u044f User Agent Extension:<br>\u0434\u043b\u044f <strong>firefox<\/strong>: <a href=\"https:\/\/addons.mozilla.org\/en-US\/firefox\/addon\/user-agent-switcher\/\" target=\"_blank\" rel=\"noreferrer noopener nofollow\">https:\/\/addons.mozilla.org\/en-US\/firefox\/addon\/user-agent-switcher\/<\/a><br>\u0434\u043b\u044f <strong>chrome<\/strong>: <a href=\"https:\/\/chrome.google.com\/webstore\/detail\/user-agent-switcher-for-c\/djflhoibgkdhkhhcedjiklpkjnoahfmg\" target=\"_blank\" rel=\"noreferrer noopener nofollow\">https:\/\/chrome.google.com\/webstore\/detail\/user-agent-switcher-for-c\/djflhoibgkdhkhhcedjiklpkjnoahfmg<\/a><\/p>\n\n\n\n<p>\u0415\u0441\u043b\u0438 \u0432\u044b \u043d\u0435 \u0445\u043e\u0442\u0438\u0442\u0435 \u0432\u043e\u0437\u0438\u0442\u044c\u0441\u044f \u0441 .htaccess \u0438 \u043a\u043e\u0434\u0430\u043c\u0438, \u0442\u043e \u043c\u043e\u0436\u043d\u043e \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u0442\u044c \u043f\u043b\u0430\u0433\u0438\u043d \u0434\u043b\u044f \u044d\u0442\u043e\u0439 \u0437\u0430\u0434\u0430\u0447\u0438.<\/p>\n\n\n\n<p>\u0415\u0441\u0442\u044c \u0431\u0435\u0441\u043f\u043b\u0430\u0442\u043d\u044b\u0435 \u043f\u043b\u0430\u0433\u0438\u043d\u044b, \u0430 \u0435\u0441\u0442\u044c \u043f\u043b\u0430\u0442\u043d\u044b\u0435, \u043d\u0430\u043f\u0440\u0438\u043c\u0435\u0440 Spyder Spanker.<br>\u0418\u0437 \u0431\u0435\u0441\u043f\u043b\u0430\u0442\u043d\u044b\u0445 \u043c\u043e\u0436\u043d\u043e \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u0442\u044c Link Privacy.<br><\/p>\n\n\n\n<p><span class=\"has-text-color has-normal-font-size has-vivid-cyan-blue-color\" style=\"font-size: 20px\"><strong>\u041f\u0440\u0438\u043c\u0435\u0447\u0430\u043d\u0438\u0435.<\/strong><\/span> \u041d\u0435 \u043d\u0430\u0434\u043e \u043f\u0440\u043e\u0441\u0442\u043e \u0441\u043b\u0435\u043f\u043e \u043a\u043e\u043f\u0438\u0440\u043e\u0432\u0430\u0442\u044c \u044d\u0442\u0438 \u0444\u0430\u0439\u043b\u044b, \u043d\u0435 \u043f\u0440\u043e\u0441\u043c\u043e\u0442\u0440\u0435\u0432 \u0438\u0445 \u0438 \u043d\u0435 \u043f\u0440\u043e\u0432\u0435\u0440\u0438\u0432 \u0434\u0432\u0430\u0436\u0434\u044b, \u0447\u0442\u043e\u0431\u044b \u0443\u0431\u0435\u0434\u0438\u0442\u044c\u0441\u044f, \u0447\u0442\u043e \u0432\u044b \u043d\u0435 \u0431\u043b\u043e\u043a\u0438\u0440\u0443\u0435\u0442\u0435 \u0442\u0435\u0445 \u0431\u043e\u0442\u043e\u0432, \u043a\u043e\u0442\u043e\u0440\u044b\u0435 \u0434\u043e\u043b\u0436\u043d\u044b \u043b\u0430\u0437\u0438\u0442\u044c \u043f\u043e \u0432\u0430\u0448\u0435\u043c\u0443 \u0441\u0430\u0439\u0442\u0443. \u041d\u0430\u043f\u0440\u0438\u043c\u0435\u0440, \u044d\u0442\u0438 \u0444\u0430\u0439\u043b\u044b \u0431\u043b\u043e\u043a\u0438\u0440\u0443\u044e\u0442 'curl', 'python', 'perl' \u0438 \u0434\u0430\u0436\u0435 'SEMRush' - \u0435\u0441\u043b\u0438 \u0432\u044b \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u0443\u0435\u0442\u0435 \u0431\u043e\u0442\u043e\u0432 \u0438\u043b\u0438 \u0441\u0435\u0440\u0432\u0438\u0441\u044b, \u0437\u0430\u0431\u043b\u043e\u0447\u0435\u043d\u043d\u044b\u0435 \u0432 \u044d\u0442\u0438\u0445 \u044d\u0442\u0438 \u0444\u0430\u0439\u043b\u0430\u0445, \u0432\u044b \u043d\u0435 \u0442\u043e\u043b\u044c\u043a\u043e \u0431\u043b\u043e\u043a\u0438\u0440\u0443\u0435\u0442\u0435 \u043f\u043e\u043f\u044b\u0442\u043a\u0438 \u0441\u0432\u043e\u0438\u0445 \u043a\u043e\u043d\u043a\u0443\u0440\u0435\u043d\u0442\u043e\u0432 \u044e\u0437\u0430\u0442\u044c \u044d\u0442\u0438\u0445 \u0441\u0435\u0440\u0432\u0438\u0441\u044b, \u0447\u0442\u043e\u0431\u044b \u0441\u043e\u0431\u0438\u0440\u0430\u0442\u044c \u0438\u043d\u0444\u0443 \u043f\u0440\u043e \u0432\u0430\u0448 \u0434\u043e\u043c\u0435\u043d, \u043d\u043e \u0438 \u0441\u0432\u043e\u0438 \u043f\u043e\u043f\u044b\u0442\u043a\u0438. \u042f \u0431\u043b\u043e\u043a\u0438\u0440\u0443\u044e \u0412\u0421\u0401!<\/p>\n\n\n\n<div class=\"gutentoc tocactive nostyle\" style=\"max-width:292px\"><div class=\"gutentoc-toc-wrap\"><div class=\"gutentoc-toc-title-wrap\"><div class=\"gutentoc-toc-title\">\u0421\u043e\u0434\u0435\u0440\u0436\u0430\u043d\u0438\u0435<\/div><div id=\"open\" class=\"toggletwo\">show<\/div><\/div><div id=\"toclist\"><div class=\"gutentoc-toc__list-wrap\"><ul class=\"gutentoc-toc__list\"><li><a href=\"#\u0432\u0435\u0440\u0441\u0438\u044f-\u0434\u043b\u044f-robotstxt\">\u0412\u0435\u0440\u0441\u0438\u044f \u0434\u043b\u044f Robots.txt<\/a><\/li><li><a href=\"#\u0432\u0435\u0440\u0441\u0438\u044f-\u0434\u043b\u044f-httaccess-apache-\">\u0412\u0435\u0440\u0441\u0438\u044f \u0434\u043b\u044f .httaccess (Apache) <\/a><\/li><li><a href=\"#\u0432\u0435\u0440\u0441\u0438\u044f-\u0434\u043b\u044f-nginxconf-nginx\">\u0412\u0435\u0440\u0441\u0438\u044f \u0434\u043b\u044f nginx.conf (NGINX)<\/a><\/li><\/ul><\/div><\/div><\/div><\/div>\n\n\n<h2 class=\"wp-block-heading has-text-color\" id=\"\u0432\u0435\u0440\u0441\u0438\u044f-\u0434\u043b\u044f-robotstxt\" style=\"color:#3cc22b\">\u0412\u0435\u0440\u0441\u0438\u044f \u0434\u043b\u044f Robots.txt<\/h2>\n\n\n<p>\u0414\u043b\u044f \u043d\u0430\u0447\u0430\u043b\u0430 \u0432\u0435\u0440\u0445\u043d\u0435\u0443\u0440\u043e\u0432\u043d\u0435\u0432\u044b\u0439 robots.txt, \u00ab\u0445\u043e\u0440\u043e\u0448\u0438\u0435\u00bb \u0440\u043e\u0431\u043e\u0442\u044b \u00ab\u0434\u043e\u043b\u0436\u043d\u044b\u00bb \u0441\u043e\u0431\u043b\u044e\u0434\u0430\u0442\u044c \u044d\u0442\u043e:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>User-agent: AhrefsBot\nUser-agent: AhrefsSiteAudit\nUser-agent: adbeat_bot\nUser-agent: Alexibot\nUser-agent: AppEngine\nUser-agent: Aqua_Products\nUser-agent: archive.org_bot\nUser-agent: archive\nUser-agent: asterias\nUser-agent: b2w\/0.1\nUser-agent: BackDoorBot\/1.0\nUser-agent: BecomeBot\nUser-agent: BlekkoBot\nUser-agent: Blexbot\nUser-agent: BlowFish\/1.0\nUser-agent: Bookmark search tool\nUser-agent: BotALot\nUser-agent: BuiltBotTough\nUser-agent: Bullseye\/1.0\nUser-agent: BunnySlippers\nUser-agent: CCBot\nUser-agent: CheeseBot\nUser-agent: CherryPicker\nUser-agent: CherryPickerElite\/1.0\nUser-agent: CherryPickerSE\/1.0\nUser-agent: chroot\nUser-agent: Copernic\nUser-agent: CopyRightCheck\nUser-agent: cosmos\nUser-agent: Crescent\nUser-agent: Crescent Internet ToolPak HTTP OLE Control v.1.0\nUser-agent: DittoSpyder\nUser-agent: dotbot\nUser-agent: dumbot\nUser-agent: EmailCollector\nUser-agent: EmailSiphon\nUser-agent: EmailWolf\nUser-agent: Enterprise_Search\nUser-agent: Enterprise_Search\/1.0\nUser-agent: EroCrawler\nUser-agent: es\nUser-agent: exabot\nUser-agent: ExtractorPro\nUser-agent: FairAd Client\nUser-agent: Flaming AttackBot\nUser-agent: Foobot\nUser-agent: Gaisbot\nUser-agent: GetRight\/4.2\nUser-agent: gigabot\nUser-agent: grub\nUser-agent: grub-client\nUser-agent: Go-http-client\nUser-agent: Harvest\/1.5\nUser-agent: Hatena Antenna\nUser-agent: hloader\nUser-agent: http:\/\/www.SearchEngineWorld.com bot\nUser-agent: http:\/\/www.WebmasterWorld.com bot\nUser-agent: httplib\nUser-agent: humanlinks\nUser-agent: ia_archiver\nUser-agent: ia_archiver\/1.6\nUser-agent: InfoNaviRobot\nUser-agent: Iron33\/1.0.2\nUser-agent: JamesBOT\nUser-agent: JennyBot\nUser-agent: Jetbot\nUser-agent: Jetbot\/1.0\nUser-agent: Jorgee\nUser-agent: Kenjin Spider\nUser-agent: Keyword Density\/0.9\nUser-agent: larbin\nUser-agent: LexiBot\nUser-agent: libWeb\/clsHTTP\nUser-agent: LinkextractorPro\nUser-agent: LinkpadBot\nUser-agent: LinkScan\/8.1a Unix\nUser-agent: LinkWalker\nUser-agent: LNSpiderguy\nUser-agent: looksmart\nUser-agent: lwp-trivial\nUser-agent: lwp-trivial\/1.34\nUser-agent: Mata Hari\nUser-agent: Megalodon\nUser-agent: Microsoft URL Control\nUser-agent: Microsoft URL Control - 5.01.4511\nUser-agent: Microsoft URL Control - 6.00.8169\nUser-agent: MIIxpc\nUser-agent: MIIxpc\/4.2\nUser-agent: Mister PiX\nUser-agent: MJ12bot\nUser-agent: moget\nUser-agent: moget\/2.1\nUser-agent: mozilla\nUser-agent: Mozilla\nUser-agent: mozilla\/3\nUser-agent: mozilla\/4\nUser-agent: Mozilla\/4.0 (compatible; BullsEye; Windows 95)\nUser-agent: Mozilla\/4.0 (compatible; MSIE 4.0; Windows 2000)\nUser-agent: Mozilla\/4.0 (compatible; MSIE 4.0; Windows 95)\nUser-agent: Mozilla\/4.0 (compatible; MSIE 4.0; Windows 98)\nUser-agent: Mozilla\/4.0 (compatible; MSIE 4.0; Windows NT)\nUser-agent: Mozilla\/4.0 (compatible; MSIE 4.0; Windows XP)\nUser-agent: mozilla\/5\nUser-agent: MSIECrawler\nUser-agent: naver\nUser-agent: NerdyBot\nUser-agent: NetAnts\nUser-agent: NetMechanic\nUser-agent: NICErsPRO\nUser-agent: Nutch\nUser-agent: Offline Explorer\nUser-agent: Openbot\nUser-agent: Openfind\nUser-agent: Openfind data gathere\nUser-agent: Oracle Ultra Search\nUser-agent: PerMan\nUser-agent: ProPowerBot\/2.14\nUser-agent: ProWebWalker\nUser-agent: psbot\nUser-agent: Python-urllib\nUser-agent: QueryN Metasearch\nUser-agent: Radiation Retriever 1.1\nUser-agent: RepoMonkey\nUser-agent: RepoMonkey Bait &amp; Tackle\/v1.01\nUser-agent: RMA\nUser-agent: rogerbot\nUser-agent: scooter\nUser-agent: Screaming Frog SEO Spider\nUser-agent: searchpreview\nUser-agent: SEMrushBot\nUser-agent: SemrushBot\nUser-agent: SemrushBot-SA\nUser-agent: SEOkicks-Robot\nUser-agent: SiteSnagger\nUser-agent: sootle\nUser-agent: SpankBot\nUser-agent: spanner\nUser-agent: spbot\nUser-agent: Stanford\nUser-agent: Stanford Comp Sci\nUser-agent: Stanford CompClub\nUser-agent: Stanford CompSciClub\nUser-agent: Stanford Spiderboys\nUser-agent: SurveyBot\nUser-agent: SurveyBot_IgnoreIP\nUser-agent: suzuran\nUser-agent: Szukacz\/1.4\nUser-agent: Szukacz\/1.4\nUser-agent: Teleport\nUser-agent: TeleportPro\nUser-agent: Telesoft\nUser-agent: Teoma\nUser-agent: The Intraformant\nUser-agent: TheNomad\nUser-agent: toCrawl\/UrlDispatcher\nUser-agent: True_Robot\nUser-agent: True_Robot\/1.0\nUser-agent: turingos\nUser-agent: Typhoeus\nUser-agent: URL Control\nUser-agent: URL_Spider_Pro\nUser-agent: URLy Warning\nUser-agent: VCI\nUser-agent: VCI WebViewer VCI WebViewer Win32\nUser-agent: Web Image Collector\nUser-agent: WebAuto\nUser-agent: WebBandit\nUser-agent: WebBandit\/3.50\nUser-agent: WebCopier\nUser-agent: WebEnhancer\nUser-agent: WebmasterWorld Extractor\nUser-agent: WebmasterWorldForumBot\nUser-agent: WebSauger\nUser-agent: Website Quester\nUser-agent: Webster Pro\nUser-agent: WebStripper\nUser-agent: WebVac\nUser-agent: WebZip\nUser-agent: WebZip\/4.0\nUser-agent: Wget\nUser-agent: Wget\/1.5.3\nUser-agent: Wget\/1.6\nUser-agent: WWW-Collector-E\nUser-agent: Xenu's\nUser-agent: Xenu's Link Sleuth 1.1c\nUser-agent: Zeus\nUser-agent: Zeus 32297 Webster Pro V2.9 Win32\nUser-agent: Zeus Link Scout\nDisallow: \/<\/code><\/pre>\n\n\n\n<p><\/p>\n\n\n\n<p><strong>\u041f\u0440\u0438\u043c\u0435\u0447\u0430\u043d\u0438\u0435<\/strong>. \u041d\u0435\u043a\u043e\u0442\u043e\u0440\u044b\u0435 \u043b\u044e\u0434\u0438 \u0441\u0447\u0438\u0442\u0430\u044e\u0442, \u0447\u0442\u043e \u043a\u043e\u043f\u0438\u0440\u043e\u0432\u0430\u043d\u0438\u0435 \u044d\u0442\u0438\u0445 \u0444\u0430\u0439\u043b\u043e\u0432 \u0441\u043e\u0437\u0434\u0430\u0441\u0442 \u0441\u043b\u0435\u0434\u044b (<em>\u0441\u0432\u044f\u0437\u0438, \u043f\u043e \u043a\u043e\u0442\u043e\u0440\u044b\u043c \u0441\u043c\u043e\u0433\u0443 \u043e\u0442\u0441\u043b\u0435\u0434\u0438\u0442\u044c \u043a\u043e\u043f\u0438\u043f\u0430\u0441\u0442 \u043c\u0435\u0436\u0434\u0443 \u0441\u0430\u0439\u0442\u0430\u043c\u0438 - \u043f\u0440\u0438\u043c\u0435\u0447\u0430\u043d\u0438\u0435 iConsult Agency<\/em>), \u043d\u043e \u044f \u043d\u0435 \u0437\u043d\u0430\u044e, \u0438 \u044f \u043d\u0438\u043a\u043e\u0433\u0434\u0430 \u043d\u0435 \u0441\u043c\u043e\u0433\u0443 \u043f\u043e\u043b\u0443\u0447\u0438\u0442\u044c \u043f\u0440\u044f\u043c\u043e\u0439 \u043e\u0442\u0432\u0435\u0442 \u043d\u0430 \u0432\u043e\u043f\u0440\u043e\u0441, \u043a\u0442\u043e \u043f\u0440\u043e\u0432\u0435\u0440\u044f\u0435\u0442 \u0438 \u0441\u0440\u0430\u0432\u043d\u0438\u0432\u0430\u0435\u0442 \u044d\u0442\u0438 \u0444\u0430\u0439\u043b\u044b robots.txt \u043d\u0430 \u043c\u0438\u043b\u043b\u0438\u0430\u0440\u0434\u0430\u0445 \u0434\u043e\u043c\u0435\u043d\u043e\u0432, \u0438 \u044f \u0434\u043e \u0441\u0438\u0445 \u043f\u043e\u0440 \u043d\u0435 \u043d\u0430\u0448\u0435\u043b \u043d\u0438 \u043e\u0434\u043d\u043e\u0433\u043e \u043f\u0440\u043e\u0433\u0440\u0430\u043c\u043c\u043d\u043e\u0433\u043e \u043e\u0431\u0435\u0441\u043f\u0435\u0447\u0435\u043d\u0438\u044f, \u043a\u043e\u0442\u043e\u0440\u043e\u0435 \u043f\u043e\u0442\u0440\u0443\u0434\u0438\u043b\u043e\u0441\u044c \u0431\u044b \u0441\u0434\u0435\u043b\u0430\u0442\u044c \u044d\u0442\u043e, \u0442\u0430\u043a \u0447\u0442\u043e\u2026 \u00af\\_(\u30c4)_\/\u00af .<\/p>\n\n\n\n<p>\u042f \u0431\u044b \u0431\u043e\u043b\u044c\u0448\u0435 \u0431\u0435\u0441\u043f\u043e\u043a\u043e\u0438\u043b\u0441\u044f \u043e \u0442\u043e\u043c, \u0447\u0442\u043e Google \u0441\u043c\u043e\u0436\u0435\u0442 \u0447\u0438\u0442\u0430\u0442\u044c \u0432\u0430\u0448 gmail, \u043a\u043e\u0433\u0434\u0430 \u0432\u044b \u043f\u0435\u0440\u0435\u043f\u0438\u0441\u044b\u0432\u0430\u0435\u0442\u0435\u0441\u044c \u0442\u0443\u0434\u0430-\u0441\u044e\u0434\u0430 \u0441 \u043f\u0440\u043e\u0432\u0430\u0439\u0434\u0435\u0440\u0430\u043c\u0438 PBN \u0438\u043b\u0438 \u043a\u0430\u043a\u043e\u0439-\u0442\u043e \u043a\u043b\u0438\u0435\u043d\u0442 \u0432\u0430\u0448\u0435\u0433\u043e \u043f\u0440\u043e\u0432\u0430\u0439\u0434\u0435\u0440\u0430 PBN \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u0443\u0435\u0442 gmail \u0438 \u0442\u0435\u043f\u0435\u0440\u044c Google \u043c\u043e\u0436\u0435\u0442 \u0441\u043e\u0435\u0434\u0438\u043d\u0438\u0442\u044c \u0442\u043e\u0447\u043a\u0438.<\/p>\n\n\n\n<p><\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n<h2 class=\"wp-block-heading has-text-color\" id=\"\u0432\u0435\u0440\u0441\u0438\u044f-\u0434\u043b\u044f-httaccess-apache-\" style=\"color:#3cc22b\">\u0412\u0435\u0440\u0441\u0438\u044f \u0434\u043b\u044f .httaccess (Apache) <\/h2>\n\n\n<p>\u041a\u043b\u0430\u0441\u0441\u0438\u0447\u0435\u0441\u043a\u0430\u044f \u0432\u0435\u0440\u0441\u0438\u044f Apache<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>RewriteEngine On\nRewriteCond %{REQUEST_URI} !\/robots.txt$\nRewriteCond %{HTTP_USER_AGENT} ^$ &#91;OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*EventMachine.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*NerdyBot.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Typhoeus.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*archive.org_bot.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*archive.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*adbeat_bot.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*github.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*chroot.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Jorgee.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Go\\ 1.1\\ package.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Go-http-client.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Copyscape.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*semrushbot.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*semrushbot-sa.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*JamesBOT.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*SEOkicks-Robot.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*LinkpadBot.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*getty.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*picscout.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*AppEngine.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Zend_Http_Client.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*BlackWidow.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*openlink.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*spbot.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Nutch.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Jetbot.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*WebVac.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Stanford.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*scooter.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*naver.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*dumbot.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Hatena\\ Antenna.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*grub.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*looksmart.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*WebZip.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*larbin.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*b2w\/0.1.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Copernic.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*psbot.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Python-urllib.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*NetMechanic.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*URL_Spider_Pro.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*CherryPicker.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*ExtractorPro.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*CopyRightCheck.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Crescent.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*CCBot.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*SiteSnagger.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*ProWebWalker.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*CheeseBot.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*LNSpiderguy.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*EmailCollector.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*EmailSiphon.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*WebBandit.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*EmailWolf.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*ia_archiver.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Alexibot.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Teleport.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*MIIxpc.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Telesoft.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Website\\ Quester.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*moget.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*WebStripper.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*WebSauger.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*WebCopier.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*NetAnts.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Mister\\ PiX.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*WebAuto.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*TheNomad.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*WWW-Collector-E.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*RMA.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*libWeb\/clsHTTP.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*asterias.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*httplib.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*turingos.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*spanner.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Harvest.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*InfoNaviRobot.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Bullseye.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*WebBandit.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*NICErsPRO.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Microsoft\\ URL\\ Control.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*DittoSpyder.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Foobot.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*WebmasterWorldForumBot.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*SpankBot.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*BotALot.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*lwp-trivial.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*WebmasterWorld.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*BunnySlippers.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*URLy\\ Warning.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*LinkWalker.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*cosmos.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*hloader.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*humanlinks.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*LinkextractorPro.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Offline\\ Explorer.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Mata\\ Hari.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*LexiBot.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Web\\ Image\\ Collector.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*woobot.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*The\\ Intraformant.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*True_Robot.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*BlowFish.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*SearchEngineWorld.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*JennyBot.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*MIIxpc.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*BuiltBotTough.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*ProPowerBot.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*BackDoorBot.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*toCrawl\/UrlDispatcher.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*WebEnhancer.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*suzuran.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*WebViewer.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*VCI.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Szukacz.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*QueryN.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Openfind.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Openbot.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Webster.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*EroCrawler.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*LinkScan.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Keyword.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Kenjin.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Iron33.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Bookmark\\ search\\ tool.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*GetRight.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*FairAd\\ Client.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Gaisbot.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Aqua_Products.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Radiation\\ Retriever\\ 1.1.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Flaming\\ AttackBot.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Oracle\\ Ultra\\ Search.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*MSIECrawler.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*PerMan.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*searchpreview.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*sootle.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Enterprise_Search.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Bot\\ mailto:craftbot@yahoo.com.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*ChinaClaw.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Custo.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*DISCo.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Download\\ Demon.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*eCatch.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*EirGrabber.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*EmailSiphon.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*EmailWolf.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Express\\ WebPictures.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*ExtractorPro.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*EyeNetIE.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*FlashGet.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*GetRight.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*GetWeb!.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Go!Zilla.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Go-Ahead-Got-It.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*GrabNet.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Grafula.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*HMView.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*HTTrack.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Image\\ Stripper.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Image\\ Sucker.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Indy\\ Library.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*InterGET.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Internet\\ Ninja.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*JetCar.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*JOC\\ Web\\ Spider.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*larbin.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*LeechFTP.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Mass\\ Downloader.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*MIDown\\ tool.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Mister\\ PiX.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Navroad.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*NearSite.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*NetAnts.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*NetSpider.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Net\\ Vampire.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*NetZIP.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Octopus.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Offline\\ Explorer.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Offline\\ Navigator.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*PageGrabber.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Papa\\ Foto.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*pavuk.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*pcBrowser.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*RealDownload.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*ReGet.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*SiteSnagger.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*SmartDownload.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*SuperBot.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*SuperHTTP.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Surfbot.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*tAkeOut.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Teleport\\ Pro.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*VoidEYE.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Web\\ Image\\ Collector.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Web\\ Sucker.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*WebAuto.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*WebCopier.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*WebFetch.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*WebGo\\ IS.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*WebLeacher.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*WebReaper.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*WebSauger.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*wesee.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Website\\ eXtractor.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Website\\ Quester.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*WebStripper.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*WebWhacker.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*WebZIP.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Wget.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Widow.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*WWWOFFLE.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Xaldon\\ WebSpider.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Zeus.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Semrush.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*BecomeBot.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Screaming.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Screaming\\ FrogSEO.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*SEO.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*AhrefsBot.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*AhrefsSiteAudit.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*MJ12bot.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*rogerbot.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*exabot.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Xenu.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*dotbot.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*gigabot.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Twengabot.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*htmlparser.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*libwww.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Python.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*perl.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*urllib.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*scan.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Curl.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*email.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*PycURL.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*Pyth.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*PyQ.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*WebCollector.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*WebCopy.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*webcraw.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*SurveyBot.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*SurveyBot_IgnoreIP.*$ &#91;NC,OR]\nRewriteCond %{HTTP_USER_AGENT} ^.*BlekkoBot.*$ &#91;NC]\nRewriteRule ^.*.* http:\/\/www.google.com\/ &#91;L]<\/code><\/pre>\n\n\n\n<p><\/p>\n\n\n\n<p><\/p>\n\n\n\n<p class=\"has-pale-pink-color has-text-color\"><strong>\u041f\u0440\u0438\u043c\u0435\u0447\u0430\u043d\u0438\u0435<\/strong>. \u041c\u043e\u0439 \u043a\u043e\u0434 \u0440\u0435\u0434\u0438\u0440\u0435\u043a\u0442\u0438\u0442 \u043d\u0430 google.com, \u0447\u0442\u043e\u0431\u044b \u043e\u043d \u0440\u0435\u043d\u0434\u0435\u0440\u0438\u043b \u043f\u0440\u0430\u0432\u0438\u043b\u044c\u043d\u0443\u044e html-\u0441\u0442\u0440\u0430\u043d\u0438\u0446\u0443, \u0430 \u0442\u0430\u043c \u044d\u0442\u043e \u0434\u0435\u043b\u0430\u0435\u0442 \u043f\u043b\u043e\u0445\u043e\u0439 \u0431\u043e\u0442 \u00af\\_(\u30c4)_\/\u00af LOL. \u0412 \u043e\u0447\u0435\u043d\u044c \u0441\u0442\u0430\u0440\u044b\u0445 \u0432\u0435\u0440\u0441\u0438\u044f\u0445 \u0443 \u043c\u0435\u043d\u044f \u0431\u044b\u043b\u043e \u043f\u0435\u0440\u0435\u043d\u0430\u043f\u0440\u0430\u0432\u043b\u0435\u043d\u0438\u0435 \u043d\u0430 webmd.com (\u043f\u043e\u044d\u0442\u043e\u043c\u0443 \u0432\u0441\u0435, \u0447\u0442\u043e \u0441\u0432\u044f\u0437\u0430\u043d\u043e \u0441 webmd.com, \u0438\u0441\u0445\u043e\u0434\u0438\u043b\u043e \u043e\u0442 \u043c\u0435\u043d\u044f), \u043d\u043e \u044d\u0442\u043e \u043f\u043e\u0442\u043e\u043c\u0443, \u0447\u0442\u043e \u0432 \u0442\u043e \u0432\u0440\u0435\u043c\u044f \u044f \u0440\u0430\u0431\u043e\u0442\u0430\u043b \u0432 \u0441\u0444\u0435\u0440\u0435 \u0437\u0434\u0440\u0430\u0432\u043e\u043e\u0445\u0440\u0430\u043d\u0435\u043d\u0438\u044f.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n<h2 class=\"wp-block-heading has-text-color\" id=\"\u0432\u0435\u0440\u0441\u0438\u044f-\u0434\u043b\u044f-nginxconf-nginx\" style=\"color:#3cc22b\">\u0412\u0435\u0440\u0441\u0438\u044f \u0434\u043b\u044f nginx.conf (NGINX)<\/h2>\n\n\n<p><\/p>\n\n\n\n<p>\u0412\u043d\u0443\u0442\u0440\u0438 http \u0431\u043b\u043e\u043a\u0430 (nginx.conf)<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>http {\nmap $http_user_agent $limit_bots {\n     default 0;\n     ~*(adbeat_bot|ahrefsbot|ahrefssiteaudit|alexibot|appengine|aqua_products|archive.org_bot|archive|asterias|attackbot|b2w|backdoorbot|becomebot|blackwidow|blekkobot) 1;\n     ~*(blowfish|botalot|builtbottough|bullseye|bunnyslippers|ccbot|cheesebot|cherrypicker|chinaclaw|chroot|clshttp|collector) 1;\n     ~*(control|copernic|copyrightcheck|copyscape|cosmos|craftbot|crescent|curl|custo|demon) 1;\n     ~*(disco|dittospyder|dotbot|download|downloader|dumbot|ecatch|eirgrabber|email|emailcollector) 1;\n     ~*(emailsiphon|emailwolf|enterprise_search|erocrawler|eventmachine|exabot|express|extractor|extractorpro|eyenetie) 1;\n     ~*(fairad|flaming|flashget|foobot|foto|gaisbot|getright|getty|getweb!|gigabot) 1;\n     ~*(github|go!zilla|go-ahead-got-it|go-http-client|grabnet|grafula|grub|hari|harvest|hatena|antenna|hloader) 1;\n     ~*(hmview|htmlparser|httplib|httrack|humanlinks|ia_archiver|indy|infonavirobot|interget|intraformant) 1;\n     ~*(iron33|jamesbot|jennybot|jetbot|jetcar|joc|jorgee|kenjin|keyword|larbin|leechftp) 1;\n     ~*(lexibot|library|libweb|libwww|linkextractorpro|linkpadbot|linkscan|linkwalker|lnspiderguy|looksmart) 1;\n     ~*(lwp-trivial|mass|mata|midown|miixpc|mister|mj12bot|moget|msiecrawler|naver) 1;\n     ~*(navroad|nearsite|nerdybot|netants|netmechanic|netspider|netzip|nicerspro|ninja|nutch) 1;\n     ~*(octopus|offline|openbot|openfind|openlink|pagegrabber|papa|pavuk|pcbrowser|perl) 1;\n     ~*(perman|picscout|propowerbot|prowebwalker|psbot|pycurl|pyq|pyth|python) 1;\n     ~*(python-urllib|queryn|quester|radiation|realdownload|reget|retriever|rma|rogerbot|scan|screaming|frog|seo) 1;\n     ~*(scooter|searchengineworld|searchpreview|semrush|semrushbot|semrushbot-sa|seokicks-robot|sitesnagger|smartdownload|sootle) 1;\n     ~*(spankbot|spanner|spbot|spider|stanford|stripper|sucker|superbot|superhttp|surfbot|surveybot) 1;\n     ~*(suzuran|szukacz|takeout|teleport|telesoft|thenomad|tocrawl|tool|true_robot|turingos) 1;\n     ~*(twengabot|typhoeus|url_spider_pro|urldispatcher|urllib|urly|vampire|vci|voideye|warning) 1;\n     ~*(webauto|webbandit|webcollector|webcopier|webcopy|webcraw|webenhancer|webfetch|webgo|webleacher) 1;\n     ~*(webmasterworld|webmasterworldforumbot|webpictures|webreaper|websauger|webspider|webster|webstripper|webvac|webviewer) 1;\n     ~*(webwhacker|webzip|webzip|wesee|wget|widow|woobot|www-collector-e|wwwoffle|xenu) 1;\n}\n}<\/code><\/pre>\n\n\n\n<p><\/p>\n\n\n\n<p>\u0412\u043d\u0443\u0442\u0440\u0438 \u043a\u0430\u0436\u0434\u043e\u0433\u043e \u0431\u043b\u043e\u043a\u0430 \u0441\u0435\u0440\u0432\u0435\u0440\u0430 (\u0432 \u0432\u0430\u0448\u0435\u043c \u0444\u0430\u0439\u043b\u0435 example.com \u0432\u043d\u0443\u0442\u0440\u0438 \u043f\u0430\u043f\u043a\u0438, \u0434\u043e\u0441\u0442\u0443\u043f\u043d\u043e\u0439 \u0434\u043b\u044f \u0441\u0430\u0439\u0442\u043e\u0432)<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>server {\nlocation \/ {\n#blocks blank user_agents\nif ($http_user_agent = \"\") { return  301 $scheme:\/\/www.google.com\/; }\n\n  if ($limit_bots = 1) {\n  return  301 $scheme:\/\/www.google.com\/;\n  }\n}\n}<\/code><\/pre>\n\n\n\n<p><\/p>\n\n\n\n<p>\u0415\u0441\u043b\u0438 \u0431\u0443\u0434\u0443\u0442 \u0432\u043e\u043f\u0440\u043e\u0441\u044b, \u0433\u043b\u044f\u043d\u044c\u0442\u0435 <a href=\"https:\/\/www.buildersociety.com\/threads\/block-unwanted-bots-on-apache-nginx-constantly-updated.1898\/\" target=\"_blank\" rel=\"noreferrer noopener nofollow\">\u043e\u0440\u0438\u0433\u0438\u043d\u0430\u043b\u044c\u043d\u0443\u044e \u0432\u0435\u0442\u043a\u0443 \u0444\u043e\u0440\u0443\u043c\u0430<\/a>. \u041e\u043d\u0430 \u0440\u0435\u0433\u0443\u043b\u044f\u0440\u043d\u043e \u043e\u0431\u043d\u043e\u0432\u043b\u044f\u0435\u0442\u0441\u044f.<\/p>\n\n\n\n<p><\/p>\n","protected":false},"excerpt":{"rendered":"<p>\u042d\u0442\u043e \u043f\u0435\u0440\u0435\u0432\u043e\u0434 \u0441\u0442\u0430\u0442\u044c\u0438 \u0441 Builder Society. \u0421\u0441\u044b\u043b\u043a\u0430 \u043d\u0430 \u0438\u0441\u0442\u043e\u0447\u043d\u0438\u043a \u0432 \u043a\u043e\u043d\u0446\u0435. \u0427\u0442\u043e\u0431\u044b \u043d\u0435 \u043e\u0442\u043f\u0440\u0430\u0432\u043b\u044f\u0442\u044c \u043b\u044e\u0434\u0435\u0439 \u043d\u0430 \u043c\u043e\u0439 \u0441\u0430\u0439\u0442 moneyoverethics.com \u0437\u0430 \u043f\u043e\u0441\u043b\u0435\u0434\u043d\u0438\u043c\u0438 \u0432\u0435\u0440\u0441\u0438\u044f\u043c\u0438 \u0440\u043e\u0431\u043e\u0442\u043e\u0432, \u044f \u0440\u0435\u0448\u0438\u043b \u043f\u0440\u043e\u0441\u0442\u043e \u0441\u043e\u0437\u0434\u0430\u0442\u044c \u0440\u0430\u0431\u043e\u0442\u0430\u044e\u0449\u0443\u044e\/\u043e\u0431\u043d\u043e\u0432\u043b\u044f\u0435\u043c\u0443\u044e \u0432\u0435\u0440\u0441\u0438\u044e \u0437\u0434\u0435\u0441\u044c. \u042f \u0440\u0435\u043a\u043e\u043c\u0435\u043d\u0434\u0443\u044e \u0438\u0441\u043f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u0442\u044c \u0441\u043b\u0435\u0434\u0443\u044e\u0449\u0438\u0435 \u0434\u043e\u043f\u043e\u043b\u043d\u0435\u043d\u0438\u044f \u0434\u043b\u044f \u0431\u0440\u0430\u0443\u0437\u0435\u0440\u0430, \u0447\u0442\u043e\u0431\u044b \u043f\u0440\u043e\u0432\u0435\u0440\u0438\u0442\u044c, \u0434\u0435\u0439\u0441\u0442\u0432\u0438\u0442\u0435\u043b\u044c\u043d\u043e \u043b\u0438 \u0432\u044b \u0431\u043b\u043e\u043a\u0438\u0440\u0443\u0435\u0442\u0435 \u0442\u0435\u0445 \u0431\u043e\u0442\u043e\u0432, \u043a\u043e\u0442\u043e\u0440\u044b\u0445 \u0445\u043e\u0442\u0438\u0442\u0435 \u0437\u0430\u0431\u043b\u043e\u043a\u0438\u0440\u043e\u0432\u0430\u0442\u044c. \ud83d\udc47<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[2,5],"tags":[18,16],"class_list":["post-1301","post","type-post","status-publish","format-standard","hentry","category-seo","category-instruktsii","tag-18","tag-16"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v23.0 - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>\u041a\u0430\u043a \u0437\u0430\u0431\u043b\u043e\u043a\u0438\u0440\u043e\u0432\u0430\u0442\u044c \u043d\u0435\u0436\u0435\u043b\u0430\u0442\u0435\u043b\u044c\u043d\u044b\u0445 \u0440\u043e\u0431\u043e\u0442\u043e\u0432 \u0432 Apache \u0438 Nginx, \u0430 \u0442\u0430\u043a\u0436\u0435 Robots.txt - \u0411\u043b\u043e\u0433 \u043e\u0431 \u0438\u043d\u0442\u0435\u0440\u043d\u0435\u0442-\u043c\u0430\u0440\u043a\u0435\u0442\u0438\u043d\u0433\u0435 - iConsult Agency<\/title>\n<meta name=\"description\" content=\"\u0413\u043e\u0442\u043e\u0432\u044b\u0435 \u0441\u043f\u0438\u0441\u043a\u0438 \u0432\u0441\u0435\u0445 \u0431\u043e\u0442\u043e\u0432 \u043f\u043e\u0438\u0441\u043a\u043e\u0432\u044b\u0445 \u0441\u0438\u0441\u0442\u0435\u043c \u0438 SEO \u0441\u0435\u0440\u0432\u0438\u0441\u043e\u0432 \u0434\u043b\u044f \u0431\u043b\u043e\u043a\u0438\u0440\u043e\u0432\u043a\u0438 \u0432 Apache \u0438 Nginx.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/iconsult.agency\/blog\/kak-zablokirovat-nezhelatelnikh-robotov-v-apache-i-nginx\/\" \/>\n<meta property=\"og:locale\" content=\"ru_RU\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"\u041a\u0430\u043a \u0437\u0430\u0431\u043b\u043e\u043a\u0438\u0440\u043e\u0432\u0430\u0442\u044c \u043d\u0435\u0436\u0435\u043b\u0430\u0442\u0435\u043b\u044c\u043d\u044b\u0445 \u0440\u043e\u0431\u043e\u0442\u043e\u0432 \u0432 Apache \u0438 Nginx, \u0430 \u0442\u0430\u043a\u0436\u0435 Robots.txt - \u0411\u043b\u043e\u0433 \u043e\u0431 \u0438\u043d\u0442\u0435\u0440\u043d\u0435\u0442-\u043c\u0430\u0440\u043a\u0435\u0442\u0438\u043d\u0433\u0435 - iConsult Agency\" \/>\n<meta property=\"og:description\" content=\"\u0413\u043e\u0442\u043e\u0432\u044b\u0435 \u0441\u043f\u0438\u0441\u043a\u0438 \u0432\u0441\u0435\u0445 \u0431\u043e\u0442\u043e\u0432 \u043f\u043e\u0438\u0441\u043a\u043e\u0432\u044b\u0445 \u0441\u0438\u0441\u0442\u0435\u043c \u0438 SEO \u0441\u0435\u0440\u0432\u0438\u0441\u043e\u0432 \u0434\u043b\u044f \u0431\u043b\u043e\u043a\u0438\u0440\u043e\u0432\u043a\u0438 \u0432 Apache \u0438 Nginx.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/iconsult.agency\/blog\/kak-zablokirovat-nezhelatelnikh-robotov-v-apache-i-nginx\/\" \/>\n<meta property=\"og:site_name\" content=\"\u0411\u043b\u043e\u0433 \u043e\u0431 \u0438\u043d\u0442\u0435\u0440\u043d\u0435\u0442-\u043c\u0430\u0440\u043a\u0435\u0442\u0438\u043d\u0433\u0435 - iConsult Agency\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/iConsult.Agency\/\" \/>\n<meta property=\"article:published_time\" content=\"2023-05-31T15:07:42+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2024-12-13T11:34:39+00:00\" \/>\n<meta name=\"author\" content=\"Konstantin Ryabenko\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@kostyll\" \/>\n<meta name=\"twitter:label1\" content=\"\u041d\u0430\u043f\u0438\u0441\u0430\u043d\u043e \u0430\u0432\u0442\u043e\u0440\u043e\u043c\" \/>\n\t<meta name=\"twitter:data1\" content=\"Konstantin Ryabenko\" \/>\n\t<meta name=\"twitter:label2\" content=\"\u041f\u0440\u0438\u043c\u0435\u0440\u043d\u043e\u0435 \u0432\u0440\u0435\u043c\u044f \u0434\u043b\u044f \u0447\u0442\u0435\u043d\u0438\u044f\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 \u043c\u0438\u043d\u0443\u0442\u044b\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/iconsult.agency\/blog\/kak-zablokirovat-nezhelatelnikh-robotov-v-apache-i-nginx\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/iconsult.agency\/blog\/kak-zablokirovat-nezhelatelnikh-robotov-v-apache-i-nginx\/\"},\"author\":{\"name\":\"Konstantin Ryabenko\",\"@id\":\"https:\/\/iconsult.agency\/blog\/#\/schema\/person\/3b3a7f9bd1b5234273ea31683fb4b33f\"},\"headline\":\"\u041a\u0430\u043a \u0437\u0430\u0431\u043b\u043e\u043a\u0438\u0440\u043e\u0432\u0430\u0442\u044c \u043d\u0435\u0436\u0435\u043b\u0430\u0442\u0435\u043b\u044c\u043d\u044b\u0445 \u0440\u043e\u0431\u043e\u0442\u043e\u0432 \u0432 Apache \u0438 Nginx, \u0430 \u0442\u0430\u043a\u0436\u0435 Robots.txt\",\"datePublished\":\"2023-05-31T15:07:42+00:00\",\"dateModified\":\"2024-12-13T11:34:39+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/iconsult.agency\/blog\/kak-zablokirovat-nezhelatelnikh-robotov-v-apache-i-nginx\/\"},\"wordCount\":420,\"commentCount\":20,\"publisher\":{\"@id\":\"https:\/\/iconsult.agency\/blog\/#organization\"},\"keywords\":[\"\u0440\u043e\u0431\u043e\u0442\u044b\",\"\u0442\u0435\u0445\u043d\u0438\u0447\u0435\u0441\u043a\u0430\u044f \u043e\u043f\u0442\u0438\u043c\u0438\u0437\u0430\u0446\u0438\u044f\"],\"articleSection\":[\"SEO\",\"\u0418\u043d\u0441\u0442\u0440\u0443\u043a\u0446\u0438\u0438\"],\"inLanguage\":\"ru-RU\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\/\/iconsult.agency\/blog\/kak-zablokirovat-nezhelatelnikh-robotov-v-apache-i-nginx\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/iconsult.agency\/blog\/kak-zablokirovat-nezhelatelnikh-robotov-v-apache-i-nginx\/\",\"url\":\"https:\/\/iconsult.agency\/blog\/kak-zablokirovat-nezhelatelnikh-robotov-v-apache-i-nginx\/\",\"name\":\"\u041a\u0430\u043a \u0437\u0430\u0431\u043b\u043e\u043a\u0438\u0440\u043e\u0432\u0430\u0442\u044c \u043d\u0435\u0436\u0435\u043b\u0430\u0442\u0435\u043b\u044c\u043d\u044b\u0445 \u0440\u043e\u0431\u043e\u0442\u043e\u0432 \u0432 Apache \u0438 Nginx, \u0430 \u0442\u0430\u043a\u0436\u0435 Robots.txt - \u0411\u043b\u043e\u0433 \u043e\u0431 \u0438\u043d\u0442\u0435\u0440\u043d\u0435\u0442-\u043c\u0430\u0440\u043a\u0435\u0442\u0438\u043d\u0433\u0435 - iConsult Agency\",\"isPartOf\":{\"@id\":\"https:\/\/iconsult.agency\/blog\/#website\"},\"datePublished\":\"2023-05-31T15:07:42+00:00\",\"dateModified\":\"2024-12-13T11:34:39+00:00\",\"description\":\"\u0413\u043e\u0442\u043e\u0432\u044b\u0435 \u0441\u043f\u0438\u0441\u043a\u0438 \u0432\u0441\u0435\u0445 \u0431\u043e\u0442\u043e\u0432 \u043f\u043e\u0438\u0441\u043a\u043e\u0432\u044b\u0445 \u0441\u0438\u0441\u0442\u0435\u043c \u0438 SEO \u0441\u0435\u0440\u0432\u0438\u0441\u043e\u0432 \u0434\u043b\u044f \u0431\u043b\u043e\u043a\u0438\u0440\u043e\u0432\u043a\u0438 \u0432 Apache \u0438 Nginx.\",\"breadcrumb\":{\"@id\":\"https:\/\/iconsult.agency\/blog\/kak-zablokirovat-nezhelatelnikh-robotov-v-apache-i-nginx\/#breadcrumb\"},\"inLanguage\":\"ru-RU\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/iconsult.agency\/blog\/kak-zablokirovat-nezhelatelnikh-robotov-v-apache-i-nginx\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/iconsult.agency\/blog\/kak-zablokirovat-nezhelatelnikh-robotov-v-apache-i-nginx\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"\u0413\u043b\u0430\u0432\u043d\u0430\u044f \u0441\u0442\u0440\u0430\u043d\u0438\u0446\u0430\",\"item\":\"https:\/\/iconsult.agency\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"\u041a\u0430\u043a \u0437\u0430\u0431\u043b\u043e\u043a\u0438\u0440\u043e\u0432\u0430\u0442\u044c \u043d\u0435\u0436\u0435\u043b\u0430\u0442\u0435\u043b\u044c\u043d\u044b\u0445 \u0440\u043e\u0431\u043e\u0442\u043e\u0432 \u0432 Apache \u0438 Nginx, \u0430 \u0442\u0430\u043a\u0436\u0435 Robots.txt\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/iconsult.agency\/blog\/#website\",\"url\":\"https:\/\/iconsult.agency\/blog\/\",\"name\":\"\u0411\u043b\u043e\u0433 \u043e\u0431 \u0438\u043d\u0442\u0435\u0440\u043d\u0435\u0442-\u043c\u0430\u0440\u043a\u0435\u0442\u0438\u043d\u0433\u0435 - iConsult Agency\",\"description\":\"\u0411\u043b\u043e\u0433 \u043e \u043f\u0440\u043e\u0434\u0432\u0438\u0436\u0435\u043d\u0438\u0438 \u0441\u0430\u0439\u0442\u043e\u0432 (SEO) \u0438 \u0432\u0435\u0431-\u0430\u043d\u0430\u043b\u0438\u0442\u0438\u043a\u0435\",\"publisher\":{\"@id\":\"https:\/\/iconsult.agency\/blog\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/iconsult.agency\/blog\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"ru-RU\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/iconsult.agency\/blog\/#organization\",\"name\":\"iConsult Agency\",\"url\":\"https:\/\/iconsult.agency\/blog\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"ru-RU\",\"@id\":\"https:\/\/iconsult.agency\/blog\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/iconsult.agency\/blog\/wp-content\/uploads\/2020\/03\/logo_dark.png\",\"contentUrl\":\"https:\/\/iconsult.agency\/blog\/wp-content\/uploads\/2020\/03\/logo_dark.png\",\"width\":100,\"height\":62,\"caption\":\"iConsult Agency\"},\"image\":{\"@id\":\"https:\/\/iconsult.agency\/blog\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/iConsult.Agency\/\",\"https:\/\/www.instagram.com\/_iconsult_\/\",\"https:\/\/www.linkedin.com\/company\/iconsult-agency\/\",\"https:\/\/www.youtube.com\/channel\/UCZeb3oR03SIY5uJc0EJ6guA\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/iconsult.agency\/blog\/#\/schema\/person\/3b3a7f9bd1b5234273ea31683fb4b33f\",\"name\":\"Konstantin Ryabenko\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"ru-RU\",\"@id\":\"https:\/\/iconsult.agency\/blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/iconsult.agency\/blog\/wp-content\/uploads\/2020\/03\/ceo.png\",\"contentUrl\":\"https:\/\/iconsult.agency\/blog\/wp-content\/uploads\/2020\/03\/ceo.png\",\"caption\":\"Konstantin Ryabenko\"},\"sameAs\":[\"https:\/\/www.linkedin.com\/in\/ryabenko\/\",\"https:\/\/x.com\/kostyll\"],\"url\":\"https:\/\/iconsult.agency\/blog\/author\/admin\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"\u041a\u0430\u043a \u0437\u0430\u0431\u043b\u043e\u043a\u0438\u0440\u043e\u0432\u0430\u0442\u044c \u043d\u0435\u0436\u0435\u043b\u0430\u0442\u0435\u043b\u044c\u043d\u044b\u0445 \u0440\u043e\u0431\u043e\u0442\u043e\u0432 \u0432 Apache \u0438 Nginx, \u0430 \u0442\u0430\u043a\u0436\u0435 Robots.txt - \u0411\u043b\u043e\u0433 \u043e\u0431 \u0438\u043d\u0442\u0435\u0440\u043d\u0435\u0442-\u043c\u0430\u0440\u043a\u0435\u0442\u0438\u043d\u0433\u0435 - iConsult Agency","description":"\u0413\u043e\u0442\u043e\u0432\u044b\u0435 \u0441\u043f\u0438\u0441\u043a\u0438 \u0432\u0441\u0435\u0445 \u0431\u043e\u0442\u043e\u0432 \u043f\u043e\u0438\u0441\u043a\u043e\u0432\u044b\u0445 \u0441\u0438\u0441\u0442\u0435\u043c \u0438 SEO \u0441\u0435\u0440\u0432\u0438\u0441\u043e\u0432 \u0434\u043b\u044f \u0431\u043b\u043e\u043a\u0438\u0440\u043e\u0432\u043a\u0438 \u0432 Apache \u0438 Nginx.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/iconsult.agency\/blog\/kak-zablokirovat-nezhelatelnikh-robotov-v-apache-i-nginx\/","og_locale":"ru_RU","og_type":"article","og_title":"\u041a\u0430\u043a \u0437\u0430\u0431\u043b\u043e\u043a\u0438\u0440\u043e\u0432\u0430\u0442\u044c \u043d\u0435\u0436\u0435\u043b\u0430\u0442\u0435\u043b\u044c\u043d\u044b\u0445 \u0440\u043e\u0431\u043e\u0442\u043e\u0432 \u0432 Apache \u0438 Nginx, \u0430 \u0442\u0430\u043a\u0436\u0435 Robots.txt - \u0411\u043b\u043e\u0433 \u043e\u0431 \u0438\u043d\u0442\u0435\u0440\u043d\u0435\u0442-\u043c\u0430\u0440\u043a\u0435\u0442\u0438\u043d\u0433\u0435 - iConsult Agency","og_description":"\u0413\u043e\u0442\u043e\u0432\u044b\u0435 \u0441\u043f\u0438\u0441\u043a\u0438 \u0432\u0441\u0435\u0445 \u0431\u043e\u0442\u043e\u0432 \u043f\u043e\u0438\u0441\u043a\u043e\u0432\u044b\u0445 \u0441\u0438\u0441\u0442\u0435\u043c \u0438 SEO \u0441\u0435\u0440\u0432\u0438\u0441\u043e\u0432 \u0434\u043b\u044f \u0431\u043b\u043e\u043a\u0438\u0440\u043e\u0432\u043a\u0438 \u0432 Apache \u0438 Nginx.","og_url":"https:\/\/iconsult.agency\/blog\/kak-zablokirovat-nezhelatelnikh-robotov-v-apache-i-nginx\/","og_site_name":"\u0411\u043b\u043e\u0433 \u043e\u0431 \u0438\u043d\u0442\u0435\u0440\u043d\u0435\u0442-\u043c\u0430\u0440\u043a\u0435\u0442\u0438\u043d\u0433\u0435 - iConsult Agency","article_publisher":"https:\/\/www.facebook.com\/iConsult.Agency\/","article_published_time":"2023-05-31T15:07:42+00:00","article_modified_time":"2024-12-13T11:34:39+00:00","author":"Konstantin Ryabenko","twitter_card":"summary_large_image","twitter_creator":"@kostyll","twitter_misc":{"\u041d\u0430\u043f\u0438\u0441\u0430\u043d\u043e \u0430\u0432\u0442\u043e\u0440\u043e\u043c":"Konstantin Ryabenko","\u041f\u0440\u0438\u043c\u0435\u0440\u043d\u043e\u0435 \u0432\u0440\u0435\u043c\u044f \u0434\u043b\u044f \u0447\u0442\u0435\u043d\u0438\u044f":"3 \u043c\u0438\u043d\u0443\u0442\u044b"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/iconsult.agency\/blog\/kak-zablokirovat-nezhelatelnikh-robotov-v-apache-i-nginx\/#article","isPartOf":{"@id":"https:\/\/iconsult.agency\/blog\/kak-zablokirovat-nezhelatelnikh-robotov-v-apache-i-nginx\/"},"author":{"name":"Konstantin Ryabenko","@id":"https:\/\/iconsult.agency\/blog\/#\/schema\/person\/3b3a7f9bd1b5234273ea31683fb4b33f"},"headline":"\u041a\u0430\u043a \u0437\u0430\u0431\u043b\u043e\u043a\u0438\u0440\u043e\u0432\u0430\u0442\u044c \u043d\u0435\u0436\u0435\u043b\u0430\u0442\u0435\u043b\u044c\u043d\u044b\u0445 \u0440\u043e\u0431\u043e\u0442\u043e\u0432 \u0432 Apache \u0438 Nginx, \u0430 \u0442\u0430\u043a\u0436\u0435 Robots.txt","datePublished":"2023-05-31T15:07:42+00:00","dateModified":"2024-12-13T11:34:39+00:00","mainEntityOfPage":{"@id":"https:\/\/iconsult.agency\/blog\/kak-zablokirovat-nezhelatelnikh-robotov-v-apache-i-nginx\/"},"wordCount":420,"commentCount":20,"publisher":{"@id":"https:\/\/iconsult.agency\/blog\/#organization"},"keywords":["\u0440\u043e\u0431\u043e\u0442\u044b","\u0442\u0435\u0445\u043d\u0438\u0447\u0435\u0441\u043a\u0430\u044f \u043e\u043f\u0442\u0438\u043c\u0438\u0437\u0430\u0446\u0438\u044f"],"articleSection":["SEO","\u0418\u043d\u0441\u0442\u0440\u0443\u043a\u0446\u0438\u0438"],"inLanguage":"ru-RU","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/iconsult.agency\/blog\/kak-zablokirovat-nezhelatelnikh-robotov-v-apache-i-nginx\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/iconsult.agency\/blog\/kak-zablokirovat-nezhelatelnikh-robotov-v-apache-i-nginx\/","url":"https:\/\/iconsult.agency\/blog\/kak-zablokirovat-nezhelatelnikh-robotov-v-apache-i-nginx\/","name":"\u041a\u0430\u043a \u0437\u0430\u0431\u043b\u043e\u043a\u0438\u0440\u043e\u0432\u0430\u0442\u044c \u043d\u0435\u0436\u0435\u043b\u0430\u0442\u0435\u043b\u044c\u043d\u044b\u0445 \u0440\u043e\u0431\u043e\u0442\u043e\u0432 \u0432 Apache \u0438 Nginx, \u0430 \u0442\u0430\u043a\u0436\u0435 Robots.txt - \u0411\u043b\u043e\u0433 \u043e\u0431 \u0438\u043d\u0442\u0435\u0440\u043d\u0435\u0442-\u043c\u0430\u0440\u043a\u0435\u0442\u0438\u043d\u0433\u0435 - iConsult Agency","isPartOf":{"@id":"https:\/\/iconsult.agency\/blog\/#website"},"datePublished":"2023-05-31T15:07:42+00:00","dateModified":"2024-12-13T11:34:39+00:00","description":"\u0413\u043e\u0442\u043e\u0432\u044b\u0435 \u0441\u043f\u0438\u0441\u043a\u0438 \u0432\u0441\u0435\u0445 \u0431\u043e\u0442\u043e\u0432 \u043f\u043e\u0438\u0441\u043a\u043e\u0432\u044b\u0445 \u0441\u0438\u0441\u0442\u0435\u043c \u0438 SEO \u0441\u0435\u0440\u0432\u0438\u0441\u043e\u0432 \u0434\u043b\u044f \u0431\u043b\u043e\u043a\u0438\u0440\u043e\u0432\u043a\u0438 \u0432 Apache \u0438 Nginx.","breadcrumb":{"@id":"https:\/\/iconsult.agency\/blog\/kak-zablokirovat-nezhelatelnikh-robotov-v-apache-i-nginx\/#breadcrumb"},"inLanguage":"ru-RU","potentialAction":[{"@type":"ReadAction","target":["https:\/\/iconsult.agency\/blog\/kak-zablokirovat-nezhelatelnikh-robotov-v-apache-i-nginx\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/iconsult.agency\/blog\/kak-zablokirovat-nezhelatelnikh-robotov-v-apache-i-nginx\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"\u0413\u043b\u0430\u0432\u043d\u0430\u044f \u0441\u0442\u0440\u0430\u043d\u0438\u0446\u0430","item":"https:\/\/iconsult.agency\/blog\/"},{"@type":"ListItem","position":2,"name":"\u041a\u0430\u043a \u0437\u0430\u0431\u043b\u043e\u043a\u0438\u0440\u043e\u0432\u0430\u0442\u044c \u043d\u0435\u0436\u0435\u043b\u0430\u0442\u0435\u043b\u044c\u043d\u044b\u0445 \u0440\u043e\u0431\u043e\u0442\u043e\u0432 \u0432 Apache \u0438 Nginx, \u0430 \u0442\u0430\u043a\u0436\u0435 Robots.txt"}]},{"@type":"WebSite","@id":"https:\/\/iconsult.agency\/blog\/#website","url":"https:\/\/iconsult.agency\/blog\/","name":"\u0411\u043b\u043e\u0433 \u043e\u0431 \u0438\u043d\u0442\u0435\u0440\u043d\u0435\u0442-\u043c\u0430\u0440\u043a\u0435\u0442\u0438\u043d\u0433\u0435 - iConsult Agency","description":"\u0411\u043b\u043e\u0433 \u043e \u043f\u0440\u043e\u0434\u0432\u0438\u0436\u0435\u043d\u0438\u0438 \u0441\u0430\u0439\u0442\u043e\u0432 (SEO) \u0438 \u0432\u0435\u0431-\u0430\u043d\u0430\u043b\u0438\u0442\u0438\u043a\u0435","publisher":{"@id":"https:\/\/iconsult.agency\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/iconsult.agency\/blog\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"ru-RU"},{"@type":"Organization","@id":"https:\/\/iconsult.agency\/blog\/#organization","name":"iConsult Agency","url":"https:\/\/iconsult.agency\/blog\/","logo":{"@type":"ImageObject","inLanguage":"ru-RU","@id":"https:\/\/iconsult.agency\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/iconsult.agency\/blog\/wp-content\/uploads\/2020\/03\/logo_dark.png","contentUrl":"https:\/\/iconsult.agency\/blog\/wp-content\/uploads\/2020\/03\/logo_dark.png","width":100,"height":62,"caption":"iConsult Agency"},"image":{"@id":"https:\/\/iconsult.agency\/blog\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/iConsult.Agency\/","https:\/\/www.instagram.com\/_iconsult_\/","https:\/\/www.linkedin.com\/company\/iconsult-agency\/","https:\/\/www.youtube.com\/channel\/UCZeb3oR03SIY5uJc0EJ6guA"]},{"@type":"Person","@id":"https:\/\/iconsult.agency\/blog\/#\/schema\/person\/3b3a7f9bd1b5234273ea31683fb4b33f","name":"Konstantin Ryabenko","image":{"@type":"ImageObject","inLanguage":"ru-RU","@id":"https:\/\/iconsult.agency\/blog\/#\/schema\/person\/image\/","url":"https:\/\/iconsult.agency\/blog\/wp-content\/uploads\/2020\/03\/ceo.png","contentUrl":"https:\/\/iconsult.agency\/blog\/wp-content\/uploads\/2020\/03\/ceo.png","caption":"Konstantin Ryabenko"},"sameAs":["https:\/\/www.linkedin.com\/in\/ryabenko\/","https:\/\/x.com\/kostyll"],"url":"https:\/\/iconsult.agency\/blog\/author\/admin\/"}]}},"lang":"ru","translations":{"ru":1301,"uk":2193,"en":3624},"pll_sync_post":[],"_links":{"self":[{"href":"https:\/\/iconsult.agency\/blog\/wp-json\/wp\/v2\/posts\/1301"}],"collection":[{"href":"https:\/\/iconsult.agency\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/iconsult.agency\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/iconsult.agency\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/iconsult.agency\/blog\/wp-json\/wp\/v2\/comments?post=1301"}],"version-history":[{"count":10,"href":"https:\/\/iconsult.agency\/blog\/wp-json\/wp\/v2\/posts\/1301\/revisions"}],"predecessor-version":[{"id":2838,"href":"https:\/\/iconsult.agency\/blog\/wp-json\/wp\/v2\/posts\/1301\/revisions\/2838"}],"wp:attachment":[{"href":"https:\/\/iconsult.agency\/blog\/wp-json\/wp\/v2\/media?parent=1301"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/iconsult.agency\/blog\/wp-json\/wp\/v2\/categories?post=1301"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/iconsult.agency\/blog\/wp-json\/wp\/v2\/tags?post=1301"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}