如题,谢谢!我的首页搜索了几年了,写了robots.txt规则,如下所示
- User-agent: *
- Disallow: /
复制代码
其次nginx.conf规则也写了如下规则
- if ($http_user_agent ~* qihoobot|Baiduspider|Googlebot|Googlebot-Mobile|Googlebot-Image|Mediapartners-Google|Adsbot-Google|Feedfetcher-Google|Yahoo! Slurp|Yahoo! Slurp China|YoudaoBot|Sosospider|Sogou spider|Sogou web spider|MSNBot|ia_archiver|Tomato Bot|360Spider)
- {
- return 403;
- }
复制代码 为啥360还搜录了首页呢?请问还需要做什么呢,才可以不搜索任何页面?
|
|
|
|
评论
直达楼层