Tagged: 

Viewing 5 posts - 1 through 5 (of 5 total)
  • Author
    Posts
  • April 27, 2015 at 10:47 am #435135
    Ogau

    Hey,
    After reading a lot of posts on this support page, I haven’t founded an answer about how to optimize a robot.txt file with Enfold. Generally, such a robot.txt file is structured like this (based on different articles founded on the web) :
    User-agent: *
    Disallow: /*?
    Disallow: /wp-login.php
    Disallow: */trackback
    Disallow: /*/feed
    Disallow: /*/comments
    Disallow: /cgi-bin
    Disallow: /*.php$
    Disallow: /*.inc$
    Disallow: /*.gz
    Disallow: /*.cgi
    Allow: /*css?*
    Allow: /*js?*

    I think it could be nice for Enfold. But naturally, not sure. Especially, it seems that it forbidden to recognize the website as responsive.
    Perhaps do you have some good advices on this.
    Thanks for help and have a nice day.

    April 27, 2015 at 11:02 am #435143
    Ogau

    Sometime, we can find such a file : What do you think about it to optimize Enfold ?
    User-agent: *
    # On empêche l’indexation des dossiers sensibles
    Disallow: /cgi-bin
    Disallow: /wp-login.php
    Disallow: /wp-admin
    Disallow: /wp-includes
    Disallow: /wp-content/plugins
    Disallow: /wp-content/cache
    Disallow: /wp-content/themes
    Disallow: /category/*/*
    Disallow: */trackback
    Disallow: */feed
    Disallow: */comments
    Disallow: /*?
    # On empêche l’indexation des fichiers sensibles
    User-agent: Googlebot
    Disallow: /*.php$
    Disallow: /*.js$
    Disallow: /*.inc$
    Disallow: /*.css$
    Disallow: /*.gz$
    Disallow: /*.swf$
    Disallow: /*.wmv$
    Disallow: /*.cgi$
    Disallow: /*.xhtml$
    # Autoriser Google Image
    User-agent: Googlebot-Image
    Disallow:
    Allow: /*
    # Autoriser Google AdSense
    User-agent: Mediapartners-Google*
    Disallow:
    Allow: /*
    # On indique au spider le lien vers notre sitemap
    Sitemap: http://www.website.com/sitemap.xml

    April 28, 2015 at 4:38 pm #436169
    Elliott

    Hi!

    What are you trying to accomplish with your robots.txt file? If your wanting to optimize your site then the best way would be to install a caching plugin. A robots.txt is for blocking access to certain areas.

    Best regards,
    Elliott

    • This reply was modified 9 years, 8 months ago by Elliott.
    April 28, 2015 at 6:15 pm #436264
    Ogau

    Hi,

    Thanks for your help Elliott. I’ve already a caching plugin. Its just to know witch files to disallow without causing problems. For example, if I use a slider, or a java script, I dont know if I can disallow /.js. I have read a comment on Enfold’s support that explain that it could cause some problems with responsive design too. Perhaps I’ve badly understand.

    Thanks again. Best regards.
    Ogau

    April 29, 2015 at 5:50 pm #436890
    Elliott

    Hi!

    Your supposed to use it for preventing the search engine crawlers to index certain areas of your site. Like if you did not want certain pages to display in the search engines or preventing the archives from loading.

    I don’t think it’s going to matter performance wise if you disable the javascript. It might actually hurt your SEO though if the crawler needs the JS to parse your page. I’m not too sure about it. I would try asking on an SEO forum or the google webmaster forums.

    Cheers!
    Elliott

    • This reply was modified 9 years, 8 months ago by Elliott.
  • Author
    Posts
Viewing 5 posts - 1 through 5 (of 5 total)
  • You must be logged in to reply to this topic.