brainchime.com Latest Questions

Admin
Asked: 2024-10-06T20:45:15+05:30 In:

How do I disallow specific page from robots.txt

I am getting the Soft 404 error on Google Search Console which means when a URL returns a 200 (success) status code along with a page informing the user that the page does not exist. It means that it returns a user-friendly message Sorry, No Results found but not a 404 HTTP response code. This page is an error page based on its content, therefore Google Search Console will show a soft 404 error in the site’s page indexing report. These pages are not indexed or served on Google. Suppose, The page is https://example.com/?s={search_term_string} and I do not want the Google bot to even discover it when it crawls on website pages through a submitted sitemap. How can I disallow it in the robots.txt file so that the Google bot ignores it during the next crawl? Thanks

robots.txt

Related Questions

Leave an answer

Leave an answer

Browse
Browse

Choose from here the video type.

Put Video ID here: https://www.youtube.com/watch?v=sdUUx5FdySs Ex: "sdUUx5FdySs".

1 Answer

  1. Admin

    You can edit your robots.txt file in WordPress by following the below steps:

    1. Install the FileSter plugin.
    2. Once the FileSter plugin is installed and the File Manager option is created in the WordPress dashboard menu. Click on it to open the FileSter plugin.
    3. This will show you the root folder files like .Htaccess, robots.txt, etc.
    4. Double-click the robots.txt file and open it with textarea.
    5. Now edit the robots.txt file and add the following code to the robots.txt file and save it:[Disallow: /?s=*]
      User-agent: *
      Disallow:/wp-admin/
      Disallow:/wp-includes/
      Disallow: /*?show
      Disallow: /*search?search
      Disallow: /?s=*
    6. Now, visit the Google Search console pages section, go to the soft 404 error, and start validation again. This time this page will be ignored.

    Hi, You can also refer to Google’s robot.txt file at any time to understand how this file works.