WooCommerce::robots_txt │ public │ WC 1.0

Tell bots not to index some WooCommerce-created directories.

We try to detect the default "User-agent: *" added by WordPress and add our rules to that group, because it's possible that some bots will only interpret the first group of rules if there are multiple groups with the same user agent.

Method of the class: WooCommerce{}

No Hooks.

Returns

String.

Usage

$WooCommerce = new WooCommerce();
$WooCommerce->robots_txt( $output );
$output(string) (required)
The contents that WordPress will output in a robots.txt file.

WooCommerce::robots_txt() code WC 10.3.6

woocommerce/includes/class-woocommerce.php 
public function robots_txt( $output ) {
	$path = ( ! empty( $site_url['path'] ) ) ? $site_url['path'] : '';

	$lines       = preg_split( '/\r\n|\r|\n/', $output );
	$agent_index = array_search( 'User-agent: *', $lines, true );

	if ( false !== $agent_index ) {
		$above = array_slice( $lines, 0, $agent_index + 1 );
		$below = array_slice( $lines, $agent_index + 1 );
	} else {
		$above = $lines;
		$below = array();

		$above[] = '';
		$above[] = 'User-agent: *';
	}

	$above[] = "Disallow: $path/wp-content/uploads/wc-logs/";
	$above[] = "Disallow: $path/wp-content/uploads/woocommerce_transient_files/";
	$above[] = "Disallow: $path/wp-content/uploads/woocommerce_uploads/";
	$above[] = 'Disallow: /*?add-to-cart=';
	$above[] = 'Disallow: /*?*add-to-cart=';

	$lines = array_merge( $above, $below );

	return implode( PHP_EOL, $lines );
}