Fabrice Caneau post an article on SEO best practices for sites that use subscription and paid content. While Google offers something called flexible sampling, previously a first-click free program, Microsoft Bing does not.
Bing basically wants you to detect that BingBot is trying to access this subscription and paywall based content and then give BingBot full access to this content. In a sense, show Bing all the content and only show users a snippet of the content. Bing says it’s okay, it’s not a form of cover-up and it’s allowed.
Fabrice wrote “the first step is to allow search engines, like Bing, to see the full content that normally resides behind a paywall or subscription. By accessing the full content, search engines will be able to index more text, which will help match more customer requests.” “You can identify Bingbot by referencing the IP address to Bingbot’s public list of IP addresses,” he added. The list of BingBot IP addresses was released by Microsoft recently.
Next, Fabrice explained that the next step is to “prevent search engines from exposing publisher subscription-based or paywall-based content in search engine cache pages.” It helps “publishers can control whether search engines show a cached page of a document by using a special bot meta tag in the header section of the web page or, alternatively, by using a client HTTP response header returned to search engine crawler for each URL.”
His blog post explains how to make this Robots Meta Tag and/or X-Robots-Tag.
News websites, academic and research websites, enabling subscription-based and paywall indexing of your content in search engines is easy. Attract more customers to your websites and don’t have to interact with search engines. https://t.co/P8VSJcfewB
— Fabrice Canel (@facan) May 2, 2022
He then explains why they don’t take Google’s approach:
JSON-LD can help but not a strong requirement for me. This is sometimes abused, implemented with issues, and the JSON-LD solution does not work for binaries as subscription-based academic PDFs, which is a common scenario.
— Fabrice Canel (@facan) May 3, 2022