How to disable Cloudflare's firewall for Lyro? – Tidio

If your website is hosted on Cloudflare, you may be using a firewall for security. However, this might cause some issues when trying to add your website content to Lyro. You may come across "access denied" errors while trying to add your website's URLs as data sources, since Cloudflare's firewall isn't allowing it.

In this short guide, you will learn how to disable Cloudflare's firewall, allowing Lyro to scrape your website content for use as data sources.

💡

Please note: The methods described here can also work for other firewalls; Cloudflare isn't the only system that can block Lyro's ability to scrape a website. You can use the steps from this guide and apply them to your other firewall configurations.

Access your rules

Log into your Cloudflare admin panel and choose your website. Then, navigate to Security > WAF and open the Managed rules tab:

Screenshot 2024-09-03 at 13.16.29.png

Next, use the Add exception button on the right:

Screenshot 2024-09-03 at 13.19.16.png

Add a new exception

Now, you need to create the actual exception that will let Lyro's scraper scan your website. Name your new exception so that you can easily identify it later.

Using the expression builder below, select the Header field; copy and paste x-tidio-lyro-scraper as the Name; and make it equal to true as the Value:

Screenshot 2025-01-28 at 14.06.59.png

Alternatively, you can simply use the Edit expression link on the right and paste the complete expression below:

(any(http.request.headers["x-tidio-lyro-scraper"][*] eq "true"))

Deploy your new exception

Now that your new exception is ready, all you need to do is activate it. In the bottom-right, click the Deploy button:

Screenshot 2024-09-03 at 13.24.47.png

Conclusion

From now on, Lyro will no longer be blocked by Cloudflare's firewall; it should be able to scrape your website and build its knowledge base properly.

To learn more about expanding Lyro's knowledge, see this guide on data sources in general.