Locate the robots.txt file

currently we don't have robots.txt, so depending on what you need you can do it on cloudflare

also, wouldn't redirects settings solve you problem?

from what i understand you would need to redirect to something that makes sense. those urls are old and pointing to something i dont have. plus there are around 10k left

also i see robot.txt is in your seo details on the webstudio site. is it there just not accessible.

where do you see it on webstudio site?

you mean this? https://webstudio.is/robots.txt

that's something we generate but we don't have a UI to edit from the builder, follow this issue https://github.com/webstudio-is/webstudio/issues/4257

another option for you right now is to serve your own robots.txt and host it somewhere else

or host the site via cli yourself, then you can edit robots.txt as usual

easierst way is to sign into the cloudflare and redirect robots path to any static file hosting, even github would work fine

ok will do that. ty 😄