Though keep in mind even with caching first request will show bad metrics unless you request each page manually before crawler. So fast backend is the best option.
"what if" is not a great way to think about it, usually the goal is load the minimum amount of data when you server-render, then load more when necessary
depending on the use case there are different answers, but without having a use case there is no point talking about a different solution
when checking performance you can also make the same request using any 3rd party tool and see how fast the data gets fetched as as baseline for what to expect