Is there an OTB solution for this or I would need to insert a service worker and set up the PWA in react ?
You would need to set up the PWA on your own in an app host.