That is an issue with the webpack config in Adobe’s InDesign React starter.
You need to add a publicPath: “./“ to the output options for this to work. Cf. https://github.com/pklaschka/uxp-indesign-samples/blob/main/plugins/react-swc-sample/webpack.config.js.