Delete file or folder speed

Adobe XD UXP Plugin API
1

Hi, I believe I’m doing something wrong but function seems pretty obvious so I’m not sure what exactly is wrong. So the function is:

private async cleanup(folder: uxp.Folder): Promise<void> {
  const folders = await folder.getEntries();
  const tasks = folders.map(async (sub): Promise<void> => {
    if (sub.isFolder) {
      await this.cleanup(sub as uxp.Folder);
    }
    await sub.delete();
  });

  await Promise.all(tasks);
}

Pretty obvious. I have folder with like 50 folders and about 5-7 entries within each folder. This function took several minutes to remove them. Did I miss something? Is there any way to speed it up?

2

deleting files is an expensive operation, so I’d assume the function can take a while if you have a lot of files to delete. @kerrishotts any recommendations on speeding up the process?

3

Just to keep up to date I slightly adjusted function, I didn’t change speed much but anyway:

private async cleanup(folder: uxp.Folder): Promise<void> {
  const entries = await folder.getEntries();
  const self = this;
  const filesTask = entries.filter(e => !e.isFolder).map(e => e.delete());
  const foldersTask = entries.filter(e => e.isFolder).map(e => self.cleanup(e as uxp.Folder));
  await Promise.all(foldersTask);
  const tasks = entries.map((e) => e.delete());
  await Promise.all(tasks);
  await Promise.all(filesTask);
}
4

looks redundant, and maybe failing (causing slowdowns), because you’re already handling the files at this level in filesTask and the sub-folders in foldersTask.

5

Thanks for heads up! Updated the function below. But performance seems to be the same (

private async cleanup(folder: uxp.Folder): Promise<void> {
  const entries = await folder.getEntries();
  const files = entries.filter(e => !e.isFolder);
  const folders = entries.filter(e => e.isFolder);
  let foldersTask = folders.map(e => this.cleanup(e as uxp.Folder));
  const filesTask = files.map(e => e.delete());
  await Promise.all(foldersTask);
  foldersTask = folders.map((e) => e.delete());
  await Promise.all(foldersTask);
  await Promise.all(filesTask);
}
6

This takes several minutes? Ouch.

What OS are you running?

Also, could you instrument this a little to see where the delays are?

And, it would be helpful to see a directory structure (even if simulated) to get an idea for how deep you’re having to recurse and such.

7

Sorry for the delay, here’s the result of the run:
Cleanup finished in 150323 ms. I removed 510 files and 177 folders with total size of 110Mb.
OS is Windows 10 x64 version 1903.
Directory structure consists of 50 folder, each folder has 1 file and 1-5 folders. Each subfolder contains 4-5 files. So it’s pretty simple structure, production structure will contain a way more files.

8

One more side effect. When I’m removing a lot of files Runtime Broker starts consuming a lot of cpu and it’s impossible to stop it. It just hangs for several minutes.
Is it possible to add batch delete for folder with files? I don’t see any other ways to clean up temporary files :frowning:

9

Up the topic:

image
image530×163 7.65 KB

And it takes about 10 minutes to finish all the staff. All that time it consumes 70+% of cpu.

10

Woof.

Something on our list this year is to improve I/O performance (moving a lot of the moving parts to C++), so that might help. I don’t have a finer-grained time frame though.