Performance issues when using PowerShell to download files during AzureDevops/ TFS pipeline execution

If you are using the PowerShell task together with Invoke-WebRequest or Invoke-RestMethod during your build/release on Azure DevOps or TFS you may not be getting the performance you expect.

Performance issues when using PowerShell to download files during AzureDevops/ TFS pipeline execution

If you are using the PowerShell task together with Invoke-WebRequest or Invoke-RestMethod during your build/release on Azure DevOps or TFS you may not be getting the performance you expect.

We recently experienced such an issue where downloading larger files(~40Mb) during the execution of a build on TFS was taking much longer than expected.

To isolate the issue I setup a Test build and added a single PowerShell Task which executed the following inline PowerShell.

Measure-Command {
Invoke-WebRequest -Uri "https://tfsinstance/tfs/[TeamProjectCollection]/[TeamProject]/_apis/test/runs/[Test Run Id]/attachments/[Test Attachment Id]?api-version=2.0-preview"-OutFile "$(Build.ArtifactStagingDirectory)\MyBuildAgent 2018-10-10 10_10_10.trx"
}

What I saw was that it was taking over 3 minutes to download this 40Mb file, which seemed completely unreasonable as this was over the local network.

Invoke-WebRequest With Progress

I then ran the same PowerShell locally in the Powershell Integrated Scripting Environment(ISE) and saw that I was getting dramatically lower response times ~50 seconds, but this still seemed a little high! What was odd is that I could reproduce similar response times ~1.5 minutes locally if I ran the script in the PowerShell console or from command-line using powershell.exe -command(also used internally by the PowerShell build task). It seems that ISE is running the Progress UI updates on a background thread or something which improves the execution performance.

So how can we improve the performance of Invoke-WebRequest and other similar cmdlets? The difference between these cmdlets and others is that they produce some Progress UI during runtime, it seems that this behavior introduces performance issues. After some bingling I found a number of reports of performance issues with these cmdlets.

Based on these findings I changed my inline Powershell to the following.

$progressPreference = 'silentlyContinue' 
Measure-Command {
Invoke-WebRequest -Uri "https://tfsinstance/tfs/[TeamProjectCollection]/[TeamProject]/_apis/test/runs/[Test Run Id]/attachments/[Test Attachment Id]?api-version=2.0-preview"-OutFile "$(Build.ArtifactStagingDirectory)\MyBuildAgent 2018-10-10 10_10_10.trx"
}

I re-ran my test build and immediately saw a huge improvement! It took just under 2 seconds to download exactly the same file. This improvement was also across all the scenarios described above :)

Invoke-WebRequest Without Progress

Therefore if you are utilizing these cmdlets heavily during your build or release pipelines, or in fact for any non-interactive execution of PowerShell scripts - I would reccomend setting the following option during the execution of your PowerShell scripts.

$progressPreference = 'silentlyContinue' 

Hope this helps!