Complete Guide to Downloading Files with PowerShell

Introduction

PowerShell provides powerful tools for downloading files from web servers, with Invoke-WebRequest being the primary cmdlet for making HTTP requests. This guide covers everything from basic downloads to advanced scenarios involving authentication, cookies, and custom headers.

---

Basic File Downloads

Simple Download

The most straightforward way to download a file:

Invoke-WebRequest -Uri "https://example.com/file.zip" -OutFile "C:\Downloads\file.zip"

Download with Progress Bar

PowerShell automatically shows a progress bar, but you can disable it for faster downloads:

$ProgressPreference = 'SilentlyContinue'
Invoke-WebRequest -Uri "https://example.com/largefile.zip" -OutFile "largefile.zip"
$ProgressPreference = 'Continue'

Using Aliases

PowerShell provides convenient aliases:

# wget alias
wget https://example.com/file.zip -OutFile file.zip

# curl alias (PowerShell 3.0+)
curl https://example.com/file.zip -OutFile file.zip

Note: In PowerShell 7+, curl might invoke the actual curl.exe if installed. Use Invoke-WebRequest explicitly for consistency.

---

Understanding HTTP Headers

What Are HTTP Headers?

HTTP headers are metadata sent with requests and responses. They control:

• Content type and encoding
• Authentication credentials
• Caching behavior
• Session management (cookies)
• Client identification (User-Agent)

Adding Custom Headers

Invoke-WebRequest -Uri "https://api.example.com/data" `
  -Headers @{
    "Authorization" = "Bearer your-token-here"
    "Accept" = "application/json"
    "User-Agent" = "PowerShell/7.0"
  } `
  -OutFile "data.json"

Common Headers Explained

Header	Purpose	Example
Accept	Specifies expected response format	application/json, text/html
User-Agent	Identifies the client application	Mozilla/5.0 (Windows NT 10.0)
Referer	Indicates the previous page URL	https://example.com/page.html
Authorization	Provides authentication credentials	Bearer token123, Basic base64
Cookie	Sends session/state information	sessionid=abc123; user=john

---

Working with Cookies and Sessions

Understanding Web Sessions

Many web applications require maintaining state across multiple requests. This is typically done through cookies and session tokens.

Method 1: Manual Cookie String

The simplest approach for single requests:

Invoke-WebRequest -Uri "https://example.com/protected/file.pdf" `
  -Headers @{
    "Cookie" = "sessionid=abc123; userid=456; token=xyz789"
  } `
  -OutFile "file.pdf"

Limitations:

• Cookies aren’t automatically updated
• No cookie expiration handling
• Must manually track all cookies

Method 2: WebRequestSession (Recommended)

PowerShell’s WebRequestSession object properly manages cookies across multiple requests:

# Create a new session
$session = New-Object Microsoft.PowerShell.Commands.WebRequestSession

# Add a cookie to the session
$cookie = New-Object System.Net.Cookie
$cookie.Name = "SessionCookie"
$cookie.Value = "V3rBznVdGPy1MDotq4DBUoMiYfbEnEq9000"
$cookie.Domain = "10.10.10.4"
$session.Cookies.Add("http://10.10.10.4", $cookie)

# Make the request using the session
Invoke-WebRequest -Uri "http://10.10.10.4/Java/jviewer.jnlp" `
  -WebSession $session `
  -OutFile "jviewer.jnlp"

Method 3: Building a Session from Login

For applications requiring login:

# Step 1: Login and capture the session
$loginResponse = Invoke-WebRequest -Uri "https://example.com/login" `
  -Method POST `
  -Body @{
    username = "admin"
    password = "password"
  } `
  -SessionVariable session

# Step 2: Use the session for subsequent requests
Invoke-WebRequest -Uri "https://example.com/protected/file.zip" `
  -WebSession $session `
  -OutFile "file.zip"

Adding Multiple Cookies

$session = New-Object Microsoft.PowerShell.Commands.WebRequestSession

# Cookie 1
$cookie1 = New-Object System.Net.Cookie
$cookie1.Name = "SessionToken"
$cookie1.Value = "abc123"
$cookie1.Domain = "example.com"
$session.Cookies.Add("https://example.com", $cookie1)

# Cookie 2
$cookie2 = New-Object System.Net.Cookie
$cookie2.Name = "UserID"
$cookie2.Value = "user456"
$cookie2.Domain = "example.com"
$session.Cookies.Add("https://example.com", $cookie2)

# Make request
Invoke-WebRequest -Uri "https://example.com/data" `
  -WebSession $session `
  -OutFile "data.json"

---

Common Issues and Solutions

Issue 1: 405 Method Not Allowed

Cause: Server rejects the request due to missing or incorrect headers.

Solutions:

1. Add User-Agent header:

Invoke-WebRequest -Uri "http://example.com/file.pdf" `
  -UserAgent "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36" `
  -OutFile "file.pdf"

2. Use WebSession for cookies:

$session = New-Object Microsoft.PowerShell.Commands.WebRequestSession
$cookie = New-Object System.Net.Cookie
$cookie.Name = "auth"
$cookie.Value = "token123"
$cookie.Domain = "example.com"
$session.Cookies.Add("http://example.com", $cookie)

Invoke-WebRequest -Uri "http://example.com/file.pdf" `
  -WebSession $session `
  -OutFile "file.pdf"

Issue 2: 401 Unauthorized

Cause: Missing or invalid authentication.

Solution:

# Basic Authentication
$credentials = Get-Credential
Invoke-WebRequest -Uri "https://example.com/file.zip" `
  -Credential $credentials `
  -OutFile "file.zip"

# Token Authentication
Invoke-WebRequest -Uri "https://api.example.com/file.zip" `
  -Headers @{
    "Authorization" = "Bearer your-api-token"
  } `
  -OutFile "file.zip"

Issue 3: SSL/TLS Certificate Errors

Cause: Self-signed or invalid SSL certificates (common in internal networks).

Solution (Use with caution):

# Temporarily bypass certificate validation
[System.Net.ServicePointManager]::ServerCertificateValidationCallback = {$true}

Invoke-WebRequest -Uri "https://internal-server.local/file.zip" `
  -OutFile "file.zip"

# Reset validation
[System.Net.ServicePointManager]::ServerCertificateValidationCallback = $null

Warning: Only use this for trusted internal servers. Never disable certificate validation for public websites.

Issue 4: 403 Forbidden

Cause: Server blocks requests from automated tools or missing required headers.

Solution:

Invoke-WebRequest -Uri "https://example.com/file.zip" `
  -Headers @{
    "User-Agent" = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
    "Referer" = "https://example.com/download-page"
    "Accept" = "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"
  } `
  -OutFile "file.zip"

Issue 5: Redirect Issues

Cause: Server redirects to another URL, and cookies aren’t preserved.

Solution:

# Use -MaximumRedirection and -SessionVariable
Invoke-WebRequest -Uri "https://example.com/redirect" `
  -MaximumRedirection 5 `
  -SessionVariable session `
  -OutFile "file.zip"

---

Advanced Techniques

Downloading with Retry Logic

function Download-WithRetry {
    param(
        [string]$Uri,
        [string]$OutFile,
        [int]$MaxRetries = 3
    )
    
    $attempt = 0
    $success = $false
    
    while (-not $success -and $attempt -lt $MaxRetries) {
        try {
            $attempt++
            Write-Host "Attempt $attempt of $MaxRetries..."
            
            Invoke-WebRequest -Uri $Uri -OutFile $OutFile
            $success = $true
            Write-Host "Download successful!"
        }
        catch {
            Write-Host "Attempt $attempt failed: $_"
            if ($attempt -lt $MaxRetries) {
                Start-Sleep -Seconds 5
            }
        }
    }
    
    if (-not $success) {
        throw "Failed to download after $MaxRetries attempts"
    }
}

# Usage
Download-WithRetry -Uri "https://example.com/large-file.zip" -OutFile "file.zip"

Parallel Downloads

$urls = @(
    "https://example.com/file1.zip",
    "https://example.com/file2.zip",
    "https://example.com/file3.zip"
)

$urls | ForEach-Object -Parallel {
    $filename = Split-Path $_ -Leaf
    Invoke-WebRequest -Uri $_ -OutFile $filename
    Write-Host "Downloaded: $filename"
} -ThrottleLimit 3

Note: -Parallel requires PowerShell 7.0+

Download with Progress Tracking

function Download-FileWithProgress {
    param(
        [string]$Uri,
        [string]$OutFile
    )
    
    $request = [System.Net.HttpWebRequest]::Create($Uri)
    $response = $request.GetResponse()
    $totalBytes = $response.ContentLength
    $responseStream = $response.GetResponseStream()
    
    $fileStream = [System.IO.File]::Create($OutFile)
    $buffer = New-Object byte[] 10KB
    $totalRead = 0
    
    do {
        $read = $responseStream.Read($buffer, 0, $buffer.Length)
        $totalRead += $read
        $fileStream.Write($buffer, 0, $read)
        
        $percentComplete = [math]::Round(($totalRead / $totalBytes) * 100, 2)
        Write-Progress -Activity "Downloading $OutFile" `
                       -Status "$percentComplete% Complete" `
                       -PercentComplete $percentComplete
    } while ($read -gt 0)
    
    $fileStream.Close()
    $responseStream.Close()
    Write-Progress -Activity "Downloading $OutFile" -Completed
}

# Usage
Download-FileWithProgress -Uri "https://example.com/large-file.zip" -OutFile "file.zip"

Conditional Downloads (Check if File Changed)

function Download-IfNewer {
    param(
        [string]$Uri,
        [string]$OutFile
    )
    
    $headers = @{}
    
    if (Test-Path $OutFile) {
        $lastModified = (Get-Item $OutFile).LastWriteTime.ToUniversalTime()
        $headers["If-Modified-Since"] = $lastModified.ToString("R")
    }
    
    try {
        Invoke-WebRequest -Uri $Uri -OutFile $OutFile -Headers $headers
        Write-Host "File downloaded/updated"
    }
    catch {
        if ($_.Exception.Response.StatusCode -eq 304) {
            Write-Host "File not modified - no download needed"
        } else {
            throw $_
        }
    }
}

# Usage
Download-IfNewer -Uri "https://example.com/file.zip" -OutFile "file.zip"

Downloading from APIs with Pagination

$baseUri = "https://api.example.com/data"
$page = 1
$allData = @()

do {
    $response = Invoke-WebRequest -Uri "$baseUri?page=$page" `
                                   -Headers @{"Authorization" = "Bearer token123"}
    
    $data = $response.Content | ConvertFrom-Json
    $allData += $data.items
    $page++
    
} while ($data.hasMore)

$allData | ConvertTo-Json -Depth 10 | Out-File "all-data.json"

---

Best Practices

1. Use WebSession for Authenticated Requests

When working with authenticated endpoints or maintaining state:

# Good: Proper session management
$session = New-Object Microsoft.PowerShell.Commands.WebRequestSession
# Add cookies to session
Invoke-WebRequest -WebSession $session ...

# Avoid: Manual cookie strings for complex scenarios
Invoke-WebRequest -Headers @{"Cookie" = "..."} ...

2. Always Include User-Agent

Many servers block or limit requests without proper User-Agent headers:

Invoke-WebRequest -Uri $url `
  -UserAgent "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36" `
  -OutFile $file

3. Handle Errors Gracefully

try {
    Invoke-WebRequest -Uri $url -OutFile $file -ErrorAction Stop
    Write-Host "Download successful"
}
catch {
    Write-Host "Download failed: $($_.Exception.Message)"
    # Log error or retry
}

4. Use Secure Connections (HTTPS)

Always prefer HTTPS over HTTP when available:

# Good
Invoke-WebRequest -Uri "https://example.com/file.zip" -OutFile "file.zip"

# Avoid unless necessary
Invoke-WebRequest -Uri "http://example.com/file.zip" -OutFile "file.zip"

5. Validate Downloaded Files

$expectedHash = "ABC123..."
Invoke-WebRequest -Uri $url -OutFile $file

$actualHash = (Get-FileHash $file -Algorithm SHA256).Hash

if ($actualHash -eq $expectedHash) {
    Write-Host "File integrity verified"
} else {
    Write-Host "WARNING: File hash mismatch!"
    Remove-Item $file
}

6. Clean Up Sensitive Data

# Store credentials securely
$securePassword = Read-Host -AsSecureString "Enter password"
$credential = New-Object System.Management.Automation.PSCredential("username", $securePassword)

Invoke-WebRequest -Uri $url -Credential $credential -OutFile $file

# Clear variables after use
Remove-Variable credential, securePassword

7. Use Appropriate Timeouts

Invoke-WebRequest -Uri $url `
  -OutFile $file `
  -TimeoutSec 300  # 5 minutes for large files

8. Respect Rate Limits

$urls = @("url1", "url2", "url3")

foreach ($url in $urls) {
    Invoke-WebRequest -Uri $url -OutFile (Split-Path $url -Leaf)
    Start-Sleep -Seconds 2  # Wait between requests
}

---

Quick Reference

Basic Syntax

Invoke-WebRequest -Uri <url> -OutFile <filename>

With Authentication

# Basic Auth
Invoke-WebRequest -Uri <url> -Credential (Get-Credential) -OutFile <file>

# Token Auth
Invoke-WebRequest -Uri <url> -Headers @{"Authorization" = "Bearer <token>"} -OutFile <file>

With Session Management

$session = New-Object Microsoft.PowerShell.Commands.WebRequestSession
$cookie = New-Object System.Net.Cookie
$cookie.Name = "session"
$cookie.Value = "value"
$cookie.Domain = "domain.com"
$session.Cookies.Add("http://domain.com", $cookie)

Invoke-WebRequest -Uri <url> -WebSession $session -OutFile <file>

Common Parameters

Parameter	Description
-Uri	URL to download from
-OutFile	Path to save the file
-Headers	Custom HTTP headers
-UserAgent	User-Agent string
-WebSession	Session object for cookies
-Method	HTTP method (GET, POST, etc.)
-TimeoutSec	Request timeout in seconds
-MaximumRedirection	Maximum number of redirects
-Credential	Authentication credentials

---

Conclusion

PowerShell’s Invoke-WebRequest is a powerful tool for downloading files and interacting with web services. By understanding sessions, cookies, headers, and proper error handling, you can handle virtually any download scenario from simple file transfers to complex authenticated requests.

Remember to:

• Use WebRequestSession for complex scenarios with cookies
• Always include appropriate headers like User-Agent
• Handle errors gracefully with try-catch blocks
• Validate downloaded content when security matters
• Respect server rate limits and robots.txt

For more information, consult the official PowerShell documentation:

Get-Help Invoke-WebRequest -Full

---

Real-World Example: Downloading JNLP File from BMC Management Interface

This example demonstrates downloading a Java Web Start (JNLP) file from a server management interface that requires authentication via session cookies. This scenario is common when accessing IPMI, iLO, iDRAC, or other BMC (Baseboard Management Controller) interfaces.

The Challenge

The original fetch request from the browser included multiple session cookies and specific headers:

fetch("http://192.168.1.100/Java/jviewer.jnlp?EXTRNIP=192.168.1.100&JNLPSTR=JViewer", {
  "headers": {
    "accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7",
    "accept-language": "en-US,en;q=0.9,sw;q=0.8",
    "upgrade-insecure-requests": "1",
    "cookie": "test=1; SessionCookie=xK9mP2vL8wQ5rN7tY3zA4bC6dE1fG0hJ; BMC_IP_ADDR=192.168.1.100; SessionExpired=false; Username=sysadmin; PNO=4; gMultiLAN=true; settings={eth:[0,1],ethstr:['eth0','eth1'],lan:[1,8],enable:[1,1],flag:[1,1]}",
    "Referer": "http://192.168.1.100/page/jviewer_launch_new.html?JNLPSTR=JViewer&JNLPNAME=/Java/jviewer.jnlp"
  },
  "body": null,
  "method": "GET"
});

Initial Attempt (Failed with 405 Error)

The first attempt to convert this to PowerShell failed with a “405 Method Not Allowed” error:

Invoke-WebRequest -Uri "http://192.168.1.100/Java/jviewer.jnlp?EXTRNIP=192.168.1.100&JNLPSTR=JViewer" `
  -Headers @{
    "accept" = "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7"
    "accept-language" = "en-US,en;q=0.9,sw;q=0.8"
    "upgrade-insecure-requests" = "1"
    "cookie" = "test=1; SessionCookie=xK9mP2vL8wQ5rN7tY3zA4bC6dE1fG0hJ; BMC_IP_ADDR=192.168.1.100; SessionExpired=false; Username=sysadmin; PNO=4; gMultiLAN=true; settings={eth:[0,1],ethstr:['eth0','eth1'],lan:[1,8],enable:[1,1],flag:[1,1]}"
    "Referer" = "http://192.168.1.100/page/jviewer_launch_new.html?JNLPSTR=JViewer&JNLPNAME=/Java/jviewer.jnlp"
  } `
  -Method Get `
  -OutFile "jviewer.jnlp"

# Error: The remote server returned an error: (405) Method Not Allowed.

Why it failed: The server rejected the request because PowerShell’s handling of the cookie string in headers doesn’t properly replicate browser behavior.

Solution: Using WebRequestSession (Success!)

The working solution uses PowerShell’s WebRequestSession object to properly manage cookies:

# Create a new web session object
$session = New-Object Microsoft.PowerShell.Commands.WebRequestSession

# Create and configure the session cookie
$cookie = New-Object System.Net.Cookie
$cookie.Name = "SessionCookie"
$cookie.Value = "xK9mP2vL8wQ5rN7tY3zA4bC6dE1fG0hJ"
$cookie.Domain = "192.168.1.100"

# Add the cookie to the session
$session.Cookies.Add("http://192.168.1.100", $cookie)

# Make the request with proper session management
Invoke-WebRequest -Uri "http://192.168.1.100/Java/jviewer.jnlp?EXTRNIP=192.168.1.100&JNLPSTR=JViewer" `
  -WebSession $session `
  -Headers @{
    "accept" = "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8"
    "accept-language" = "en-US,en;q=0.9,sw;q=0.8"
    "Referer" = "http://192.168.1.100/page/jviewer_launch_new.html?JNLPSTR=JViewer&JNLPNAME=/Java/jviewer.jnlp"
  } `
  -UserAgent "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36" `
  -OutFile "jviewer.jnlp"

# Success! File downloaded to current directory

Complete Solution with Multiple Cookies

If you need to add all the cookies from the original request:

# Create session
$session = New-Object Microsoft.PowerShell.Commands.WebRequestSession

# Define all cookies
$cookies = @{
    "SessionCookie" = "xK9mP2vL8wQ5rN7tY3zA4bC6dE1fG0hJ"
    "BMC_IP_ADDR" = "192.168.1.100"
    "Username" = "sysadmin"
    "SessionExpired" = "false"
    "PNO" = "4"
    "test" = "1"
    "gMultiLAN" = "true"
}

# Add each cookie to the session
foreach ($cookieName in $cookies.Keys) {
    $cookie = New-Object System.Net.Cookie
    $cookie.Name = $cookieName
    $cookie.Value = $cookies[$cookieName]
    $cookie.Domain = "192.168.1.100"
    $session.Cookies.Add("http://192.168.1.100", $cookie)
}

# Note: Complex cookie values like settings={...} may need special handling
# For this specific server, the SessionCookie alone was sufficient

# Make the authenticated request
Invoke-WebRequest -Uri "http://192.168.1.100/Java/jviewer.jnlp?EXTRNIP=192.168.1.100&JNLPSTR=JViewer" `
  -WebSession $session `
  -Headers @{
    "accept" = "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8"
    "accept-language" = "en-US,en;q=0.9,sw;q=0.8"
    "Referer" = "http://192.168.1.100/page/jviewer_launch_new.html?JNLPSTR=JViewer&JNLPNAME=/Java/jviewer.jnlp"
  } `
  -UserAgent "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36" `
  -OutFile "jviewer.jnlp"

Write-Host "JNLP file downloaded successfully!"

Key Takeaways

1. WebRequestSession is essential for authenticated downloads that use cookies
2. Manual cookie strings in headers often fail with 405 or other HTTP errors
3. User-Agent header helps the server recognize the client as legitimate
4. Referer header can be required by some servers to prevent direct linking
5. Session objects properly handle cookie domain, path, and expiration rules

Using the Downloaded JNLP File

After downloading, you can launch the Java application:

# Launch with Java Web Start (if installed)
javaws jviewer.jnlp

# Or with newer Java versions
java -jar jviewer.jnlp

This pattern works for any similar scenario where you need to download files from authenticated web interfaces, including:

• Server management consoles (iLO, iDRAC, IPMI)
• Network device web interfaces
• Enterprise application portals
• Any web application requiring session-based authentication