Remove Duplicates from Array in PowerShell

Using Select-Object with the -Unique Parameter

To remove duplicates from array in PowerShell:

  • Use the array operator to create an array having duplicate values.
  • Use the Select-Object with the -Unique parameter to select unique values from the array we created in the previous step.

For this code, we used the array operator (@()) to create an array having duplicate values and saved the array in the $array variable using the assignment operator (=). Note that, in PowerShell, the variables names are prefixed with a $ sign, for instance, $VariableName.

Next, we used the Select-Object cmdlet (it is pronounced as command-let, a command used in Windows PowerShell) with the -Unique parameter to select unique values from the given array. How? Let’s break down the $array = $array | Select-Object -Unique command and understand it.

First, we specified the input array ($array) and used pipe (|) to send its results to the Select-Object cmdlet. Here, the pipe was used to send the results of the preceding command to the following command.

Then, the Select-Object cmdlet chooses the given properties of the object/set of objects; it can select the specified objects in a specific index in the array, a certain number of objects and unique objects.

We used this cmdlet with the -Unique parameter to select unique values from the given array. After that, we used the assignment operator to update the original array.

Remember, the -Unique parameter is case-sensitive, which means Hello, and hello would be considered unique values. Let’s see it in the following example containing string-type elements in the array.

Did you see, we got Hello and hello, both. It is because the difference in character casing is considered as unique values for the -Unique parameter with the Select-Object cmdlet (whose alias is Select).

Using Sort-Object with -Unique Parameter

To remove duplicates from an array in PowerShell:

  • Use the array operator to create an array with duplicate values.
  • Use the Sort-Object with the -Unique parameter to remove duplicate values from the array we created in the first step.

We’ve already learned about array creation using the array operator, variable declaration and initialization, assignment operator, pipe sign, selecting unique values and the execution flow of the cmdlet while using Select-Object.

This code example is similar to the one we used while learning Select-Object; here, we used the Sort-Object cmdlet, which can sort the objects in descending or ascending order depending on the object property values. What does that mean?

It means if we do not specify the sort property with the Sort-Object, then it will use the default sort property of the first input object. Remember, if there is no default sort property for the input object’s type, then PowerShell will compare the objects themselves. You can read more about it here

Here, properties will be sorted as case-insensitive or case-sensitive while working with string-type array elements. How? Let’s understand it using the -Unique parameter with the Sort-Object cmdlet below.

The above code fence showed that the -Unique parameter will consider Hello and hello as unique values if the -CaseSensitive parameter is given. Also, the elements will be sorted in alphabetical order considering their first letter due by using the Sort-Object cmdlet.

If you want case-insensitive results, omit the -CaseSensitive parameter and use the following command.

We can also use the Sort as follows, an alias of the Sort-Object cmdlet.

Using Get-Unique with Sort-Object

To remove duplicates from an array in PowerShell:

  • Use an array operator to create an array which contains duplicate values.
  • Use Sort-Object to sort the given array and Get-Unique to remove duplicate values from the specified array.

We have already learned about Sort-Object in the previous section. Here, we used the Get-Unique cmdlet with the Sort-Object cmdlet to remove duplicate values from the given array.

You might be wondering why we are supposed to use the Get-Unique cmdlet with the Sort-Object cmdlet. It is because the Get-Unique cmdlet selects unique values if the array is sorted. So, how the command $array = $array | Sort-Object | Get-Unique was executed?

First, we specified the input array ($array), and its results were forwarded to the Sort-Object cmdlet using a pipe sign. Then, the Sort-Object cmdlet sorted the input array in ascending order which was the default sort property of $array. Afterwards, we used another pipe to forward the sorted array to the Get-Unique cmdlet.

Now, the Get-Unique cmdlet compared every element in the provided sorted array to the next, removed the identical values and returned one instance only for every element. Finally, sorted unique values were assigned to the $array using the assignment operator (=).

Just like the -Unique parameter, the Get-Unique is also case-sensitive, meaning strings with a difference in character casing will be considered unique values. See the following example for a demonstration.

That’s all about how to remove duplicates from Array in PowerShell.

Was this post helpful?

Related Posts

  • Convert Month name to number in PowerShell
    23 January

    Convert Month Number to Name in PowerShell

    Table of ContentsUsing the Get-Culture() CmdletConvert the Hardcoded Month Number to the NameConvert the Month Number from a Date to the NameConvert the Current Month Number to the NameUsing the [cultureinfo]::InvariantCulture Property Using the Get-Culture() Cmdlet We can use the Get-Culture cmdlet in the following three possible ways: To Convert the Hardcoded Month Number to […]

  • Delete empty folders in PowerShell
    23 January

    Delete Empty Folders in PowerShell

    Table of ContentsUsing the Remove-Item with Get-ChildItem CmdletUsing the Get-ChildItem and ForEach-ObjectUsing the User-defined Function There are multiple ways that we can use to delete the empty folders in PowerShell but before going into the details, let’s have a look at the following directory structure to know how many and which folders are empty. [crayon-63d150baa6e22349853231/] […]

  • PowerShell- Where-Object with multiple conditions
    23 January

    PowerShell – Where-Object with Multiple Conditions

    Table of ContentsUsing Where-Object Cmdlet with Multiple ConditionsScript BlockComparison Statements Using Where-Object Cmdlet with Multiple Conditions In PowerShell, we have various comparison and logical operators that we can use with the Where-Object cmdlet to apply multiple conditions. Let’s explore a few scenarios below. [crayon-63d150baa06da811340607/] [crayon-63d150baa06e1937584741/] The above command returned an item where the extension of […]

  • Rename Files with PowerShell
    18 January

    Rename Files with PowerShell

    Table of ContentsRename Single File with PowerShellUsing Rename-Item with/without the -Path ParameterUsing Rename-Item with -Confirm ParameterUsing Rename-Item with -Passthru ParameterRename multiple files in PowerShell Rename Single File with PowerShell Using Rename-Item with/without the -Path Parameter Use the Rename-Item cmdlet to rename a file in PowerShell. [crayon-63d150baa6fbb795259072/] We used the Rename-Item cmdlet to rename the test.txt […]

  • PowerShell - redirect output to File
    18 January

    PowerShell – Redirect Output to File

    Table of ContentsUsing Out-File CmdletUsing Redirection OperatorsUsing StreamWriter .Net ClassUsing Add-Content CmdletUsing Set-Content Cmdlet Using Out-File Cmdlet Use the Out-File cmdlet to redirect PowerShell output to a file. [crayon-63d150baa0ee8090214436/] The Get-Process is used to retrieve all the processes that are currently running on our machine (local computer). We can filter this information by specifying parameters; […]

  • PowerShell - Logical operators
    18 January

    PowerShell – Logical Operators

    Table of ContentsUsing Logical Operators in PowerShellUse Logical AND OperatorUse Logical OR OperatorUse Logical EXCLUSIVE OR OperatorUse Logical NOT OperatorUse Multiple Logical Operators Using Logical Operators in PowerShell PowerShell’s logical operator is a potent tool for comparing data and carrying out operations depending on those comparisons. It allows the user to design an if-then scenario […]

Leave a Reply

Your email address will not be published. Required fields are marked *

Subscribe to our newletter

Get quality tutorials to your inbox. Subscribe now.