How to remove duplicate elements from an array in JavaScript?

It’s a library with a host of functions for manipulating arrays. It should also be noted that you may lose the order of the array because objects don’t keep their properties in order. Note that, unlike in python, ES6 sets are iterated in insertion order, so this code preserves the order of the original array. Additionally, we silently change the original array as a side effect – not good!

• There are multiple ways of removing duplicate elements from an array in JavaScript.
• Data array has 6.288 items, 5.284 of them are unique.
• If I come across the person who originally proposed it I will post credit.
• But let’s do it properly this time in the ES6 style.
• If you have properties with Date they will be replaced with their JSON representation (toISOString()).

Hence, during each iteration, if the element equals -1, the element is added to uniqueArr using push(). In this case, our reducer function is checking if our final array contains the item. If it doesn’t, push that item into our final array. Otherwise, skip that element and return just our final array as is . The reduce method is used to reduce the elements of the array and combine them into a final array based on some reducer function that you pass. First, we are creating a new Set by passing an array. Because Set only allows unique values, all duplicates will be removed.

# filter

UniqueBy utilizes mutations (push ) hidden inside its body. It reuses the accumulator instead of throwing it away after each iteration. Since this side effect is wrapped inside the function, everything outside remains pure. If you identify uniqueBy as the cause of a concrete performance penalty in your app, replace it with optimized code.

However, if your input is already sorted, this is the way to go . I have a very simple JavaScript array that may or may not contain duplicates.

JavaScript Fundamentals

@kittu, that is getting the ith element of the array, and putting it into the object . The key is the element, and the value is true, which is entirely arbitrary, as we only care about the keys of the object. The for…of loop is used to loop through all the elements of an arr array. We use the same approach, however we don’t have to check before adding each ID to the Set because duplicate IDs get automatically removed.

In the above program, Set is used to remove duplicate items from an array. In the above program, the duplicate elements are removed from array. In this example, you will learn to write a JavaScript program that removes duplicate values from an array. The Set object only stores unique values, so even if we add the same value to a Set multiple times, the value is only contained once in the Set.

Ways to Clone Objects

The reputation requirement helps protect this question from spam and non-answer activity. Set is great and very intuitive for those used to python. There are probably faster ways but this one is pretty decent.

• @JuanMendes I have created an order-safe version, which simply copies to the new array if the value has not been seen before.
• The indexOf() method returns -1 if the element is not in the array.
• Then it can be converted back to an array using the spread operator.
• Removing duplicates from an array using the Set() constructor takes linear time – O .
• JavaScript provides us with a handy constructor for creating sets – Set().

The callback they do allow is for an ‘iteratee’ function e.g. with args . Although concise, this algorithm is not particularly efficient for large arrays .

Surely this is a lot simpler if you’re just trying to remove duplicates. Obviously this would work better as the return statement in a function. In this tutorial, you have learned some techniques to remove duplicates from an array in JavaScript.

• However, if your input is already sorted, this is the way to go .
• Please use ide.geeksforgeeks.org, generate link and share the link here.
• It’s a library with a host of functions for manipulating arrays.
• Because the keys of a Map object are unique, creating a Map from the array of array removes the duplicate object by key .
• @timrsmith Just a small word of caution with the JSON.parse(JSON.stringify(…)) approach.
• Therefore, we highly advise you to use the Set() constructor in any case possible.
• Afterwards, provided that you encounter performance issues, try to optimize the code at the locations, which are the cause of the problem.
• I could point to all the codes that I’ve tried but I think it’s useless because they don’t work.

Reduce is always a little more difficult to grasp but performs a very powerful reduction operation, borrowed from functional programming. The filter() method creates a new array of elements with the passed condition. This will only contain the element that returns true as part of this filter method. So to achieve the removal of the duplicate elements we just need to add the condition in the filter() method and it will do the rest of the work. In the below example, we are going to use the reduce() method to delete or remove the duplicate elements from the array. The filter() method creates a new array of elements that pass the conditional we provide. In other words, if the element passes and returns true, it will be included in the filtered array.

Using the filter() And indexOf() Methods

Third, iterate over the element of the input array twice and successively compare the first element with the other elements. If two elements cause the callback function to return true, remove that element from the array using the splice() method. Loop through, remove duplicates, and create a clone array place holder because the array index will not be updated. The array is converted to Set and all the duplicate elements are automatically removed.