Find nearest number in array python

Finding nearest items across two lists/arrays in Python

Python Code : Sample Output: Pictorial Presentation: Python-Numpy Code Editor: Question: For a given array of real numbers for each element, find the number of elements which is less than current element by no more than and write to new array. With the following code: and a sample of points, I get: So the optimum is , and then the timing is: But in the worst case, you could need , and then: Which is slower than the other options: EDIT Sorting the array before searching pays off for arrays of more than 1000 items: With a 10000 element long array: For smaller arrays it performs slightly worse.

Finding nearest items across two lists/arrays in Python

I have two numpy arrays x and y containing float values. For each value in x , I want to find the closest element in y , without reusing elements from y . The output should be a 1-1 mapping of indices of elements from x to indices of elements from y. Here’s a bad way to do it that relies on sorting. It removes each element that was paired from the list. Without sorting this would be bad because the pairing would depend on the order of the original input arrays.

def min_i(values): min_index, min_value = min(enumerate(values), key=operator.itemgetter(1)) return min_index, min_value # unsorted elements unsorted_x = randn(10)*10 unsorted_y = randn(10)*10 # sort lists x = sort(unsorted_x) y = sort(unsorted_y) pairs = [] indx_to_search = range(len(y)) for x_indx, x_item in enumerate(x): if len(indx_to_search) == 0: print "ran out of items to match. " break # until match is found look for closest item possible_values = y[indx_to_search] nearest_indx, nearest_item = min_i(possible_values) orig_indx = indx_to_search[nearest_indx] # remove it indx_to_search.remove(orig_indx) pairs.append((x_indx, orig_indx)) print "paired items: " for k,v in pairs: print x[k], " paired with ", y[v] 

I prefer to do it without sorting the elements first, but if they are sorted then I want to get the indices in the original, unsorted lists unsorted_x , unsorted_y . what’s the best way to do this in numpy/scipy/Python or using pandas? thanks.

edit : to clarify I’m not trying to find the best fit across all elemets (not minimizing sum of distances for example) but rather the best fit for each element, and it’s okay if it’s sometimes at the expense of other elements. I assume that y is generally much larger than x contrary to above example and so there’s usually many very good fits for each value of x in y , and I just want to find that one efficiently.

can someone show an example of scipy kdtrees for this? The docs are quite sparse

kdtree = scipy.spatial.cKDTree([x,y]) kdtree.query([-3]*10) # ?? unsure about what query takes as arg 

EDIT 2 A solution using KDTree can perform very well if you can choose a number of neighbors that guarantees that you will have a unique neighbor for every item in your array. With the following code:

def nearest_neighbors_kd_tree(x, y, k) : x, y = map(np.asarray, (x, y)) tree =scipy.spatial.cKDTree(y[:, None]) ordered_neighbors = tree.query(x[:, None], k)[1] nearest_neighbor = np.empty((len(x),), dtype=np.intp) nearest_neighbor.fill(-1) used_y = set() for j, neigh_j in enumerate(ordered_neighbors) : for k in neigh_j : if k not in used_y : nearest_neighbor[j] = k used_y.add(k) break return nearest_neighbor 

and a sample of n=1000 points, I get:

In [9]: np.any(nearest_neighbors_kd_tree(x, y, 12) == -1) Out[9]: True In [10]: np.any(nearest_neighbors_kd_tree(x, y, 13) == -1) Out[10]: False 

So the optimum is k=13 , and then the timing is:

In [11]: %timeit nearest_neighbors_kd_tree(x, y, 13) 100 loops, best of 3: 9.26 ms per loop 

But in the worst case, you could need k=1000 , and then:

In [12]: %timeit nearest_neighbors_kd_tree(x, y, 1000) 1 loops, best of 3: 424 ms per loop 

Which is slower than the other options:

In [13]: %timeit nearest_neighbors(x, y) 10 loops, best of 3: 60 ms per loop In [14]: %timeit nearest_neighbors_sorted(x, y) 10 loops, best of 3: 47.4 ms per loop 

EDIT Sorting the array before searching pays off for arrays of more than 1000 items:

def nearest_neighbors_sorted(x, y) : x, y = map(np.asarray, (x, y)) y_idx = np.argsort(y) y = y[y_idx] nearest_neighbor = np.empty((len(x),), dtype=np.intp) for j, xj in enumerate(x) : idx = np.searchsorted(y, xj) if idx == len(y) or idx != 0 and y[idx] - xj > xj - y[idx-1] : idx -= 1 nearest_neighbor[j] = y_idx[idx] y = np.delete(y, idx) y_idx = np.delete(y_idx, idx) return nearest_neighbor 

With a 10000 element long array:

In [2]: %timeit nearest_neighbors_sorted(x, y) 1 loops, best of 3: 557 ms per loop In [3]: %timeit nearest_neighbors(x, y) 1 loops, best of 3: 1.53 s per loop 

For smaller arrays it performs slightly worse.

You are going to have to loop over all your items to implement your greedy nearest neighbor algorithm, if only to discard duplicates. With that in mind, this is the fastest I have been able to come up with:

def nearest_neighbors(x, y) : x, y = map(np.asarray, (x, y)) y = y.copy() y_idx = np.arange(len(y)) nearest_neighbor = np.empty((len(x),), dtype=np.intp) for j, xj in enumerate(x) : idx = np.argmin(np.abs(y - xj)) nearest_neighbor[j] = y_idx[idx] y = np.delete(y, idx) y_idx = np.delete(y_idx, idx) return nearest_neighbor 

n = 1000 x = np.random.rand(n) y = np.random.rand(2*n) 

In [11]: %timeit nearest_neighbors(x, y) 10 loops, best of 3: 52.4 ms per loop 

Find the nearest value and the index of NumPy Array, Approach: Take an array, say, arr [] and an element, say x to which we have to find the nearest value. Call the numpy.abs (d) function, with d as the difference between element of array and x, and store the values in a difference array, say difference_array []. The element, providing minimum difference will be …

NumPy: Find the nearest value from a given value in an array

NumPy: Random Exercise-9 with Solution

Write a NumPy program to find the nearest value from a given value in an array.

Sample Solution :

Python Code :

import numpy as np x = np.random.uniform(1, 12, 5) v = 4 n = x.flat[np.abs(x - v).argmin()] print(n) 

Pictorial Presentation:

Python-Numpy Code Editor:

Python | Find closest number to k in given list, Examples: Input : lst = [3.64, 5.2, 9.42, 9.35, 8.5, 8], First we convert the given list to an array. Find absolute difference with K of each element, Python program to find number of m contiguous elements of a List with a given sum. 31, Dec 19. Python

Fastest way to find the nearest element in the array of real numbers

For a given array of real numbers for each element, find the number of elements which is less than current element by no more than 0.5 and write to new array.

What are the algorithms and approaches to solve this problem?

It is important that the neighborhood of the points is chosen only in the negative direction, which makes it impossible to use the Kdtree or Balltree algorithms.

All of my problem is here with my try of code it.

Although the method below uses simple logic and is easy to write, it is slow. We can speed it up by using a decorated Numba function. This will speed up simple looping tasks to near assembly language speeds.

Install Numba with pip install numba .

from numba import jit import numpy as np # Create a numpy array of length 10000 with float values between 0 and 10 random_values = np.random.uniform(0.0,10.0,size=(100*100,)) @jit(nopython=True, nogil=True) def find_nearest(input): result = [] for e in input: counter = 0 for j in input: if j >= (e-0.5) and j < e: counter += 1 result.append(counter) return result result = find_nearest(random_values) 

Note that the intended result is returned for the test case:

test = [0.1, 0.7, 0.8, 0.85, 0.9, 1.5, 1.7] result = find_nearest(test) print result 

This would solve your specific task.

def find_nearest_element(original_array): result_array = [] for e in original_array: result_array.append(len(original_array[(e-0.5 < original_array) & (e >original_array)])) return result_array original_array = np.array([0.1, 0.7, 0.8, 0.85, 0.9, 1.5, 1.7]) print(find_nearest_element(original_array)) 

EDIT: Using a mask is apparantly faster than the version using numba for smaller arrays(ca. len 10000). For bigger arrays the version of using Numba is faster. So it depends on what size of arrays you want to progress.

Some runtime comparison(in seconds):

For smaller arrays(size=250): Using Numba 0.2569999694824219 Using mask 0.0350041389465332 For bigger arrays(size=40000): Using Numba 1.4619991779327393 Using mask 4.280000686645508 

The break even point on my device is around size 10000(both take ca. 0.33 seconds).

This problem is quite easy to solve for ordered arrays. You have to just search backwards and count all numbers which are bigger than the actual number-radius. If that condition isn't met anymore you can break out of the inner loop (which saves a lot of time).

import numpy as np from scipy import spatial import numba as nb @nb.njit(parallel=True) def get_counts_2(Points_sorted,ind,r): counts=np.zeros(Points_sorted.shape[0],dtype=np.int64) for i in nb.prange(0,Points_sorted.shape[0]): count=0 for j in range(i-1,0,-1): if (Points_sorted[i]-r 

r=0.001 Points=np.random.rand(1_000_000) t1=time.time() ind=np.argsort(Points) Points_sorted=Points[ind] counts=get_counts_2(Points_sorted,ind,r) print(time.time()-t1) #0.29s 

indices = [i for i, val in enumerate(data) if is_prime(val)] 

n = random.choice(data) idx = data.index(n) 

indices_idx = bisect.bisect_left(indices, idx) 

# Some additional error handling needs to happen here to make sure that the index # actually exists, but this'll work for stuff in the center of the list. prime_idx_left = indices[indices_idx - 1] prime_idx_right = indices[indices_idx] 

def find_idx_of_prime(lst, start_idx, stop_idx, dir): for ix in xrange(start_idx, stop_idx, dir): if is_prime(lst[ix]): return ix return dir*float('inf') idx = data.index(number) left_idx = find_idx_of_prime(data, idx, 0, -1) right_idx = find_idx_of_prime(data, idx, len(data), 1) prime_idx = left_idx if idx - left_idx < right_idx - idx else right_idx prime_value = data[prime_idx] # raises TypeError if no primes are in the list. 

def primes(n): ''' Return a boolean list of all primes < n ''' s = [False]*2 + [True]*(n-2) for i in xrange(2, int(n**0.5) + 1): if s[i]: s[i*i : n : i] = [False] * (1 + (n - 1)//i - i) return s 

a = [1,2,4,6,8,12,9,5,0,15,7] is_prime = primes(max(a) + 1) 

def find_idx_of_prime(lst, start_idx, stop_idx, dir): for ix in xrange(start_idx, stop_idx, dir): if is_prime[lst[ix]]: return ix return dir*float('inf')