https://kedro.org/ logo
Join Slack
Powered by
Hello, I have unexpected problem with running the...
# questions
ł
Hello, I have unexpected problem with running the Kedro project with simple pipeline, with one node using the OSMNx - it works without problems in the venv in a script:
Copy code
def get_shortest_path(
    G: nx.MultiDiGraph, origin: tuple[float, float], destination: tuple[float, float]
) -> list[tuple[int, int]]:
    """
    Get the shortest path between two points in the graph.

    :param G: The road network graph.
    :param origin: The (latitude, longitude) of the origin.
    :param destination: The (latitude, longitude) of the destination.
    :return: list of edges in the shortest path.
    """
    orig_node = ox.distance.nearest_nodes(G, origin[1], origin[0])
    dest_node = ox.distance.nearest_nodes(G, destination[1], destination[0])
    shortest_path = nx.shortest_path(G, orig_node, dest_node, weight="length")
    path_edges = list(zip(shortest_path[:-1], shortest_path[1:]))
    return path_edges
But it becomes a problem when I am trying to use it with CLI `kedro run`:
Copy code
UserWarning: An error occurred while importing the                                                                   
                             'networking_route_optimizer.pipelines.data_ingestion' module. Nothing defined therein will be                
                             returned by 'find_pipelines'.                                                                                        
                                                                                                                                                  
                             Traceback (most recent call last):                                                                                   
                               File "/home/ljaniec/.local/lib/python3.10/site-packages/kedro/framework/project/__init__.py", line                 
                             442, in find_pipelines                                                                                               
                                 pipeline_module = importlib.import_module(pipeline_module_name)                                                  
                               File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module                                       
                                 return _bootstrap._gcd_import(name, package, level)                                                              
                               File "<frozen importlib._bootstrap>", line 1050, in _gcd_import                                                    
                               File "<frozen importlib._bootstrap>", line 1027, in _find_and_load                                                 
                               File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked                                        
                               File "<frozen importlib._bootstrap>", line 688, in _load_unlocked                                                  
                               File "<frozen importlib._bootstrap_external>", line 883, in exec_module                                            
                               File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed                                       
                               File                                                                                                               
                             "/home/ljaniec/workspace/networking-route-optimizer/src/networking_route_opti                
                             mizer/pipelines/data_ingestion/__init__.py", line 1, in <module>                                                     
                                 from .pipeline import create_pipeline  # NOQA                                                                    
                               File                                                                                                               
                             "/home/ljaniec/workspace/networking-route-optimizer/src/networking_route_opti                
                             mizer/pipelines/data_ingestion/pipeline.py", line 3, in <module>                                                     
                                 from networking_route_optimizer.pipelines.data_ingestion.nodes import (                                  
                               File                                                                                                               
                             "/home/ljaniec/workspace/networking-route-optimizer/src/networking_route_opti                
                             mizer/pipelines/data_ingestion/nodes.py", line 3, in <module>                                                        
                                 import osmnx as ox                                                                                               
                             ModuleNotFoundError: No module named 'osmnx'                                                                         
                                                                                                                                                  
                               warnings.warn(
What is the problem there? I know that OSMNx standard installation uses 
conda
and I installed it with 
pip
, but I would expect this to be a problem both in script and in the pipeline...
n
Try 
import osmnx
, does it import successfully?
ł
Yes, it works I got something new, copying...
n
Next thing to try. 
which kedro
, see if it matches your virtual env. Sometimes terminal does caching on executable so you need to restart the terminal or run some command
Alternatively, you can do 
python -m kedro run
to make sure it's using the correct python interpreter
ł
Copy code
(networking) ljaniec@ljaniec-PC:~/workspace/networking-route-optimizer$ python
Python 3.10.12 (main, Feb  4 2025, 14:57:36) [GCC 11.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import osmnx
/home/ljaniec/workspace/networking-route-optimizer/venv/lib/python3.10/site-packages/scipy/__init__.py:146: UserWarning: A NumPy version >=1.17.3 and <1.25.0 is required for this version of SciPy (detected version 2.2.4
  warnings.warn(f"A NumPy version >={np_minversion} and <{np_maxversion}"
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/ljaniec/workspace/networking-route-optimizer/venv/lib/python3.10/site-packages/osmnx/__init__.py", line 14, in <module>
    from .distance import nearest_edges as nearest_edges
  File "/home/ljaniec/workspace/networking-route-optimizer/venv/lib/python3.10/site-packages/osmnx/distance.py", line 22, in <module>
    from scipy.spatial import cKDTree
  File "/home/ljaniec/workspace/networking-route-optimizer/venv/lib/python3.10/site-packages/scipy/spatial/__init__.py", line 102, in <module>
    from ._kdtree import *
  File "/home/ljaniec/workspace/networking-route-optimizer/venv/lib/python3.10/site-packages/scipy/spatial/_kdtree.py", line 5, in <module>
    from ._ckdtree import cKDTree, cKDTreeNode
  File "_ckdtree.pyx", line 1, in init scipy.spatial._ckdtree
ValueError: numpy.dtype size changed, may indicate binary incompatibility. Expected 96 from C header, got 88 from PyObject
>>> quit()
I got the same error with 
python -m kedro run
n
That's not the same error you show earlier right?
It's likely version compatibility issue with your 
numpy
scipy
packages
ł
I am checking it right now, reinstalling the libraries in the proper version for osmnx
Ok, it works now, lol - I would never thought of the restart of the terminal, it seems that was the one thing that helped (I got this other error later and it was easy to solve)
n
yeah this is a tricky bit and may depends on the which terminal you are using. but using 
python -m
is a good trick to check, sometimes 
pip
has the same problem
K 1
thankyou 1
6 Views
Open in Slack
PreviousNext
Powered by