Geoconda is a collection of python packages that facilitate the development of python scripts for geoinformatics applications. It includes following python packages:
- ArcGIS Python API - provides simple and efficient tools for sophisticated vector and raster analysis, geocoding, map making, routing and directions.
- boto3 - for working files in S3 storage, for example Allas. Example.
- cartopy - for map plotting.
- cfgrib - map GRIB files to the NetCDF Common Data Model
- dask - provides advanced parallelism for analytics, enabling performance at scale, including dask-geopandas, Dask-ML and Dask JupyterLab extension
- descartes - use Shapely or GeoJSON-like geometric objects as matplotlib paths and patches.
- fiona - reads and writes spatial data files.
- gdal - reads and writes spatial data files, and GDAL/OGR data manipulation tools.
- geoalchemy2 - provides extensions to SQLAlchemy for working with spatial databases, primarily PostGIS.
- geopandas - GeoPandas extends the datatypes used by pandas.
- igraph - for fast routing.
- [jupyter](https://jupyter.org/) - Jupyter Notebooks and JupyterLab, CSC RStudio and Jupyter Notebooks tutorial describes how to use these with Puhti
- laspy - for reading, modifying, and creating .LAS LIDAR files.
- lidar - for delineating the nested hierarchy of surface depressions in digital elevation models (DEMs).
- movingpandas - for trajectory data
- networkx - for the creation, manipulation, and study of the structure, dynamics, and functions of complex networks.
- pyproj - performs cartographic transformations and geodetic computations.
- osmnx - download spatial geometries and construct, project, visualize, and analyze street networks from OpenStreetMap's APIs.
- owslib - for retrieving data from Open Geospatial Consortium (OGC) web services
- pysal - spatial analysis functions.
- pdal - for lidar data
- pyntcloud - for working with 3D point clouds.
- pystac-client - for working with STAC Catalogs and APIs.
- python-cdo - scripting interface to CDO (Climate Data Operators).
- rasterio - access to geospatial raster data.
- rasterstats - for summarizing geospatial raster datasets based on vector geometries. It includes functions for zonal statistics and interpolated point queries.
- rtree - spatial indexing and search.
- sentinelsat - downloading Sentinel images
- shapely - manipulation and analysis of geometric objects in the Cartesian plane.
- scipy - inc pandas, numpy, matplotlib etc
- scikit-learn - machine learning for Python.
- skimage - algorithms for image processing.
- swiftclient, keystoneclient - for working with SWIFT storage, for example Allas.
- xarray - for multidimensional raster data, inc. rioxarray.
- And many more, for retrieving the full list in Puhti use:
Additionally geoconda includes:
- spyder - Scientific Python Development Environment with graphical interface (similar to RStudio for R).
- GDAL/OGR commandline tools
- GMT The Generic Mapping Tools
- PDAL Point Data Abstraction Library
- ncview for visualizing netcdf files
- proj4, geos and many more, see
/appl/soft/geo/conda/singularity/geoconda/2021/binfor full list.
Python has multiple packages for parallel computing, for example multiprocessing, joblib and dask. In our Puhti Python examples there are examples how to utilize these different parallelisation libraries.
(If you think that some important GIS package for Python is missing from here, you can ask for installation from firstname.lastname@example.org.)
geoconda module is available in Puhti:
Version number is the same as Python version.
For using Python packages and other tools listed above, you can initialize them with:
module load geoconda
By default the latest geoconda module is loaded. If you want a specific version you can specify the version number of geoconda:
module load geoconda/[VERSION]
To check the exact packages and versions included in the loaded module:
You can add more Python packages to
geoconda, see instructions from CSC Python page.
You can edit your Python code in Puhti with:
- Visual Studio Code in Puhti web interface,
- Visual Studio Code on your local laptop,
- Jupyter Notebook or Lab in Puhti web interface or
- Spyder in Puhti web interface with remote desktop.
To open Spyder in Puhti web interface with remote desktop:
- Log in to Puhti web interface.
- Open Remote desktop: Apps -> Desktop, choose Desktop:
- After launcing the remote desktop open
Host Terminal(Desktop icon) and start Spyder:
module geoconda spyder
Using Allas from Python
There are two Python libraries installed in Geoconda that can interact with Allas. Swiftclient uses the swift protocol and boto3 uses S3 protocol. You can find CSC examples how to use both here. With large quantities of data in Allas, virtual rasters should be considered. More information on how to create and use virtual rasters can be found here.
License and acknowledgement
All packages are licensed under various free and open source licenses (FOSS), see the linked pages above for exact details.
Please acknowledge CSC and Geoportti in your publications, it is important for project continuation and funding reports. As an example, you can write "The authors wish to thank CSC - IT Center for Science, Finland (urn:nbn:fi:research-infras-2016072531) and the Open Geospatial Information Infrastructure for Research (Geoportti, urn:nbn:fi:research-infras-2016072513) for computational resources and support".
- CSC Python parallelisation examples
- Python spatial libraries
- Geoprocessing with Python using Open Source GIS
- GeoExamples, a lot of examples of using Python for spatial analysis
- Automating GIS processes course materials, where most of the exercises are done using Python (University of Helsinki)
- Geohack Week materials
- Multiprocessing Basics
- Geographic Data Science with Python
- Aalto Spatial Analytics course material