Premium guide



This documentation will guide you on how to use premium datasets that you can buy on the premium site. The first section describes the format of the CSV files and an example on how to import it in PostgreSQL while the second, explain how to inject a Gisgraphy SQL dump. You can also download the PDF version for offline use.

CSV Dataset

File formats

Before buying you probably wonder how good is the coverage for a country and see how it looks like, select the country and see how many records we got for each dataset

Those formats are not what you search for ? You need a custom extract, contact us

The CSV files contain some IDS that can refer to others CSV files as an SQL table can have a foreign key that reference on other columns in other tables (primary key). For instance, the street files have a 'cityid' field that refers to the field id of the City files.

  • In addresses file, 'streetid' refers to the id field in the street file
  • In street file, 'cityid' refers to the id field in the city file
  • In city file, 'admid' refers to the id field in the adm file
  • In adm file, 'parentid' refers to the id field in the adm own file. The adm are sorted by level in order to first have the top level ADMs that can be referenced by other ADMs in the next lines of the file.
  • POIs (point of interest) are not linked to any file


It is very important to notice that those XXXID fields are optional and are sometimes not filled



So for an address, you can get the street with the streetid field that refers to the id field in the street file whith the street id , you can get the city from the cityid that refer to the id field in the city file, and then get the adm with the admid that refers to the IDin the ADM file.



Example of use


The CSV / TSV datasets is a structured format that allows being used in many ways. it can be used in Excel, or in a database, or anything that can process a text file or read a TSV file. This section only shows a common use to inject the files into a Postgres Database. (a same tutorial could be written for MySQL or Oracle).

First, you need to create the tables. You can find a SQL script that reflects the format of the CSV files. As described in the Format section (above). The files have some fields that reference ids on other files. If you want to inject files into a database and use Foreign keys mechanism, you will have to inject files in that order :

  1. Adms
  2. Cities
  3. Streets
  4. addresses

of course, if you don't have to buy all the files and skip those you haven't buy. if you buy streets, you will have a field called 'city' that reflect the city where the street is, but you won't have more information on the city. if you buy the city files, you will get alternate names, shape, population, admin center, location, ...(when available, all the fields are optional)

Gisgraphy SQL dumps

The import of data into Gisgraphy is a very long process. If you don't want to import the data by yourself, you can buy a dump of data that you can inject directly into PostgreSQL database. Once the dump downloaded (the link is provided in your dashboard or in the mail received when purchasing), you have to follow some step.

1-Install Gisgraphy

If it is not already done, You have to download and install Gisgraphy. The latest version of Gisgraphy is available at https://download.gisgraphy.com/releases/gisgraphy-latest.zip. The installation guide will help you to set it up.
Once done, follow the steps below. All those steps have to be done when gisgraphy is shutdown.

Important : You've bought a SQL dump, so you don't need to run an import of data via the admin interface. The dump you've bought is already the result of a dump that we have processed on our servers !

Those steps are independants from the installation you've choose. It can be run on Linux, Windows, Mac, on a Docker container or not.

2-Inject the dump

Go to URL given in the mail to download the dump. Note that you got 30 days to download the files.

2a-Without Docker

For those who want to go quickly, we provide (as is and without any warranty of any kind) a script to inject the dump. To use it, run the following commands (replace the words in uppercase according to your configuration) :

#clone the Github repository
git clone https://github.com/gisgraphy/gisgraphy-docker.git
cd gisgraphy-docker/dump

#put the dump file into the assets/dump/ directory
mkdir -p ./assets/dump
cp /PATH/TO/DUMP_TAR_FILE ./assets/dump/

#run the script
./inject-wo-docker.sh YOUR_POSTGRESQL_PASSWORD /PATH/TO/GISGRAPHY/DIRECTORY/


2b-With Docker

To use the dump and inject data in a Docker image :

#clone the Github repository
git clone https://github.com/gisgraphy/gisgraphy-docker.git
cd gisgraphy-docker/dump

Put the dump file into the 'assets/dump' directory :

mkdir -p ./assets/dump
cp /PATH/TO/DUMP_TAR_FILE ./assets/dump/

Build an image with the dump from a Gisgraphy Docker image (replace 'mdppostgres' and 'gisgraphyofficial' according your configuration) :

docker build  -t gisgraphydump --build-arg PGPASSWORD=mdppostgres --build-arg BASE_IMAGE=gisgraphyofficial . --build-arg SOLR_DIR=/usr/local/gisgraphy/solr/

Note that you can also use volumes create volumes on PostgreSQL data dir (/var/lib/postgresql/9.5/main) and Solr data dir (/usr/local/gisgraphy/solr/data) . See https://docs.docker.com/engine/admin/volumes/volumes/ for more infos.

2c-Step by step

For those who want to be guided, step by step, do the following instructions :

Inject the SQL dump file

  1. Decompress the file corresponding to the SQL dump. It can be a zip or a tar file. If you are on Windows, you can use 7-zip or if you are on Linux, you can untar with the following command : tar -xvf YOURFILE.tar
  2. If you 've already done an import, reset it :
    • Log on the admin page (mainMenu.html).
    • Go to the Admin / Reset import menu.
    • Confirm you want to reset the import.
    • Reset it.

  3. Inject the SQL dump in PostgreSQL (custom the following line according your settings) :
  4. pg_restore  -h 127.0.0.1 -p 5432 -U postgres  -Fc -O -v -dgisgraphy /PATH/TO/FILES/dump_localhost.gz 
    
  5. Run the createGISTIndex.sql file that is in the sql directory of the Gisgraphy distribution. (custom the following line according to your settings)
  6. psql -U postgres -h 127.0.0.1 -d gisgraphy -f ./sql/createGISTIndex.sql


    If you got a ‘duplicate entry’ error message in the User or in the Profile table. Simply ignore it ! It is because you’ve already got some users in the database.



    Setup the full-text engine (optional)

    This section is optional and could be skipped if no full-text engine dump (Solr) is provided. You only need the full-text engine if you use, geocoding, full-text / autocompletion web services. If you only use reverse geocoding, street search, and nearby web service, you can skip this section. It will save disk space (that will have no impact on performance)

    1. Unzip the full-text engine dump named 'data.zip'. It contains a directory called 'data'.
    2. Copy (or replace) this directory data in /SolR directory in the Gisgraphy distribution.

    That’s all ! (Re)start Gisgraphy with the launch.sh script and use it

    3-Support

    If you got problems, you can use the contact page or send us a mail.