Exercises

Install pycsw

If you are using OSGeoLive, pycsw is already included. To check your current version:

$ sudo apt-cache show python-pycsw

To install pycsw on a fresh Ubuntu installation:

$ sudo apt-get install python-pycsw pycsw-cgi

This will also install dependencies (lxml, Shapely, pyproj, SQLAlchemy, OWSLib)

Upgrade pycsw

In order to upgrade pycsw to current stable version:

$ sudo apt-get install python-pycsw pycsw-cgi

Tester Application

To run the pycsw tester, use the pycsw launcher from the Web Services group, or open Firefox and navigate to http://localhost/pycsw/tester/index.html

By selecting the left drop-down list, the user can see various predefined POST requests, encoded as XML, that can be sent to pycsw:

../../_images/pycsw_tester_selection.png

For example, by selecting “apiso/DescribeRecord”, and pushing the “Send” button, a description of the ISO Application Profile record is presented on the right panel:

../../_images/pycsw_tester_describe_record.png

By selecting “GetCapabilities-SOAP” and pushing the “Send” button, a SOAP request is sent to the server to advertise their web service capabilities:

../../_images/pycsw_tester_soap_capabilities.png

Also, the user can search for data records, performing a spatial bounding box query, by selecting “GetRecords-filter-bbox” and editing the coordinates in the XML request:

../../_images/pycsw_tester_bbox.png

By selecting “GetRecords-filter-anytext” and pushing the “Send” button, a full text search request is sent to the server:

../../_images/pycsw_tester_any_text1.png

The user can go through all the available requests and perform various requests from this testing application.

Capabilities Document

The capabilities of the pycsw installation can be found at http://localhost/pycsw/csw.py?service=CSW&version=2.0.2&request=GetCapabilities.

Setup service metadata

pycsw’s runtime configuration is defined by default.cfg. pycsw ships with a sample configuration (default-sample.cfg).

To edit the web service metadata, included in the capabilities document, the user can modify the file /var/www/pycsw/default.cfg under the tag [metadata:main].

[metadata:main]

  • identification_title: the title of the service
  • identification_abstract: some descriptive text about the service
  • identification_keywords: comma delimited list of keywords about the service
  • identification_keywords_type: keyword type as per the ISO 19115 MD_KeywordTypeCode codelist). Accepted values are discipline, temporal, place, theme, stratum
  • identification_fees: fees associated with the service
  • identification_accessconstraints: access constraints associated with the service
  • provider_name: the name of the service provider
  • provider_url: the URL of the service provider
  • contact_name: the name of the provider contact
  • contact_position: the position title of the provider contact
  • contact_address: the address of the provider contact
  • contact_city: the city of the provider contact
  • contact_stateorprovince: the province or territory of the provider contact
  • contact_postalcode: the postal code of the provider contact
  • contact_country: the country of the provider contact
  • contact_phone: the phone number of the provider contact
  • contact_fax: the facsimile number of the provider contact
  • contact_email: the email address of the provider contact
  • contact_url: the URL to more information about the provider contact
  • contact_hours: the hours of service to contact the provider
  • contact_instructions: the how to contact the provider contact
  • contact_role: the role of the provider contact as per the ISO 19115 CI_RoleCode codelist). Accepted values are author, processor, publisher, custodian, pointOfContact, distributor, user, resourceProvider, originator, owner, principalInvestigator

Create new database

pycsw supports the following databases:

  • SQLite3
  • PostgreSQL
  • PostgreSQL with PostGIS enabled
  • MySQL

In order to create a new SQLite database we need to:

  1. Edit default.cfg:

[repository]

  1. Setup the database:
$ cd /var/www/pycsw
$ export PYTHONPATH=`pwd`
$ sudo python ./sbin/pycsw-admin.py -c setup_db -f default.cfg

This will create the necessary tables and values for the repository.

The database created is an OGC SFSQL compliant database, and can be used with any implementing software. For example, to use with OGR:

$ ogrinfo /path/to/records.db
INFO: Open of 'records.db'
using driver 'SQLite' successful.
1: records (Polygon)
$ ogrinfo -al /path/to/records.db
# lots of output

Note

Don’t forget to test the configuration by sending a GetCapabilities request to the pycsw server.

Load demo metadata

We will use some demo GIS data from the GeoNode project. The following will download the data archive and load all metadata xml files into our new database.

$ cd ~
$ wget https://github.com/GeoNode/gisdata/archive/master.zip
$ unzip master.zip
$ cd /var/www/pycsw
$ sudo ./sbin/pycsw-admin.py -c load_records -f default.cfg -p ~/gisdata-master/gisdata/metadata/good -r

Metadata Creation

Metadata files for a spatial dataset or a spatial web service can be created with a plain XML editor. Usually a specialized application is used to create metadata XML files, e.g. through the open source implementation of inspire directive for metadata, European Open Source Metadata Editor (EUOSME). This application can be found at http://inspire-geoportal.ec.europa.eu/editor/. Source code is available at https://joinup.ec.europa.eu/svn/euosme/trunk

../../_images/pycsw_euosme_home.png

The user fills the mandatory metadata fields going through the application tabs, adding information like the name of the data owner, keywords, resource location on the web, geographic location (using a bounding box or the name of a country) etc.

../../_images/pycsw_euosme_map.png

After the user has added all the information available, must push the validation button on top of the page, so that a check to be performed for consistency with the INSPIRE directive.

After a sucessful validation, the XML file can be saved to the local disk and viewed through a text editor or a browser.

../../_images/pycsw_euosme_save.png

Metadata editing alternatives include:

Note

Try to create new XML files and then try to load them into the pycsw server as shown previously.

Metadata Harvesting

We can harvest a working WMS service with the following:

<?xml version="1.0" encoding="UTF-8"?>
<Harvest xmlns="http://www.opengis.net/cat/csw/2.0.2" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.opengis.net/cat/csw/2.0.2 http://schemas.opengis.net/csw/2.0.2/CSW-publication.xsd" service="CSW" version="2.0.2">
  <Source>http://webservices.nationalatlas.gov/wms/1million</Source>
  <ResourceType>http://www.opengis.net/wms</ResourceType>
  <ResourceFormat>application/xml</ResourceFormat>
</Harvest>

Save the file to something like “post.xml” and run:

$ cd /var/www/pycsw
$ python ./sbin/pycsw-admin.py -c post_xml -u http://localhost/pycsw/csw.py -x ~/post.xml

Note

In order to enable Harvesting, you need to edit default.cfg and set transactions=true.

Note

For transactions to be functional when using SQLite3, the SQLite3 database file (and its parent directory) must be fully writable. For example:

$ mkdir /path/data
$ chmod 777 /path/data
$ chmod 666 test.db
$ mv test.db /path/data

For CSW-T deployments, it is strongly advised that this directory reside in an area that is not accessible by HTTP.