The HTTP request/response mechanism is not friendly enough to the end user in order to perform queries to the Catalogue Service. For this workshop, we will use the QGIS <http://www.qgis.org/> OGC Catalogue Service Client plugin.
To install the plugin in OSGeoLive:
pycsw Workshop$ cd /usr/share/qgis/python/plugins
$ sudo svn co https://qgiscommunitypl.svn.sourceforge.net/svnroot/qgiscommunitypl/python/plugins/qgcsw/trunk/ qgcsw
Note
Remember that the root password in OSGeoLive is “user”.
Start QGIS from the Desktop GIS group and go to “Manage Plugins”.
Enable the CSW plugin (version 0.0.9) from the list
Then select the CSW button from the toolbar and launch CSW Client
Add the pycsw server by pressing the “New” button and type in http://localhost/pycsw/csw.py
The user can add some default servers using the “Add default servers” button and also get the capabilities of the server using “Server info” button
Raw server responces are available through the “GetCapabilities” button.
Note
The button name is not correct right now. It will show the last server responce even if it is not a GetCapabilities responce.
Perform search using the catalogue, either by a string value or with a bounding box
Discovery of data can be also performed through the Tester application by setting the appropriate requests eg. any text search with the string “imagery” leads to discovering XML metadata including the word “imagery”.
Another way to use a pycsw server is through a web application, acting like a CSW client. Such functionality is available through OpenLayers and GeoExt Javascript libraries.
For this workshop we have hacked a small demonstration in GeoExt (thanks Bart van den Eijnden) using a demo pycsw installation at http://demo.pycsw.org/services/csw:
Click icon “find layers”
Enter “airports” (without double quotes)
Click “search” or hit Enter
See results
Click the “add to map” icon beside the last result on that result set (“1 Million Scale - Airports”)
See layer added to map panel
For this exercise we will use the OWSLib Python library.
OWSLib is a Python package for client programming with Open Geospatial Consortium (OGC) web service (hence OWS) interface standards, and their related content models.
OWSLib is already installed in OSGeoLive as a pycsw dependency.
For installation on another system:
$ easy_install OWSLib
or in Ubuntu:
$ sudo apt-get install python-owslib
Connect to local pycsw server, and inspect its properties:
>>> from owslib.csw import CatalogueServiceWeb
>>> csw = CatalogueServiceWeb('http://localhost/pycsw/csw.py')
>>> csw.identification.type
'CSW'
>>> [op.name for op in csw.operations]
['GetCapabilities', 'GetRecords', 'GetRecordById', 'DescribeRecord', 'GetDomain']
You can try to use the following demo deployment for more content: http://geo.gov.ckan.org/csw
Get supported resultType’s:
>>> csw.getdomain('GetRecords.resultType')
>>> csw.results
{'values': ['results', 'validate', 'hits'], 'parameter': 'GetRecords.resultType', 'type': 'csw:DomainValuesType'}
>>>
Search for casino data:
>>> csw.getrecords(keywords=['casino'], maxrecords=20)
>>> csw.results
{'matches': 5, 'nextrecord': 0, 'returned': 5}
>>> for rec in csw.records:
... print csw.records[rec].title
...
CasinoSites
Parcels_North
Parcels
Parcels_East
Parcels_South
>>>
Search for casino data in Canada:
>>> csw.getrecords(keywords=['casino'],bbox=[-141,42,-52,84])
>>> csw.results
{'matches': 0, 'nextrecord': 0, 'returned': 0}
>>>
Search for ‘casino’ or ‘bar’
>>> csw.getrecords(keywords=['casino', 'bar'])
>>> csw.results
{'matches': 11, 'nextrecord': 11, 'returned': 10}
>>>
Search for a specific record:
>>> csw.getrecordbyid(id=['http://capita.wustl.edu/DataspaceMetadata_ISO/CIRA.VIEWS.dv.xml'])
>>> csw.records['http://capita.wustl.edu/DataspaceMetadata_ISO/CIRA.VIEWS.dv.xml'].title
'VIEWS.dv'
Search with a CQL query
>>> csw.getrecords(cql='csw:AnyText like "%bar%"')
Transaction: insert
>>> from urllib2 import urlopen
>>> content = urlopen('http://demo.pycsw.org/waf/Clause_10.2.5.3.2_Example03_Full.xml').read()
>>> csw.transaction(ttype='insert', typename='csw:Record', record=content)
>>> print csw.response
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<csw:TransactionResponse xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:inspire_common="http://inspire.ec.europa.eu/schemas/common/1.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:dct="http://purl.org/dc/terms/" xmlns:ows="http://www.opengis.net/ows" xmlns:apiso="http://www.opengis.net/cat/csw/apiso/1.0" xmlns:gml="http://www.opengis.net/gml" xmlns:dif="http://gcmd.gsfc.nasa.gov/Aboutus/xml/dif/" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:gco="http://www.isotc211.org/2005/gco" xmlns:gmd="http://www.isotc211.org/2005/gmd" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:srv="http://www.isotc211.org/2005/srv" xmlns:ogc="http://www.opengis.net/ogc" xmlns:fgdc="http://www.opengis.net/cat/csw/csdgm" xmlns:inspire_ds="http://inspire.ec.europa.eu/schemas/inspire_ds/1.0" xmlns:csw="http://www.opengis.net/cat/csw/2.0.2" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:os="http://a9.com/-/spec/opensearch/1.1/" xmlns:soapenv="http://www.w3.org/2003/05/soap-envelope" xmlns:sitemap="http://www.sitemaps.org/schemas/sitemap/0.9" version="2.0.2" xsi:schemaLocation="http://www.opengis.net/cat/csw/2.0.2 http://schemas.opengis.net/csw/2.0.2/CSW-publication.xsd">
<csw:TransactionSummary>
<csw:totalInserted>1</csw:totalInserted>
<csw:totalUpdated>0</csw:totalUpdated>
<csw:totalDeleted>0</csw:totalDeleted>
</csw:TransactionSummary>
<csw:InsertResult>
<csw:BriefRecord>
<dc:identifier>00180e67-b7cf-40a3-861d-b3a09337b174</dc:identifier>
<dc:title>Image2000 Product 1 (at1) Multispectral</dc:title>
</csw:BriefRecord>
</csw:InsertResult>
</csw:TransactionResponse>
>>> # search
>>> csw.getrecords(keywords=['image2000'])
>>> print csw.response
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<csw:GetRecordsResponse xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:inspire_common="http://inspire.ec.europa.eu/schemas/common/1.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:dct="http://purl.org/dc/terms/" xmlns:ows="http://www.opengis.net/ows" xmlns:apiso="http://www.opengis.net/cat/csw/apiso/1.0" xmlns:gml="http://www.opengis.net/gml" xmlns:dif="http://gcmd.gsfc.nasa.gov/Aboutus/xml/dif/" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:gco="http://www.isotc211.org/2005/gco" xmlns:gmd="http://www.isotc211.org/2005/gmd" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:srv="http://www.isotc211.org/2005/srv" xmlns:ogc="http://www.opengis.net/ogc" xmlns:fgdc="http://www.opengis.net/cat/csw/csdgm" xmlns:inspire_ds="http://inspire.ec.europa.eu/schemas/inspire_ds/1.0" xmlns:csw="http://www.opengis.net/cat/csw/2.0.2" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:os="http://a9.com/-/spec/opensearch/1.1/" xmlns:soapenv="http://www.w3.org/2003/05/soap-envelope" xmlns:sitemap="http://www.sitemaps.org/schemas/sitemap/0.9" version="2.0.2" xsi:schemaLocation="http://www.opengis.net/cat/csw/2.0.2 http://schemas.opengis.net/csw/2.0.2/CSW-discovery.xsd">
<csw:SearchStatus timestamp="2013-06-13T08:37:07Z"/>
<csw:SearchResults nextRecord="0" numberOfRecordsMatched="1" numberOfRecordsReturned="1" recordSchema="http://www.opengis.net/cat/csw/2.0.2" elementSet="summary">
<csw:SummaryRecord>
<dc:identifier>00180e67-b7cf-40a3-861d-b3a09337b174</dc:identifier>
<dc:title>Image2000 Product 1 (at1) Multispectral</dc:title>
<dc:type>dataset</dc:type>
<dc:subject>imagery</dc:subject>
<dc:subject>baseMaps</dc:subject>
<dc:subject>earthCover</dc:subject>
<dc:format>BIL</dc:format>
<dct:modified>2004-10-04 00:00:00</dct:modified>
<dct:abstract>IMAGE2000 product 1 individual orthorectified scenes. IMAGE2000 was produced from ETM+ Landsat 7 satellite data and provides a consistent European coverage of individual orthorectified scenes in national map projection systems.</dct:abstract>
</csw:SummaryRecord>
</csw:SearchResults>
</csw:GetRecordsResponse>
Transaction: update
>>> # upload an updated version of that metadata
>>> content2 = content.replace('Image2000', 'footitle')
>>> csw.transaction(ttype='update', record=content2)
>>> print csw.response # raw
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<csw:TransactionResponse xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:inspire_common="http://inspire.ec.europa.eu/schemas/common/1.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:dct="http://purl.org/dc/terms/" xmlns:ows="http://www.opengis.net/ows" xmlns:apiso="http://www.opengis.net/cat/csw/apiso/1.0" xmlns:gml="http://www.opengis.net/gml" xmlns:dif="http://gcmd.gsfc.nasa.gov/Aboutus/xml/dif/" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:gco="http://www.isotc211.org/2005/gco" xmlns:gmd="http://www.isotc211.org/2005/gmd" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:srv="http://www.isotc211.org/2005/srv" xmlns:ogc="http://www.opengis.net/ogc" xmlns:fgdc="http://www.opengis.net/cat/csw/csdgm" xmlns:inspire_ds="http://inspire.ec.europa.eu/schemas/inspire_ds/1.0" xmlns:csw="http://www.opengis.net/cat/csw/2.0.2" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:os="http://a9.com/-/spec/opensearch/1.1/" xmlns:soapenv="http://www.w3.org/2003/05/soap-envelope" xmlns:sitemap="http://www.sitemaps.org/schemas/sitemap/0.9" version="2.0.2" xsi:schemaLocation="http://www.opengis.net/cat/csw/2.0.2 http://schemas.opengis.net/csw/2.0.2/CSW-publication.xsd">
<csw:TransactionSummary>
<csw:totalInserted>0</csw:totalInserted>
<csw:totalUpdated>1</csw:totalUpdated>
<csw:totalDeleted>0</csw:totalDeleted>
</csw:TransactionSummary>
</csw:TransactionResponse>
>>> print csw.results
>>> {'deleted': 0, 'insertresults': [], 'updated': 1, 'inserted': 0, 'requestid': None}
Transaction: delete
>>> # delete record we inserted
>>> csw.transaction(ttype='delete', identifier='00180e67-b7cf-40a3-861d-b3a09337b174')
>>> csw.results
{'deleted': 1, 'insertresults': [], 'updated': 0, 'inserted': 0, 'requestid': None}
Harvest a resource
>>> help(csw.harvest)
Help on method harvest in module owslib.csw:
harvest(self, source, resourcetype, resourceformat=None, harvestinterval=None, responsehandler=None) method of owslib.csw.CatalogueServiceWeb instance
Construct and process a Harvest request
Parameters
----------
- source: a URI to harvest
- resourcetype: namespace identifying the type of resource
- resourceformat: MIME type of the resource
- harvestinterval: frequency of harvesting, in ISO8601
- responsehandler: endpoint that CSW should responsd to with response
(END)
>>> csw.harvest('http://webservices.nationalatlas.gov/wms/1million', resourcetype='http://www.opengis.net/wms')
>>> print csw.response
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<csw:HarvestResponse xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:inspire_common="http://inspire.ec.europa.eu/schemas/common/1.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:dct="http://purl.org/dc/terms/" xmlns:ows="http://www.opengis.net/ows" xmlns:apiso="http://www.opengis.net/cat/csw/apiso/1.0" xmlns:gml="http://www.opengis.net/gml" xmlns:dif="http://gcmd.gsfc.nasa.gov/Aboutus/xml/dif/" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:gco="http://www.isotc211.org/2005/gco" xmlns:gmd="http://www.isotc211.org/2005/gmd" xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#" xmlns:srv="http://www.isotc211.org/2005/srv" xmlns:ogc="http://www.opengis.net/ogc" xmlns:fgdc="http://www.opengis.net/cat/csw/csdgm" xmlns:inspire_ds="http://inspire.ec.europa.eu/schemas/inspire_ds/1.0" xmlns:csw="http://www.opengis.net/cat/csw/2.0.2" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:os="http://a9.com/-/spec/opensearch/1.1/" xmlns:soapenv="http://www.w3.org/2003/05/soap-envelope" xmlns:sitemap="http://www.sitemaps.org/schemas/sitemap/0.9" xsi:schemaLocation="http://www.opengis.net/cat/csw/2.0.2 http://schemas.opengis.net/csw/2.0.2/CSW-publication.xsd">
<csw:TransactionResponse version="2.0.2">
<csw:TransactionSummary>
<csw:totalInserted>13</csw:totalInserted>
<csw:totalUpdated>0</csw:totalUpdated>
<csw:totalDeleted>0</csw:totalDeleted>
</csw:TransactionSummary>
<csw:InsertResult>
<csw:BriefRecord>
<dc:identifier>urn:uuid:8ed68189-c33e-43c8-9ac0-59ade5cbf682</dc:identifier>
<dc:title>1 Million Scale WMS Layers from the National Atlas of the United States</dc:title>
</csw:BriefRecord>
<csw:BriefRecord>
<dc:identifier>urn:uuid:8ed68189-c33e-43c8-9ac0-59ade5cbf682-ports1m</dc:identifier>
<dc:title>1 Million Scale - Ports</dc:title>
</csw:BriefRecord>
<csw:BriefRecord>
<dc:identifier>urn:uuid:8ed68189-c33e-43c8-9ac0-59ade5cbf682-national1m</dc:identifier>
<dc:title>1 Million Scale - National Boundary</dc:title>
</csw:BriefRecord>
<csw:BriefRecord>
<dc:identifier>urn:uuid:8ed68189-c33e-43c8-9ac0-59ade5cbf682-impervious</dc:identifier>
<dc:title>1 Million Scale - Impervious Surface 100 Meter Resolution</dc:title>
</csw:BriefRecord>
<csw:BriefRecord>
<dc:identifier>urn:uuid:8ed68189-c33e-43c8-9ac0-59ade5cbf682-coast1m</dc:identifier>
<dc:title>1 Million Scale - Coastlines</dc:title>
</csw:BriefRecord>
<csw:BriefRecord>
<dc:identifier>urn:uuid:8ed68189-c33e-43c8-9ac0-59ade5cbf682-cdl</dc:identifier>
<dc:title>1 Million Scale - 113th Congressional Districts</dc:title>
</csw:BriefRecord>
<csw:BriefRecord>
<dc:identifier>urn:uuid:8ed68189-c33e-43c8-9ac0-59ade5cbf682-landcov100m</dc:identifier>
<dc:title>1 Million Scale - Land Cover 100 Meter Resolution</dc:title>
</csw:BriefRecord>
<csw:BriefRecord>
<dc:identifier>urn:uuid:8ed68189-c33e-43c8-9ac0-59ade5cbf682-cdp</dc:identifier>
<dc:title>1 Million Scale - 113th Congressional Districts by Party</dc:title>
</csw:BriefRecord>
<csw:BriefRecord>
<dc:identifier>urn:uuid:8ed68189-c33e-43c8-9ac0-59ade5cbf682-amtrak1m</dc:identifier>
<dc:title>1 Million Scale - Railroad and Bus Passenger Stations</dc:title>
</csw:BriefRecord>
<csw:BriefRecord>
<dc:identifier>urn:uuid:8ed68189-c33e-43c8-9ac0-59ade5cbf682-airports1m</dc:identifier>
<dc:title>1 Million Scale - Airports</dc:title>
</csw:BriefRecord>
<csw:BriefRecord>
<dc:identifier>urn:uuid:8ed68189-c33e-43c8-9ac0-59ade5cbf682-one_million</dc:identifier>
<dc:title>1 Million Scale WMS Layers from the National Atlas of the United States</dc:title>
</csw:BriefRecord>
<csw:BriefRecord>
<dc:identifier>urn:uuid:8ed68189-c33e-43c8-9ac0-59ade5cbf682-states1m</dc:identifier>
<dc:title>1 Million Scale - States</dc:title>
</csw:BriefRecord>
<csw:BriefRecord>
<dc:identifier>urn:uuid:8ed68189-c33e-43c8-9ac0-59ade5cbf682-treecanopy</dc:identifier>
<dc:title>1 Million Scale - Tree Canopy 100 Meter Resolution</dc:title>
</csw:BriefRecord>
</csw:InsertResult>
</csw:TransactionResponse>
</csw:HarvestResponse>