Added geospatial catalog capabilities #4
Conversation
|
this is a valid and interesting one for sure. the intent though here is a not, separate but equal kind of thing. this guidance is intended to have a minimum core at /data. at this stage, i am inclined to keep it simple like the guidance suggests, clearly lots of discussion is warranted though. but others should join the discussion like @MarinaMartin |
|
The geospatial community has nearly half a million records already in a variety of directories and standards-based catalog services from different vendors. There are additional data sources from NASA and USGS (imagery) that will soon be enabled through APIs to provide access to imagery catalogs. It is impractical and duplicative to impose a new non-standard cataloguing methodology on a community that has implemented a robust solution with state and local partners using international standards. What we propose here with support from GSA and DOI is to supply the core metadata for each catalog/collection. This is consistent with the policy and easily implemented by the CKAN developers, allowing each agency to still have one endpoint to work from. The only thing we need to add to the guidance is the pattern for describing a catalog and then enable the agency-side tool to describe such resources. (Oops - I didn't mean to Close this one...) |
|
On 5/10/13 8:41 AM, Michael Byrne wrote:
Doug. Douglas D. Nebert |
|
The open data movement is involved in a lot of wheel reinvention viz. the geospatial/GIS community. Have you ever wondered why most open data portals are 90% geospatial data? It's largely because that community already has systems in place for data quality, provenance tracking, data privacy, metadata, archival, etc. and has created many machine-readable formats defined by open standards over the past decades. These are the main reasons why geospatial datasets have been so easy to release. On the other hand, very simple and important non-GIS datasets - like a list of elected officials with contact information - are few and far between, because such datasets lack a similar infrastructure, which the open data community is only building now. I think it makes sense to allow the GIS community to continue using the open standards and technologies that have served them well over the last decades and to extend this olive branch which recognizes that many of the things we're doing in open data aren't new. Unless you are really committed to inventing your own wheel. |
|
Less reinvention of the wheel is certainly good, but remember we also aren't exactly talking complete apples-and-oranges issues here. For example, FGDC CSGDM geospatial metadata records and ISO geospatial metadata records contain a lot of the same core information as to Dublin Core and others, and these have already been crosswalked, the mappings already exist at least on those Dublin Core style "who, what, when, where, why" types of questions. If anything, the minimal disconnect is in that the geo community generally goes beyond a lot of other types of information access portals present, along with lacking some of the more innovative features or such things as the ability to navigate effectively through a multitude of related information resources. |
|
The mappings into CKAN - the actual index and means for presentation of results - have already been made for both FGDC and ISO metadata. The proposed seed to identify a Harvest Source (catalog) in lieu of duplicating the metadata is sufficient for all needs. The latest CKAN deployment at catalog.data.gov enables the discovery and navigation of composite data set collections based on parent-child associations asserted in the metadata. This allows 'series' like Landsat or Census data to be more intuitively found and browsed, and allows initial results to not be overwhelmed by thousands of near identical data hits. The parent is found first, and then children can be easily filtered on based on any indexed property. Increasingly, we are also describing applications and Web Services - some of which are automatically launched in map viewers to allow users to explore and mix data. All this based on the geospatial metadata content. |
|
@DruidSmith I think the point Doug makes here is not about features of websites, but of the process of describing resources (datasets, services, applications, ...) in standardized ways (per FGDC or ISO and hopefully soon merging specs...) and exposing the agency registries of these resources in standardized ways (through OGC CSW and now also more and more through OpenSearch). This history and investment in the describing and sharing of geospatial resources means that new formats and descriptions result in extra work for the agencies providing this information while they have been doing the same describing and sharing for over a decade through programs like Geospatial One-Stop and now Geo.data.gov. From my time in these programs I have heard frequently from agencies that they want to publish once and find their content everywhere (Data.gov, Geoplatform.gov, GEOSS, GSDI, OneGeology, US GIN, ...). What I think we can avoid is that all agencies have to develop new tools and processes to provide the listing in DCAT format asked for here. Hence our work to implement the DCAT format in our technology (Geoportal Server) which can be seen in progress at http://gptogc.esri.com/geoportal/rest/find/document?f=dcat). @ddnebert Doug, can you share these mapping of FGDC and ISO metadata into DCAT? would be good to add these mappings to this site so that other solution providers can implement the same mappings. Have you thought about guidelines for data publishers with respect to the fields that are not part of the FGDC/ISO metadata but are asked for in the DCAT listing of their data catalogs? |
|
Whoever allows the commits, please merge my Geospatial practices into the http://project-open-data.github.io/implementation-guide/ document, the prose-patch-1 branch on ddnebert/project-open-data.github.io. We need to get the clear guidance out there for the geospatial community that allows for the "core" documentation of geospatial metadata collections as a single entry in the json feed that permits their traversal by CKAN harvester. This is consistent with many of the points in the documentation that encourage communities to develop metadata extensions, use standards, even catalogs that support DCAT. Duplication or export of subset geospatial metadata into a feed is not appropriate or necessary and will require significant extra work on geospatial publishers. Who is able to take action on this inclusion? |
|
Does your update also include mapping fgdc and iso metadata to dcat json? Sent via the Samsung Galaxy S™ III, an AT&T 4G LTE smartphone -------- Original message -------- Whoever allows the commits, please merge my Geospatial practices into the http://project-open-data.github.io/implementation-guide/ document, the prose-patch-1 branch on ddnebert/project-open-data.github.io. We need to get the clear guidance out there for the geospatial community that allows for the "core" documentation of geospatial metadata collections as a single entry in the json feed that permits their traversal by CKAN harvester. This is consistent with many of the points in the documentation that encourage communities to develop metadata extensions, use standards, even catalogs that support DCAT. Duplication or export of subset geospatial metadata into a feed is not appropriate or necessary and will require significant extra work on geospatial publishers. Who is able to take action on this inclusion? — |
|
On 5/28/13 1:22 PM, Marten wrote:
Doug.
Douglas D. Nebert |
Start index from scratch
|
For now, I believe the current open data policy implementation approach is working simply and symbiotically with the geospatial policy requirements. See also: https://www.digitalgov.gov/resources/how-to-get-your-open-data-on-data-gov/#federal-geospatial-data |
This pull request has been automatically generated by Prose.io.