Home / Specifications / cdsc-wg1-01 - Server Metadata

Server Metadata

Looking for a more human-readable summary of this specification? Check out our overview.

Below is a rendering of the original specification source: https://github.com/daniel-utilityapi/Connectivity/blob/main/specifications/cdsc-wg1-01.md

CDSC-WG1-01 - Server Metadata

Abstract 🔗

This specification defines how utilities and other entities may provide structured “metadata”, such as reference information about them, functionality they offer, and how organizations can interoperate with them. The defined base metadata endpoint is intended to act as a starting point for organizations to discover details and capabilities of energy utilities and other similar entities.

Status 🔗

WARNING: This specification is a DRAFT and has not achieved consensus by the working group.

Copyright 🔗

TODO

Table of Contents 🔗

• 1. Introduction
• 2. Terminology
• 3. Metadata Endpoint
  • 3.1. Metadata Well-Known URI
  • 3.2. Metadata Object Format
  • 3.3. Metadata Endpoint Organization
  • 3.4. Server Capabilities
• 4. Coverage Endpoint
  • 4.1. Coverage Listing Format
  • 4.2. Coverage Listing Filters
  • 4.3. Coverage Entry Format
  • 4.4. Coverage Entry Types
  • 4.5. Capability Roles
  • 4.6. Infrastructure Types
  • 4.7. Commodity Types
• 5. Server Migration
• 6. Extensions
• 7. Examples
• 8. Security Considerations
  • 8.1. Restricted Access
  • 8.2. Rate Limiting
• 9. References
• 10. Acknowledgments
• 11. Authors’ Addresses

1. Introduction 🔗

This specification was developed as part of the global effort to combat the climate crisis. Specifically, in order to scalably measure carbon emissions of organizations and calculate the impact of deploying and operating clean energy technologies, companies need an efficient means to discover the details and capabilities of energy utilities and other similar entities.

There are thousands of utilities serving customers across the world, and each have their own way of organizing and structuring data. This specification defines a way for these utilities and other entities to provide standardized, structured reference information about them, the functionality they offer, and how organizations can interoperate with them. By serving “metadata” defined in this specification, utilities and other entities can automatically incorporate their details and capabilities into carbon tracking websites, apps, products, and other clean energy technologies.

This specification is intended to be a starting point for providing basic information about utilities and other similar entities, then extended by other specifications to define additional information and functionality. For example, this specification may be extended to define how organizations can dynamically register with a utility in order to gain access to customer-authorized energy usage data.

2. Terminology 🔗

🔗 “Server” - The entity hosting the metadata endpoint. A Server can be an energy utility or another type of entity that want to provide structured information and capabilities about themselves. These entities can include, but are not limited to, distribution utilities, grid operators, electric retailers, community choice aggregators, government agencies, data warehouses, and private infrastructure providers.

🔗 “Client” - The entity requesting Server’s metadata endpoints. A Client can be any organization or user seeking the Server’s metadata on their details and capabilities. These entities can include, but are not limited to, carbon tracking applications, electric vehicle companies, clean energy technology providers, commercial utility customers, grid management applications, and energy efficiency organizations.

🔗 Referenced Technologies: “HTTPS”, “Request Methods”, “Status Codes”, “JSON”, “Well-Known URI”, “ISO8601”, “OAuth”, “GeoJSON”, TODO: add more as needed are defined by their referenced standards documents.

🔗 “string” - A series of unicode characters as defined in RFC 8259.

🔗 “datetime” - A string representing date and time in the format of date-time as defined by RFC 3339 (e.g. “2024-01-01T00:00:00Z”).

🔗 “URL” - A string representing resource as defined in RFC 3986 (e.g. “https://example.com/page1”).

🔗 “MIME type” - A string representing a document media type as defined in RFC 6838 (e.g. “image/png”).

🔗 Key Words: “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “NOT RECOMMENDED”, “MAY”, and “OPTIONAL” are defined in accordance with BCP 14.

3. Metadata Endpoint 🔗

Servers supporting this specification SHALL offer one or more HTTPS URLs from which a Client can perform a GET request to retrieve Server metadata, and the Server will respond with the Server’s metadata JSON object.

For successful responses, Servers MUST respond with a “200 OK” Status Code and body containing a JSON object that is formatted as defined in Metadata Object Format. Cache-optimized responses, such as “304 Not Modified”, to Client requests with cached value headers, such as If-None-Match, is also acceptable.

For invalid requests and error responses, Servers MUST respond with the appropriate Response Code for the type of issue or error (e.g. “503 Service Unavailable” if the Server is temporarily down for maintenance).

3.1. Metadata Well-Known URI 🔗

If the Server wishes to provide metadata publicly, the Server MUST support a metadata endpoint at the Well-Known URI "/.well-known/carbon-data-spec.json" on their hostname (e.g. https://example.com/.well-known/carbon-data-spec.json).

Servers MAY additionally support other public and restricted URLs for special case or private metadata endpoints, as needed.

3.2. Metadata Object Format 🔗

Metadata objects are formatted as JSON objects and contain named values, some of which are required and some of which are optional. The following values are included in the default list available in metadata objects.

• cds_metadata_version - string - (REQUIRED) The version of metadata object format this Server follows, which for this version of the specification is v1.
• cds_metadata_url - string - (REQUIRED) A URL link to this metadata object’s canonical network location.
• created - datetime - (REQUIRED) When the metadata url was first made available.
• updated - datetime - (REQUIRED) When this metadata object was last modified.
• name - string - (REQUIRED) The name of the utility or entity (e.g. “Demo Gas & Electric”).
• description - string - (REQUIRED) A brief description about the utility or entity (e.g. “An electric and gas utility serving customers in Northern California”).
• website - URL - (REQUIRED) Where users can learn more about the utility or entity and its capabilities.
• documentation - URL - (REQUIRED) Where Clients can find the utility or entity’s technical reference documentation.
• support - URL - (REQUIRED) Where Clients can find contact information for technical support on the Server’s capabilities.
• capabilities - Array[ServerCapability] - (REQUIRED) What capabilities and functionality this Server supports, which may be an empty array if the Server is not offering any additional capabilities beyond just this metadata object. If the list of capabilities includes coverage, this array MUST also include a union of the capabilities listed in the Coverage Endpoint list of Coverage Entry objects’ capabilities values.
• coverage - URL - (OPTIONAL) Where to find the Coverage Endpoint. This field is REQUIRED if coverage is listed in the capabilities array.
• related_metadata - Array[URL] - (OPTIONAL) Where to find other related Metadata Endpoints.

3.3. Metadata Endpoint Organization 🔗

Many utilities and other similar entities are structured as multiple divisions, operating companies, territories, legal constructs, or other types of groupings. These organizational complexities and legal structures can make it difficult to publish a single representative metadata endpoint for an entire entity.

When utilities and other similar entities wish to publicly provide multiple metadata objects that more cleanly define the metadata and capabilities for their entity, their Server MUST publish a single metadata endpoint at the well-known URI and within that metadata object include an array of related_metadata URLs that link the Client to the various subdivided or related metadata endpoints.

If capabilities are available by a parent organization across a group of subsidiary utilities or entities (e.g. a holding company offers universal customer data access across many of their utility operating company territories), the parent organization MAY publish a single metatdata object with coverage entries that list the individual subsidiary utility or entity coverage entries.

It is acceptable for the initial well-known metadata endpoint object to include a minimal or empty array of capabilities, as it represents the entire entity, and leave enumerating the capabilities to the related metadata endpoints.

3.4. Server Capabilities 🔗

Server capabilities are what specific interoperability or informational capabilities a Server is providing. The following list of strings are an enumerated set of capabilities that are defined by default in this specification.

• coverage - The Server is providing coverage details related to itself as it relates the other listed capabilities. If included, a coverage value is REQUIRED in the metadata object.

Other specifications that define interoperability or informational capabilities for utilities and other entities are encouraged to define an extension that adds their capability as a string to the above list. See the Extensions section for details on how to extend this list of Server capabilities.

4. Coverage Endpoint 🔗

If Servers include the coverage Server capability, a URL to the utility or entity’s coverage MUST be included in the metadata object.

When the URL is requested using the HTTP GET method and the request is valid, the Server MUST respond with a “200 OK” Status Code and body containing JSON object formatted according to the Coverage Listing Format. Cache-optimized responses, such as “304 Not Modified”, to Client requests with cached value headers, such as If-None-Match, is also acceptable.

For invalid requests and error responses, Servers MUST respond with the appropriate Response Code for the type of issue or error (e.g. “503 Service Unavailable” if the Server is temporarily down for maintenance).

4.1. Coverage Listing Format 🔗

Coverage objects are formatted as JSON objects and contain the following named values. This object serves mostly as an object wrapper around a list of coverage entries, so that the list of coverage entries can be paginated by the Server if too long for one response.

• coverage_entries - Array[CoverageEntry] - (REQUIRED) A list of segments this Server covers
• next - null or URL - (REQUIRED) Link to the next page of coverage entries (null if no more subsequent pages)
• previous - null or URL - (REQUIRED) Link to the previous page of coverage entries (null if no more prior pages)

Listings of Coverage Entry objects MUST be ordered in reverse chronological order by updated datetime, where the most recently updated relevant Coverage Entry MUST be first in each listing.

4.2. Coverage Listing Filters 🔗

Servers MUST support Clients adding any of the following URL parameters to the Coverage Endpoint GET request, which will filter the list of Coverage Entry objects to be the intersection of results for each of the URL parameters filters:

• ids - A space-separated list of id values for which the Servers MUST filter the Coverage Entries.

4.3. Coverage Entry Format 🔗

Coverage entries are formatted as JSON objects and contain named values, some of which are required and some of which are optional. The following values are included in the default list available in coverage entries.

• id - string - (REQUIRED) A unique identifier for the coverage entry.
• created - datetime - (REQUIRED) When this coverage entry was first made available.
• updated - datetime - (REQUIRED) When this coverage entry was last modified.
• entity_name - string - (REQUIRED) A human readable name for the entity being covered (e.g. “Demo Gas & Electric”). The entity name in a coverage entry MAY be different than the Server metadata name value in situations where the Server is operated by a parent or other entity.
• entity_abbreviation - null or string - (REQUIRED) The abbreviation for the entity being covered by the coverage entry (e.g. “DG&E”). This field is to assist Clients in parsing coverage for entities that are commonly known by their abbreviated names (e.g. “DG&E” rather than “Demo Gas & Electric”). If no abbreviation is available for the entity being covered, this value is null.
• name - string - (REQUIRED) A human readable name for the coverage area or category (e.g. “Upstate New York electric territory”).
• description - string - (OPTIONAL) A brief description the coverage area or category (e.g. “Example Electric’s coverage for the northern half of New York state”).
• type - CoverageType - (REQUIRED) The type of coverage entry.
• role - CapabilityRole - (REQUIRED) The role of the Server in offering capabilities for the coverage entry.
• infrastructure_types - Array[InfrastructureType] - (REQUIRED) A list of infrastructure categories that are applicable to the coverage entry, which may be an empty array if the entry is not any of the possible infrastructure types.
• commodity_types - Array[CommodityType] - (REQUIRED) A list of services or commodities that are applicable to the coverage entry, which may be an empty array if the entry does not have an applicable commodity.
• capabilities - Array[ServerCapability] - (REQUIRED) What capabilities and functionality this Server supports for the coverage entry, which may be an empty array if the Server is not offering any capabilities for the coverage entry and is just publishing the coverage entry itself for informational purposes. This list MUST NOT include the coverage capability.
• map_resource - URL - (OPTIONAL) Link to a map of the coverage territory.
• map_content_type - MIME type - (OPTIONAL) Content-Type format of the map_resource.
• geojson_resource - URL - (OPTIONAL) Link to a GeoJSON object mapping the coverage territory.

When type is geographic, one or both of map_resource and geojson_resource MUST be provided. If map_resource is provided, then map_content_type MUST also be provided.

4.4. Coverage Entry Types 🔗

Coverage entry types describe what group is being detailed in the coverage entry. Coverage entries MAY represent physical geographic territories (e.g. where a utility provides electric service) or logical groupings (e.g. a certain segment of customers). Many utilities have different geographic or logical coverage for their different territories and capabilities (e.g. their electric service territory is different from their natural gas service territory), so Servers MAY include multiple coverage entries for different types of service without having to publish multiple metadata endpoints.

The following list of strings are an enumerated set of coverage entry types that are valid type values in the coverage entry object.

• geographic - The coverage entry represents a geographic area (e.g. electric service territory).
• logical - The coverage entry represents a non-geographic logical grouping (e.g. a set of commerical customers).

4.5. Capability Roles 🔗

Capability Roles disclose the Server’s role in the provision of capabilities covered by the coverage entry. The following list of strings are an enumerated set of role types that MAY be set as the role value in the coverage entry object.

• authoritative - The Server represents the authoritative entity for the capabilities covered by the coverage entry (e.g. the utility itselfis providing the capability). Only one Server SHOULD mark itself as authoritative for the same coverage and capability, to prevent any abiguity for Clients as to a “source of truth”.
• official - The Server represents a contractually obligated or government mandated entity that has an official connection to the authoritative entity and is offering funcational capabilities for the direct entity in an official capacity (e.g. a state-wide data hub offering customer data access). Multiple Servers could mark themselves as official for the same coverage and capability if the authoritative entity has multiple implementations of capabilities with officially designated service providers or data hubs.
• aggregator - The Server represents an external entity that offers capabilities by proxy and does not have an official agreement with the direct entity (e.g. a service provider that aggregates many utility territory capabilities).

4.6. Infrastructure Types 🔗

Infrastructure types are general categories of utility and energy grid entities that are likely to need to provide access to metadata and capabilities under this specification. The following list of strings are an enumerated set of infrastructure types that MAY be included in the infrastructure_types array of values in the coverage entry object.

• distribution_utility - A electric, natural gas, and/or water utility that distributes the utility service to the end customer (e.g. “who maintains the wires to your house”).
• metering_provider - An entity that maintains the utility metering infrastructure and/or reads end customer meters (e.g. “who reads your meter”).
• supplier - An energy supplier for the end customer (e.g. “who you buy power from”).
• generator - An entity that operates energy generation assets (e.g. a power plant owner and/or operator).
• market_operator - An entity that operates an energy or grid market (e.g. a wholesale energy trading system).
• distribution_service_operator - Commonly called a “DSO”, this is a control entity manages one or more distribution grids.
• transmission_service_operator - Commonly called a “TSO” or Interstate Service Operator (“ISO”), this is a control entity manages one or more distribution grids.
• service_provider - An entity that provides customer, utilty, grid, or other services (e.g. a virtual power plant operator).

4.7. Commodity Types 🔗

Commodity types are what general categories of services a utility or entity provides to end customers. The following list of strings are an enumerated set of commodity types that MAY be included in the commodity_types array of values in the coverage entry object.

• electricity - Has metadata or capabilities around utility electric services
• natural_gas - Has metadata or capabilities around utility natural gas services
• fuel_oil - Has metadata or capabilities around utility fuel oil services
• water - Has metadata or capabilities around utility water services
• wastewater - Has metadata or capabilities around utility wastewater services
• trash - Has metadata or capabilities around utility trash services
• distributed_energy - Has metadata or capabilities around distributed energy resource (DER) services

5. Server Migration 🔗

Utilities and other entities frequently change their organizational structures, such as buying another utility and merging the utility territory together. As utilities and other entities evolve, their Server metadata endpoints, including the domains from which they are served, may no longer be valid.

This section describes how Servers MUST transition their well-known metadata endpoint when migrating or merging into another Server.

For organizational changes where the utility or other entity is not changing their domain, the Server MUST update their existing well-known metadata endpoint in place to match the new organizational structure.

For organizational changes where the new entity structure will not match the existing domain, the Server MUST create a new well-known metadata endpoint on their new domain and update the metadata contents to match the new organization. The Server MUST also respond to requests to the previous well-known metadata endpoint with a “301 Moved Permanently” or “302 Found” Status Code with a Location header containing the URL of the new updated metadata endpoint so that Clients may be automatically redirected to the overriding. The Server MUST maintain the redirect response for as long as the Server retains control over the domain or up to 3 years, whichever is less.

6. Extensions 🔗

Other specifications MAY extend this specification to allow for seamless expansion in Server metadata, capabilities, and coverage.

Metadata Object, Coverage Listing, and Coverage Entry MAY be extended by other specifications to allow for additional values to be possible in their object formats. When extending the object format, other specifications MUST reference the relevant section in this specification and denote that they are extending the object to add a new named value. The additional named value MUST be specified with a general description, the value’s format, and whether the value is REQUIRED or OPTIONAL.

The enumerated lists for valid Server Capabilities, Coverage Listing Filters, Coverage Entry Types, Capability Roles, Infrastructure Types, Commodity Types, and MAY be extended by other specifications to allow for additional strings to be valid. When extending enumerated list, other specifications MUST reference the relevant section in this specification and denote that they are extending the list to add a new string. The additional string MUST be specified with a description of what that string means when it is included in the relevant array.

To facilitate forwards compatibility, Clients MUST ignore unknown or undocumented object values and enumerated strings. If a Client cannot provide adequate functionality based on too many unknown or undocumented object values or enumerated strings, the Client SHOULD refer to the Server’s technical documentation (the documentation value in the metadata object) or contact the Server’s technical support (via the support value in the metadata object).

7. Examples 🔗

The following is a non-normative example of a Client request and Server response to a metadata endpoint.

==Request==
GET /.well-known/carbon-data-spec.json HTTP/1.1
Host: example.com

==Response==
HTTP/1.1 200 OK
Content-Type: application/json;charset=UTF-8

{
    "cds_metadata_version": "v1",
    "cds_metadata_url": "https://example.com/.well-known/carbon-data-spec.json",

    "created": "2022-01-01T00:00:00Z",
    "updated": "2022-06-01T00:00:00Z",

    "name": "Example Data Hub",
    "description": "A fictional regional data hub that offers information about the region's utilities.",
    "website": "https://example.com/data-access",
    "documentation": "https://example.com/data-access/docs",
    "support": "https://example.com/developers/contact",
    "capabilities": [
        "coverage",
    ],

    "coverage": "https://static.example.com/cds-coverage.json"
}

The following is a non-normative example of a Client request and Server response to a coverage endpoint.

==Request==
GET /cds-coverage.json HTTP/1.1
Host: static.example.com

==Response==
HTTP/1.1 200 OK
Content-Type: application/json;charset=UTF-8

{
    "coverage_entries": [
        {
            "id": "dge_elec_west",

            "created": "2022-06-01T00:00:00.000000Z",
            "updated": "2022-06-01T00:00:00.000000Z",

            "entity_name": "Demo Gas & Electric",
            "entity_abbreviation": "DG&E",

            "name": "Western electric service territory",
            "description": "This is the electric service coverage for the western half of the region.",

            "type": "geographic",
            "role": "official",
            "infrastructure_types": [
                "distribution_utility",
            ],
            "commodity_types": [
                "electricity",
            ],
            "capabilities": [],

            "map_resource": "https://static.example.com/cds-coverage-map.png",
            "map_content_type": "image/png",
            "geojson_resource": "https://static.example.com/cds-coverage-map-geo.json"
        }
    ],
    "next": null,
    "previous": null
}

8. Security Considerations 🔗

This specification involves a utility or other entity publishing details about itself. Server operators MUST review all material published in the metadata and coverage endpoints and make sure it is information that is allowed to be made available to Clients. This is especially true for well-known metadata endpoints, as not only are these endpoints public, they are intentionally published with a URL structure that allows them to auto-discovered by Clients.

8.1. Restricted Access 🔗

Servers that wish to restrict access to a metadata or coverage endpoint to certain Clients MUST configure well established authentication processes for Clients to ensure that only the approved Clients may access the restricted endpoint.

This specification does not describe specifically how Servers will authenticate Clients, as these restricted access protocols are context dependent. For example, if a Server is providing the metadata endpoint as part of an existing logged in portal, then they can use that logged in portal’s session cookie to authenticate Client requests to the metadata endpoint.

Well-known metadata endpoints MUST NOT require Client authentication to access, since these metadata endpoints are intended to be published publicly.

Resources provided as URLs in metadata and coverage objects MUST be accessible under at least the same level of authentication requirements as accessing the metadata or coverage objects themselves. For example, if a coverage entry object requires cookie-based authentication to access, then and linked the GeoJSON object must also be accessible using the same cookie-based authentication. Servers MAY also choose to also make those resources available publicly, where any included authentication in the request is ignored. If the metadata or coverage object is made available publicly, then the linked resource must also be accessible publicly.

8.2. Rate Limiting 🔗

For well-known metadata endpoints and other publicly accessible metadata and coverage endpoints, Serves SHOULD configure rate limiting restrictions so that bots and misconfigured scripts will not flood and overwhelm the endpoints with requests, while still allowing legitimate and low-volume requests have access to the endpoints.

9. References 🔗

TODO

10. Acknowledgments 🔗

The authors would like to thank the late Shuli Goodman, who was the Executive Director of LFEnergy, for her incredible leadership in initially organizing the Carbon Data Specification.

11. Authors’ Addresses 🔗

Daniel Roesler
UtilityAPI
Email: daniel@utilityapi.com
URI: https://utilityapi.com/

Other Drafts 🔗

• Maintainer’s draft: [Website] [Code]