# Introduction
Welcome to Open Data Blend Docs. You are at the right place to learn everything you need to know about the Open Data Blend services.
* [Open Data Blend Datasets](/open-data-blend-datasets/datasets): Data lake as a service for data engineers and data scientists.
* [Open Data Blend Analytics](/open-data-blend-analytics/analytics): Business intelligence semantic model as a service for data analysts.
* [Open Data Blend Insights](/open-data-blend-insights/insights): Insights as a service for information workers and decision makers.
# Datasets
An Open Data Blend Dataset is a collection of analytics-ready data files packaged with rich metadata.
You can access Open Data Blend Datasets in two ways:
* **Open Data Blend Dataset UI:**
* Targets the broader community of information workers
* For ad-hoc dataset acquisition and evaluation
* Reading the documentation is optional
* **Open Data Blend Dataset API:**
* Targets technical individuals who are comfortable with code
* For integrating datasets with a broader solution
* Reading the documentation is recommended
## Data Files
To maximise the accessibility and usefulness of the data that we publish, we support the three most popular open data file formats:
* Compressed (Gzip) CSV
* Apache ORC
* Apache Parquet
{% hint style="info" %}
Only previews of the top 100 rows are available in uncompressed CSV data files. The full versions of CSV data files are always Gzip compressed to reduce download times and save disk space.
{% endhint %}
The choice of which data file formats you use will often be driven by the set of tools and platforms that you intend to use them with.
{% hint style="info" %}
A Gzip CSV or ORC data file is typically 40-50% smaller than the corresponding Parquet version. Only the ORC and Parquet data files are optimal for interactive analytical workloads.
{% endhint %}
Below are some examples of the types of file format choices that may be made:
| Platform or Tool | Supported Formats | Chosen Format |
| ---------------- | ----------------- | ------------- |
| Apache Spark | CSV, ORC, Parquet | Parquet |
| Apache Hive | CSV, ORC, Parquet | ORC |
| Presto | CSV, ORC, Parquet | ORC |
| Power BI Desktop | CSV, Parquet | CSV |
| Python | CSV, ORC, Parquet | Parquet |
| R | CSV, Parquet | Parquet |
| Tableau Desktop | CSV | CSV |
{% hint style="info" %}
In the table above 'CSV' refers to both uncompressed and compressed (Gzip) CSV data files.
{% endhint %}
# Dataset Versions
## Dataset Immutability
Changes to Open Data Blend Datasets are always published as new Dataset Versions. Once a dataset version has been created, it remains unchanged. It is immutable.
Every data file change is versioned. This means that all data file updates are reflected through new data file versions, and new data file versions result in new dataset versions.
The URLs for data files always point to a specific data file version. This allows you to share these links knowing that they will always download the same data. The dataset version URLs are currently only surfaced through the [Open Data Blend Dataset API](/open-data-blend-datasets/dataset-api) and are visible in the `snapshot_path` property.
#### Example of the Snapshot Path for a Dataset
```javascript
{
"profile": "data-package",
"name": "open-data-blend-prescribing",
"title": "Prescribing",
"description": "NHS England prescriptions that have been dispensed in the UK",
...
"snapshot_path": "https://packages.opendatablend.io/v1/open-data-blend-prescribing/20210715T154432Z/datapackage.json",
...
}
```
{% hint style="info" %}
The [Open Data Blend Datasets page](https://www.opendatablend.io/datasets) and [Open Data Blend Catalogue](https://packages.opendatablend.io/v1) endpoint always point to the latest versions of datasets.
{% endhint %}
## Reproducible Analysis and Research
A pleasant side-effect of Dataset Immutability is that you can produce analysis or research using an Open Data Blend Dataset knowing that others can reproduce the same result at a later point in time.
Bundling the data with the analysis or research output isn't always practical, especially when the data files are very large. If you are using the Open Data Blend Dataset UI, you can do the following to get the dataset and data file version URLs:
Click the 'Get metadata' button on the Dataset page.
Then save the Open Data Blend Dataset API response (datapackage.json).
Note that the versioned URL value can be seen in the `snapshot_path` property.
{% hint style="info" %}
The`snapshot_path`property value is like a permalink but for Open Data Blend Datasets. Data file URLs always point to a specific version.
{% endhint %}
If you are using the Open Data Blend Dataset API, simply make a dataset request as usual and write-down the response to a long-term data store for future reference.
{% hint style="info" %}
Dataset versions will remain available for at least **24 months** from the date they are superseded. We strongly recommend that you download and keep a local copy of any dataset versions and data file(s) where you need to guarantee that your analysis or research remains reproducible beyond this.
{% endhint %}
# Dataset UI
The [Open Data Blend Dataset UI](https://www.opendatablend.io/datasets) makes it easy for both technical and non-technical users to browse through our catalogue of datasets and download data files. Behind the scenes, the Open Data Blend Dataset UI uses the same metadata that is openly available through the [Open Data Blend Dataset API](/open-data-blend-datasets/dataset-api).
We have designed the Open Data Blend Dataset UI to be simple and intuitive enough for you to use without documentation. That said, there are still some things worth highlighting.
## Data File Download Limit
There is a limit to the number of data files that can be downloaded without an access key. See the [Usage Limit](/open-data-blend-datasets/dataset-api#usage-limits) section for more details.
## Datasets Grid
If you want to see the datasets organised into a grid, you can click the link in the information bar that says, 'To view the datasets by name click here'.
## Relationship Diagrams
If you are viewing a dataset that has at least one direct relationship with one or more other tables in the same dataset, you will find relationship diagram links under the 'Useful Resources' section towards the bottom of the page. This is ideal for getting a high-level view of how the tables within a dataset are related.
## Source Transparency
We are transparent about the data sources that are used to create Open Data Blend Datasets. You will always find links to the data sources that were used under the 'Data Sources' section on the Dataset page. If there were any other significant non-data source inputs that supported the creation of an Open Data Blend Dataset, such as documentation, you will find links to these in the 'Useful Resources' section.
## Blend Classes
We use Blend Classes to provide an at-a-glance indication of how refined an Open Data Blend Dataset is. Each Open Data Blend Dataset has a 'Class I' status at a minimum. Typically, new datasets are published with a Class I badge and refined over time to achieve Class II and Class III.
Below is a table showing the name and description of each Blend Class:
| Blend Class | Blend Class Description |
| ----------- | -------------------------------------------------------- |
| Class I | Carefully structured and optimised for data analytics |
| Class II | Cleansed and enriched using domain knowledge |
| Class III | Supports drilling across with one or more other datasets |
{% hint style="info" %}
'Other datasets' in the above table is referring to Open Data Blend Datasets in particular.
{% endhint %}
## Beta Datasets
From time to time, we will publish new Open Data Blend Datasets as betas to get early feedback and suggestions from our community of users. These datasets may go through any number of structural or data content revisions before coming out of beta. We do not recommend them for production use.
All beta datasets are clearly distinguished by a 'Beta' label.
# Dataset API
## Open Specification
The Open Data Blend Dataset API implements and extends the [Frictionless Data Specifications](https://frictionlessdata.io/specs). You can learn more about this in the [Frictionless Data Compatibility](/open-data-blend-datasets/frictionless-data) section.
Terminology such as 'Packages' and 'Resources' is inherited from the Frictionless Data specifications. You can loosely translate these terms as referring to dataset and data file metadata, respectively.
{% hint style="info" %}
You can translate 'Package' as meaning 'Dataset' and 'Resource' as meaning 'Data File'. An exception to this is that in the Open Data Blend Catalogue metadata, 'Package' refers to the 'Catalogue' and 'Resource' refers to a 'Dataset'.
{% endhint %}
## High Availability
If you have an [Analytics Plan](https://www.opendatablend.io/pricing) subscription, your usage of the Open Data Blend Dataset API is covered by a [service-level agreement (SLA)](https://www.opendatablend.io/service-level-agreement) that includes a 99.5% up-time guarantee. You can check the [Status Page](https://status.opendatablend.io) for current and historic up-time statuses at any time.
## Fair Usage
We regularly monitor usage and take reasonable actions to ensure that the Open Data Blend Dataset API service is used fairly and sustainably. We only impose hard limits on data file endpoints (see the Usage Limits section below).
## Usage Limits
A data file can be downloaded without providing an access key. All requests without an access key are limited to 15 requests per month. Requests that exceed this limit will receive a HTTP 401 ('Unauthorized') response.
{% hint style="info" %}
We always aim to strike a careful balance between openness, in terms of accessibility, and the on-going sustainability of the service. Many of our data files can be quite large. Imposing the 15 request per month limit on free downloads helps to keep the associated bandwidth and compute costs at a sustainable level.
{% endhint %}
There is no limit to the number of requests that can be made with a valid access key. All requests above the number included in your subscription will incur an additional cost. The cost per additional data file request can be found on the [pricing page](https://www.opendatablend.io/pricing).
{% hint style="info" %}
Preview CSV data files do not count towards the limit.
{% endhint %}
## Rate Limits
All data file requests are limited to 30 requests per minute. Exceeding this limit will result in a HTTP 429 ('Too Many Requests') response being returned for up to an hour.
## API Reference
The Open Data Blend Dataset API is simple and has three types of requests:
* Catalogue
* Dataset
* Data File
All responses are in JSON with the following exceptions:
* **Requests for data files:** the backend response is a data file.
* **Requests that result in a pass-through error:** the backend response is presented verbatim which could be in XML, for example.
Each request type is described in this section with examples, where applicable.
\
Request Methods
---------------
Unless specified otherwise, requests should be made using the HTTP GET method.
### Catalogue
The Catalogue endpoint returns a set of metadata for the published Open Data Blend Datasets. The request can be made using any of three endpoints.
#### Non-versioned Endpoint
```http
https://packages.opendatablend.io
```
{% hint style="warning" %}
The non-versioned endpoint is provided for convenience and will always point to the latest versioned endpoint of the Open Data Blend Dataset API. We recommend that you use the standard or Frictionless Data Specification endpoint for any production use. This will ensure that your code is not repointed to a new versioned endpoint if one is introduced in the future.
{% endhint %}
#### Standard Endpoint (Recommended)
```http
https://packages.opendatablend.io/v1
```
#### Frictionless Data Specification Endpoint (Recommended)
```http
https://packages.opendatablend.io/v1/open-data-blend-catalogue/datapackage.json
```
#### Response Example
All variants of the Open Data Blend Catalogue endpoint will return the same JSON response.
```javascript
{
"profile": "data-package-catalog",
"name": "open-data-blend-catalogue",
"title": "Open Data Blend Catalogue",
"description": "A catalogue of the released Open Data Blend Packages",
"terms_of_service": "https://www.opendatablend.io/terms",
"updated": "2020-11-18T15:45:31Z",
"contributors": [
{
"title": "Open Data Blend Team (Nimble Learn Ltd)",
"path": "https://www.opendatablend.io",
"role": "author"
}
],
"resources": [
{
"name": "open-data-blend-anonymised-mot",
"path": "https://packages.opendatablend.io/v1/open-data-blend-anonymised-mot/datapackage.json",
"title": "Anonymised MOT",
"description": "MOT tests and results since the MOT system was computerised in 2005",
"updated": "2020-10-19T08:17:52Z",
"is_beta": true,
"format": "json",
"image": "https://packages.opendatablend.io/image/automotive.svg",
"theme": {
"name": "automotive",
"title": "Automotive",
"image": "https://packages.opendatablend.io/image/automotive.svg"
},
"blend_classes": [
{
"name": "class-1",
"title": "Class I",
"image": "https://packages.opendatablend.io/image/class-1.svg",
"description": "Carefully structured and optimised for data analytics"
},
{
"name": "class-2",
"title": "Class II",
"image": "https://packages.opendatablend.io/image/class-2.svg",
"description": "Cleansed and enriched using domain knowledge"
},
{
"name": "class-3",
"title": "Class III",
"image": "https://packages.opendatablend.io/image/class-3.svg",
"description": "Supports drilling across with one or more other blends"
}
],
"resource_group_count": 37,
"resource_count": 148
},
{
"name": "open-data-blend-date",
"path": "https://packages.opendatablend.io/v1/open-data-blend-date/datapackage.json",
"title": "Date",
"description": "A collection of sequential dates with several levels of roll-up",
"updated": "2020-10-19T08:17:52Z",
"is_beta": false,
"format": "json",
"image": "https://packages.opendatablend.io/image/temporal.svg",
"theme": {
"name": "temporal",
"title": "Temporal",
"image": "https://packages.opendatablend.io/image/temporal.svg"
},
"blend_classes": [
{
"name": "class-1",
"title": "Class I",
"image": "https://packages.opendatablend.io/image/class-1.svg",
"description": "Carefully structured and optimised for data analytics"
},
{
"name": "class-3",
"title": "Class III",
"image": "https://packages.opendatablend.io/image/class-3.svg",
"description": "Supports drilling across with one or more other blends"
}
],
"resource_group_count": 1,
"resource_count": 4
}
]
}
```
#### Metadata Definitions
Open Data Blend Catalogue metadata is split into two main parts:
* **Package Properties:** The collection of properties at the root level that describe the catalogue.
* **Resource Properties:** The collection of properties within the 'resources' property that describe the published datasets.
*Package Properties*
| Name | Description |
| ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| profile | Name of the [Data Package profile](https://specs.frictionlessdata.io/profiles/). The value will always be 'data-package-catalog' which indicates that it is compatible with the Frictionless Data pattern for [Data Package Catalogues](https://specs.frictionlessdata.io/patterns/#describing-data-package-catalogs-using-the-data-package-format). |
| name | Name of the catalogue in slug form |
| title | Name of the catalogue in title case |
| description | Brief description of the catalogue |
| terms\_of\_service | URL of the Open Data Blend terms of service |
| updated | Last date that the catalogue was updated |
| contributors | Details of the catalogue contributors |
| resources | List of published datasets and their properties. See the ‘Resource Properties’ section below for resource object definitions. |
*Resource Properties*
| Name | Description |
| ---------------------- | ----------------------------------------------------------------------------------------- |
| name | Name of the dataset in slug form |
| path | URL of the dataset metadata |
| title | Name of the Dataset in title case |
| description | Brief description of the dataset |
| updated | Last date that the dataset was updated |
| is\_beta | Beta status of the dataset. `true` means in beta and `false` means stable |
| format | File format of dataset metadata. This is always `.json`. |
| image | URL of the dataset icon |
| theme |
Theme of the dataset which is used to group like themed datasets together
|
| blend\_classes | Blend Class of the dataset. Indicates the maturity of the dataset in terms of refinement. |
| resource\_group\_count | Number of data file groups. Indicates the number of logical tables. |
| resource\_count | Number of data files. |
### Dataset
The Dataset endpoint returns the complete set of metadata for an Open Data Blend Dataset. The request can be made using any of two endpoints. In both endpoint variants, `name` is the name of the dataset in slug form e.g. 'open-data-blend-date'.
#### Standard Endpoint
{% tabs %}
{% tab title="Request" %}
```http
https://packages.opendatablend.io/v1/:name
```
{% endtab %}
{% tab title="Example" %}
```http
https://packages.opendatablend.io/v1/open-data-blend-date
```
{% endtab %}
{% endtabs %}
#### Frictionless Data Specification Endpoint
{% tabs %}
{% tab title="Request" %}
```http
https://packages.opendatablend.io/v1/:name/datapackage.json
```
{% endtab %}
{% tab title="Example" %}
```http
https://packages.opendatablend.io/v1/open-data-blend-date/datapackage.json
```
{% endtab %}
{% endtabs %}
#### Response Example
{% hint style="info" %}
The below response example has been abbreviated for clarity.
{% endhint %}
```javascript
{
"profile": "data-package",
"name": "open-data-blend-date",
"title": "Date",
"description": "A collection of sequential dates with several levels of roll-up",
"is_beta": false,
"terms_of_service": "https://www.opendatablend.io/terms",
"contributors": [
{
"title": "Open Data Blend Team (Nimble Learn Ltd)",
"path": "https://www.opendatablend.io",
"role": "author"
}
],
"theme": {
"name": "temporal",
"title": "Temporal",
"image": "https://packages.opendatablend.io/image/temporal.svg"
},
"homepage": "https://www.opendatablend.io/package/?name=open-data-blend-date",
"image": "https://packages.opendatablend.io/image/temporal.svg",
"updated": "2021-02-12T23:56:39Z",
"path": "https://packages.opendatablend.io/v1/open-data-blend-date/datapackage.json",
"snapshot_path": "https://packages.opendatablend.io/v1/open-data-blend-date/20210212T235639Z/datapackage.json",
"keywords": [
"Time Intelligence",
"Trend Analysis"
],
"blend_classes": [
{
"name": "class-1",
"title": "Class I",
"image": "https://packages.opendatablend.io/image/class-1.svg",
"description": "Carefully structured and optimised for data analytics"
},
{
"name": "class-3",
"title": "Class III",
"image": "https://packages.opendatablend.io/image/class-3.svg",
"description": "Supports drilling across with one or more other blends"
}
],
"resources": [
{
"group_name": "date",
"group_title": "Date",
"group_description": "Collection of sequential dates starting from the 1800s and running well into the future, with several levels of roll-up",
"group_row_count": 80720,
"is_preview": true,
"name": "date-csv",
"path": "https://packages.opendatablend.io/data/dimension/date/20201018T195237Z/date_preview.csv",
"title": "Date (.csv)",
"description": "Collection of sequential dates starting from the 1800s and running well into the future, with several levels of roll-up",
"format": "csv",
"licenses": [
{
"name": "ODbL-1.0",
"title": "Open Data Commons Open Database License 1.0",
"path": "https://www.opendatablend.io/open-database-licence"
}
],
"profile": "data-resource",
"row_count": 100,
"updated": "2020-10-18T19:52:37Z",
"schema": {
"fields": [
{
"name": "nll_licence_code",
"type": "string",
"format": "default",
"title": "Licence Code",
"description": "Open Definition licence ID (http://licenses.opendefinition.org/)."
},
{
"name": "nll_licence_name",
"type": "string",
"format": "default",
"title": "Licence Name",
"description": "Open Definition Licence name."
},
{
"name": "nll_licence_url",
"type": "string",
"format": "default",
"title": "Licence URL",
"description": "URL where the licence details can be found."
},
{
"name": "drv_date_key",
"type": "integer",
"format": "default",
"title": "Date Key",
"description": "Primary key."
},
{
"name": "drv_date",
"type": "string",
"format": "default",
"title": "Date",
"description": "Date in ISO format yyyy-mm-dd. The value is always a valid date."
},
...
]
}
},
{
"group_name": "date",
"group_title": "Date",
"group_description": "Collection of sequential dates starting from the 1800s and running well into the future, with several levels of roll-up",
"group_row_count": 80720,
"is_preview": false,
"name": "date-csv-gz",
"path": "https://packages.opendatablend.io/data/dimension/date/20201018T195237Z/date.csv.gz",
"title": "Date (.csv.gz)",
"description": "Collection of sequential dates starting from the 1800s and running well into the future, with several levels of roll-up",
"format": "csv",
"compression": "gz",
"licenses": [
{
"name": "ODbL-1.0",
"title": "Open Data Commons Open Database License 1.0",
"path": "https://www.opendatablend.io/open-database-licence"
}
],
"profile": "data-resource",
"row_count": 80720,
"updated": "2020-10-18T19:52:37Z",
"schema": {
"fields": [
{
"name": "nll_licence_code",
"type": "string",
"format": "default",
"title": "Licence Code",
"description": "Open Definition licence ID (http://licenses.opendefinition.org/)."
},
{
"name": "nll_licence_name",
"type": "string",
"format": "default",
"title": "Licence Name",
"description": "Open Definition Licence name."
},
{
"name": "nll_licence_url",
"type": "string",
"format": "default",
"title": "Licence URL",
"description": "URL where the licence details can be found."
},
{
"name": "drv_date_key",
"type": "integer",
"format": "default",
"title": "Date Key",
"description": "Primary key."
},
{
"name": "drv_date",
"type": "string",
"format": "default",
"title": "Date",
"description": "Date in ISO format yyyy-mm-dd. The value is always a valid date."
},
...
]
}
},
{
"group_name": "date",
"group_title": "Date",
"group_description": "Collection of sequential dates starting from the 1800s and running well into the future, with several levels of roll-up",
"group_row_count": 80720,
"is_preview": false,
"name": "date-orc",
"path": "https://packages.opendatablend.io/data/dimension/date/20201018T195237Z/date.orc",
"title": "Date (.orc)",
"description": "Collection of sequential dates starting from the 1800s and running well into the future, with several levels of roll-up",
"format": "orc",
"licenses": [
{
"name": "ODbL-1.0",
"title": "Open Data Commons Open Database License 1.0",
"path": "https://www.opendatablend.io/open-database-licence"
}
],
"profile": "data-resource",
"row_count": 80720,
"updated": "2020-10-18T19:52:37Z",
"schema": {
"fields": [
{
"name": "nll_licence_code",
"type": "string",
"format": "default",
"title": "Licence Code",
"description": "Open Definition licence ID (http://licenses.opendefinition.org/)."
},
{
"name": "nll_licence_name",
"type": "string",
"format": "default",
"title": "Licence Name",
"description": "Open Definition Licence name."
},
{
"name": "nll_licence_url",
"type": "string",
"format": "default",
"title": "Licence URL",
"description": "URL where the licence details can be found."
},
{
"name": "drv_date_key",
"type": "integer",
"format": "default",
"title": "Date Key",
"description": "Primary key."
},
{
"name": "drv_date",
"type": "string",
"format": "default",
"title": "Date",
"description": "Date in ISO format yyyy-mm-dd. The value is always a valid date."
},
...
]
}
},
{
"group_name": "date",
"group_title": "Date",
"group_description": "Collection of sequential dates starting from the 1800s and running well into the future, with several levels of roll-up",
"group_row_count": 80720,
"is_preview": false,
"name": "date-parquet",
"path": "https://packages.opendatablend.io/data/dimension/date/20201018T195237Z/date.parquet",
"title": "Date (.parquet)",
"description": "Collection of sequential dates starting from the 1800s and running well into the future, with several levels of roll-up",
"format": "parquet",
"licenses": [
{
"name": "ODbL-1.0",
"title": "Open Data Commons Open Database License 1.0",
"path": "https://www.opendatablend.io/open-database-licence"
}
],
"profile": "data-resource",
"row_count": 80720,
"updated": "2020-10-18T19:52:37Z",
"schema": {
"fields": [
{
"name": "nll_licence_code",
"type": "string",
"format": "default",
"title": "Licence Code",
"description": "Open Definition licence ID (http://licenses.opendefinition.org/)."
},
{
"name": "nll_licence_name",
"type": "string",
"format": "default",
"title": "Licence Name",
"description": "Open Definition Licence name."
},
{
"name": "nll_licence_url",
"type": "string",
"format": "default",
"title": "Licence URL",
"description": "URL where the licence details can be found."
},
{
"name": "drv_date_key",
"type": "integer",
"format": "default",
"title": "Date Key",
"description": "Primary key."
},
{
"name": "drv_date",
"type": "string",
"format": "default",
"title": "Date",
"description": "Date in ISO format yyyy-mm-dd. The value is always a valid date."
},
...
]
}
}
],
"sources": [
{
"name": "open-data-blend",
"title": "Open Data Blend",
"path": "https://www.opendatablend.io/about-us"
}
],
"reuse_ideas": [
"Time Intelligence",
"Trend Analysis"
],
"showcases": [
{
"name": "placeholder-showcase-1",
"title": "Placeholder Showcase 1",
"image": "https://packages.opendatablend.io/image/showcase-placeholder.jpg",
"description": "Placeholder showcase 1 caption."
},
{
"name": "placeholder-showcase-2",
"title": "Placeholder Showcase 2",
"image": "https://packages.opendatablend.io/image/showcase-placeholder.jpg",
"description": "Placeholder showcase 2 caption."
},
{
"name": "placeholder-showcase-3",
"title": "Placeholder Showcase 3",
"image": "https://packages.opendatablend.io/image/showcase-placeholder.jpg",
"description": "Placeholder showcase 3 caption."
}
],
"useful_resources": [
{
"name": "open-data-blend-feedback",
"title": "Open Data Blend Feedback",
"path": "https://github.com/opendatablend/feedback",
"description": "Report data quality issues, provide feedback, share domain knowledge, and suggest new datasets."
},
{
"name": "open-data-blend-help-centre",
"title": "Open Data Blend Help Centre",
"path": "https://www.opendatablend.io/help-centre",
"description": "Covers topics/questions relating to Open Data Blend including etiquette, licensing, and tooling."
}
],
"related_packages": [
{
"name": "open-data-blend-age",
"title": "Open Data Blend Age",
"homepage": "https://www.opendatablend.io/dataset?name=open-data-blend-age",
"path": "https://packages.opendatablend.io/v1/open-data-blend-age/datapackage.json"
},
{
"name": "open-data-blend-mileage",
"title": "Open Data Blend Mileage",
"homepage": "https://www.opendatablend.io/dataset?name=open-data-blend-mileage",
"path": "https://packages.opendatablend.io/v1/open-data-blend-mileage/datapackage.json"
},
{
"name": "open-data-blend-time-of-day",
"title": "Open Data Blend Time of Day",
"homepage": "https://www.opendatablend.io/dataset?name=open-data-blend-time-of-day",
"path": "https://packages.opendatablend.io/v1/open-data-blend-time-of-day/datapackage.json"
}
]
}
```
#### Metadata Definitions
Open Data Blend Dataset metadata is split into two main parts:
* **Package Properties:** The collection of properties at the root level that describe the dataset.
* **Resource Properties:** The collection of properties within the 'resources' property that describe the data files.
*Package Properties*
| Name | Description |
| ------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| profile | Name of the [Data Package profile](https://specs.frictionlessdata.io/profiles/). The value will always be 'data-package' which indicates that it is compatible with the Frictionless Data specification for [Data Packages](https://specs.frictionlessdata.io/data-package/). |
| name | Name of the dataset in slug form |
| title | Name of the dataset in title case |
| description | Brief description of the catalogue |
| is\_beta | Beta status of the dataset. `true` means in beta and `false` means stable |
| terms\_of\_service | URL of the Open Data Blend terms of service |
| contributors | Details of the catalogue contributors |
| theme | Theme of the dataset which is used to group like themed datasets together. |
| homepage | URL for the dataset homepage |
| image | URL for the dataset icon |
| updated | Last date that the dataset was updated |
| path | Location of the dataset metadata |
| snapshot\_path | Location of the specific version of the dataset metadata |
| keywords | Keywords that describe the dataset |
| blend\_classes | Blend Class of the dataset. Indicates the maturity of the dataset in terms of refinement. |
| resources | List of data files in the dataset. See the ‘Resource Properties’ section below for resource object definitions. |
| sources | The data sources and information sources that were used to create the dataset |
| reuse\_ideas | Use cases for the dataset |
| showcases | List of showcases that demonstrate how the dataset could be used |
| useful\_resources | List of useful resources such as relationship diagrams and data source documentation |
| related\_packages | List of related or relevant datasets |
*Resource Properties*
| Name | Description |
| ----------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| group\_name | Name of the logical group of data files. Except for preview data files, all data files with the same group name have the same table schema and contain the same data. |
| group\_title | Name of the data file group in title-case |
| group\_description | Description for the data file group |
| group\_row*\_*count | Row count of eachh data file in the data file group. Except for preview data files, all data files in the same group will have matching row counts. |
| is\_preview | Indicates whether the data file has a top 100 row preview or all of the rows. `true` means the data file is a preview and `false` means the data file has all the rows. |
| name | Name of the dataset in slug form |
| path | URL of the dataset metadata |
| title | Name of the Dataset in title case |
| description | Brief description of the dataset |
| format | File format of the data file. This will be either `.csv`, `.csv.gz`, `.orc`, or `.parquet`. |
| licenses | Details of the licences that the data file has been licensed with |
| profile | Name of the [Data Resource profile](https://specs.frictionlessdata.io/profiles/). The value will always be 'data-resource' which indicates that it is compatible with the Frictionless Data specification for [Data Resources](https://specs.frictionlessdata.io/data-resource/). |
| row\_count | Number of rows in the data file excluding headers |
| updated | Last date that the dataset was updated |
| schema | Definition of the data file schema. The definition conforms to the [Table Schema](https://specs.frictionlessdata.io/table-schema/) specification. |
### Data File
The Data File endpoint is the bulk data API and returns the requested data file. Both public (i.e. unauthenticated) and authenticated requests can be made through the same endpoint.
#### Standard Endpoint
{% tabs %}
{% tab title="Request" %}
```http
https://packages.opendatablend.io/v1/:datafile:extension
```
{% endtab %}
{% tab title="Example" %}
```http
https://packages.opendatablend.io/data/dimension/date/20221022T074643Z/date.csv.gz
```
{% endtab %}
{% endtabs %}
`datafile` The path of the data file excluding the file format extension
`extension` The file format extension of the data file which can be one of the following:
* .csv
* .csv.gz
* .orc
* .parquet
{% hint style="info" %}
The '.csv' version of a data file is for preview purposes and only contains the first 100 rows.
{% endhint %}
#### Authenticated Requests
[Access keys](/subscription-management/managing-access-keys) are used to authenticate data file requests. Without an access key, the number of requests you can make will be [limited](/open-data-blend-datasets/dataset-api#usage-limits).
You can provide your access key in one of the following ways:
1. Using the `Open-Data-Blend-Access-Key` header with the POST method
2. Using the `accesskey` query parameter with the GET method
3. Using the `accesskey` body parameter with the POST method
{% hint style="info" %}
The order of precedence for applying your access key matches the listed order above.
{% endhint %}
Below are examples of each authentication method.
Access Key in a Header
```javascript
Open-Data-Blend-Access-Key : YOUR_ACCESS_KEY
```
Access Key in a Query Parameter
```http
https://packages.opendatablend.io/data/dimension/date/20221022T074643Z/date.csv.gz?accesskey=YOUR_ACCESS_KEY
```
Access Key in the Request Body
```javascript
{
"accesskey" : "YOUR_ACCESS_KEY"
}
```
# Frictionless Data Compatibility
## Compatibility
The Open Data Blend Dataset API is compatible with version 1 of the following [Frictionless Data](https://frictionlessdata.io/) specifications:
* [Data Package](https://specs.frictionlessdata.io/data-package/)
* [Data Resource](https://specs.frictionlessdata.io/data-resource/)
* [Table Schema](https://specs.frictionlessdata.io/table-schema/)
It also incorporates the following Frictionless Data patterns:
* [Data Package Catalogues](https://specs.frictionlessdata.io/patterns/#describing-data-package-catalogs-using-the-data-package-format)
* [Compression of Resources](https://specs.frictionlessdata.io/patterns/#compression-of-resources)
This means there is a growing ecosystem of data tools that you can use to work with Open Data Blend Datasets.
The [Frictionless Libraries](https://frictionlessdata.io/universe/) simplify integrations with the Open Data Blend Dataset API from many languages, including Python and R. More specifically, you can use the Data Package libraries to programmatically access our datasets and integrate them into your solutions.
An up-to-date list of the supported libraries can be found [here](https://frictionlessdata.io/universe/), along with documentation on how to use them. Each library varies slightly in its implementation, but the general concepts for working with a Data Package are the same.
## Python Library
The following examples demonstrate how you could use the [Python library](https://pypi.org/project/frictionless/) to interact with the Open Data Blend Dataset API. This is the [most mature](https://framework.frictionlessdata.io/) Frictionless library to date, and one that we recommend. The libraries for the other languages implement a similar method to load data packages and these are explained in each library's [documentation](https://framework.frictionlessdata.io/).
### Installing the Python Library
Install the `frictionless` module.
```python
pip install frictionless
```
{% hint style="warning" %}
Ensure that `frictionless 4.0` or later is installed. Prior versions are not stable releases.
{% endhint %}
Import the `Package` submodule from the `frictionless` module.
```python
from frictionless import Package
```
### Working with Open Data Blend Catalogue Metadata
Loading the Open Data Blend Catalogue metadata.
```python
catalogue = Package('https://packages.opendatablend.io/v1/open-data-blend-catalogue/datapackage.json')
```
Loading the catalogue metadata for an Open Data Blend Dataset.
```python
dataset = catalogue.resources[0]
```
{% hint style="info" %}
The `resources` property is an array of resources. In this context, 'resources' means 'datasets'. In the above example, we are referencing a dataset by its zero-based index position.
Note: The metadata for a dataset in the catalogue is a subset of its full metadata.
{% endhint %}
Getting the name of the dataset.
```python
dataset_name = dataset.name
```
Getting the friendly name of the dataset.
```python
dataset_name = dataset.title
```
Getting the description of the dataset.
```python
dataset_description = dataset.description
```
Getting the endpoint of the dataset.
```python
dataset_path = dataset.path
```
### Working with Open Data Blend Dataset Metadata
Loading Open Data Blend Dataset metadata.
```python
dataset = Package('https://packages.opendatablend.io/v1/open-data-blend-anonymised-mot/datapackage.json')
```
Loading the metadata of a data file.
```python
data_file = dataset.resources[0]
```
{% hint style="info" %}
The `resources` property is an array of resources. In this context, 'resources' means 'data files'. In the above example, we are referencing a data file by its zero-based index position.
{% endhint %}
Getting the name of the data file.
```python
data_file_name = data_file.name
```
Getting the friendly name of the data file.
```python
data_file_title = data_file.title
```
Getting the description of the data file.
```python
data_file_description = data_file.description
```
Getting the table schema of the data file.
```python
data_file_schema = data_file.schema
```
Getting the download location of the data file.
```python
data_file_path = data_file.path
```
# Modelling Conventions
## Star Schemas
All Open Data Blend Datasets are created by using [dimensional modelling techniques](https://www.kimballgroup.com/data-warehouse-business-intelligence-resources/kimball-techniques/dimensional-modeling-techniques/) to optimise the data for data analysis. We carefully transform the source data into facts (i.e. the tables with columns containing measurable values such as the number of prescription items) and dimensions (i.e. tables with the columns containing descriptive values such as the name of a prescribed medicine).
There are a vast number of resources that explain what these modelling practices are and why they exist. Rather than reinventing the wheel, we recommend looking through this comprehensive [glossary](http://www.kimballgroup.com/data-warehouse-business-intelligence-resources/kimball-techniques/dimensional-modeling-techniques) by the Kimball Group.
## Naming Conventions
### Column Prefixes
We use the following column prefixes in our data files:
| Prefix | Description |
| ------ | ------------------------------------------- |
| nlm\_ | Metadata column\* |
| nll\_ | Licence information column\* |
| src\_ | Source column |
| srk\_ | Source column that is also a natural key |
| drv\_ | Derived column\* |
| drk\_ | Derived column that is also a natural key\* |
\*The values in these columns either originate from Open Data Blend or are derived from source columns by Open Data Blend.
### Column Names
All column names are in lower-case, alpha-numeric (no special characters), and spaced using underscores. This is to ensure the broadest level of compatibility across all tools and platforms.
### Acronym Handling
Acronyms are expanded to their corresponding word parts except for when the acronym is very well known and non-ambiguous. Even then, it may still be expanded.
### Foreign Key References
Foreign key references are implied. It's safe to assume that whenever two tables each have a column name that starts with `drv_` and ends with `_key` (e.g. `drv_prescribing_practice_key`), these columns are intended to be joined.
# Loading Data Files in Excel
{% hint style="info" %}
The instructions on this page are based on Excel 2019.
{% endhint %}
## Supported Formats
Excel 2013 (or later) on Windows can load compressed CSV (`csv.gz`) data files through Power Query into a Power Pivot (also known as the Excel 'Data Model').
All versions of Excel can load uncompressed CSV data files without using Power Query, but this is not recommended for large data files.
{% hint style="danger" %}
We only recommend using Excel with the large data files if you intend to use Power Query to load the data directly into [Power Pivot](https://support.microsoft.com/en-us/office/get-started-with-power-pivot-in-microsoft-excel-fdfcf944-7876-424a-8437-1a6c1043a80b), which is designed to handle millions of rows. Once loaded into Power Pivot, the data can be analysed using PivotTables and PivotCharts.
{% endhint %}
## Download the Data Files
Download and save the data files to a suitable location. In the examples that follow, the data has been saved to `C:\data`.
{% hint style="info" %}
Although you could load data files directly from the data file URLs, this is not recommended because you may quickly hit usage limits or incur additional costs. We always recommend saving the files locally or to cloud storage first using the Open Data Blend Dataset UI, Open Data Blend Dataset API, or [Open Data Blend for Python](https://github.com/opendatablend/opendatablend-py).
{% endhint %}
## Loading Compressed (Gzip) CSV Data Files
You can use the below steps as a guide on how you can load compressed (Gzip) data files in Excel.
Navigate to the 'Data' tab in the Excel ribbon and click `Get Data > From Other Sources > Blank Query` to create a new blank query.
Use the Advanced Editor to add a Power Query M expression like the following to the blank query. This will allow the compressed (Gzip) data file to be loaded directly.
```bash
let
Source = Binary.Decompress(File.Contents("C:\data\date\date.csv.gz"), Compression.GZip),
#"Imported CSV" = Csv.Document(Source,[Delimiter=",", Encoding=1252, QuoteStyle=QuoteStyle.None])
in
#"Imported CSV"
```
{% hint style="info" %}
Note that the Power Query M expression explicitly applies Gzip decompression in the 'Source' step. Without this, Power Query is not able to read the compressed (Gzip) data file directly.
{% endhint %}
Click 'Done' on the Advanced Editor window and a preview of the data should appear.
Apply the 'Use First Row as Header' transformation and rename the query to something appropriate.
Click `Close & Load > Close & Load to...` and ensure that you have 'Only Create Connection' selected and the 'Add this data to the Data Model' checkbox ticked.
## Using Excel for Data Analysis
Guidance on how to analyse data in Excel is beyond the scope of this documentation. You may find the following external links helpful:
* [What is Power Query?](https://docs.microsoft.com/en-us/power-query/power-query-what-is-power-query)
* [Get started with Power Pivot in Microsoft Excel](https://support.microsoft.com/en-us/office/get-started-with-power-pivot-in-microsoft-excel-fdfcf944-7876-424a-8437-1a6c1043a80b)
# Loading Data Files in Power BI Desktop
{% hint style="info" %}
The instructions on this page are based on the Power BI Desktop February 2021 release.
{% endhint %}
## Supported Formats
Power BI Desktop can load the following data file formats:
* Compressed CSV (`csv.gz`)
* Parquet (`.parquet`)
## Download the Data Files
Download and save the data files to a suitable location. In the examples that follow, the data has been saved to `C:\data`.
{% hint style="info" %}
Although you could load data files directly from the data file URLs, this is not recommended because you may quickly hit usage limits or incur additional costs. We always recommend saving the files locally or to cloud storage first using the Open Data Blend Dataset UI, Open Data Blend Dataset API, or [Open Data Blend for Python](https://github.com/opendatablend/opendatablend-py).
{% endhint %}
## Loading Compressed (Gzip) CSV Data Files
You can use the below steps as a guide on how you can load compressed (Gzip) CSV data files in Power BI Desktop.
On the Power BI ribbon, click `Get Data > Blank Query`.
Use the Advanced Editor to add a Power Query M expression like the following to the blank query. This will allow the compressed (Gzip) data file to be loaded directly.
```bash
let
Source = Binary.Decompress(File.Contents("C:\data\date\date.csv.gz"), Compression.GZip),
#"Imported CSV" = Csv.Document(Source,[Delimiter=",", Encoding=1252, QuoteStyle=QuoteStyle.None])
in
#"Imported CSV"
```
{% hint style="info" %}
Note that the Power Query M expression explicitly applies Gzip decompression in the 'Source' step. Without this, Power Query is not able to read the compressed (Gzip) data file directly.
{% endhint %}
Click 'Done' on the Advanced Editor window and a preview of the data should appear.
Apply the 'Use First Row as Header' transformation and rename the query to something appropriate.
Click 'Close & Apply' to load the data into the data model.
## Loading Parquet Data Files
You can use the below steps as a guide on how you can load Parquet data files in Power BI Desktop.
Click 'Get Data' from the Power BI ribbon and then 'More...'.
Click 'Parquet' in the list of data sources and then click 'Connect'.
Enter the path of the data file.
{% hint style="info" %}
You may have noticed that the path field is labelled 'URL'. This field accepts both local and remote paths.
{% endhint %}
When presented with a preview of the data, click 'Load'.
The data will be loaded into the data model.
## Using Power BI Desktop for Data Analysis
Guidance on how to analyse data in Power BI Desktop is beyond the scope of this documentation.
You may find the following helpful:
* [Power BI Documentation](https://docs.microsoft.com/en-us/power-bi/)
* [Microsoft Lean for Power BI](https://docs.microsoft.com/en-gb/learn/powerplatform/power-bi?WT.mc_id=powerbi_landingpage-marketing-page)
# Loading Data Files in Tableau Desktop
{% hint style="info" %}
The instructions on this page are based on Tableau Desktop Professional Edition 2020.4 and Tableau Desktop Public Edition 2020.4.
{% endhint %}
## Supported Formats
Tableau Desktop can't load the compressed CSV data files (`csv.gz`) directly but it can load the CSVs once they’ve been extracted.
## Download the Data Files
Download and save the data files to a suitable location. In the examples that follow, the data has been saved to `C:\data`.
{% hint style="info" %}
Although you could load data files directly from the data file URLs, this is not recommended because you may quickly hit usage limits or incur additional costs. We always recommend saving the files locally or to cloud storage first using the Open Data Blend Dataset UI, Open Data Blend Dataset API, or [Open Data Blend for Python](https://github.com/opendatablend/opendatablend-py).
{% endhint %}
## Extracting the CSV Data Files
Use a decompression tool such as [7-Zip](https://www.7-zip.org/) to extract compressed CSV data file to a suitable location.
## Loading CSV Data Files
You can use the below steps as a guide on how you can load extracted CSV data files in Tableau Desktop.
In the Connect pane under the 'To a File' section, click 'Text File'
In the Open File dialog, navigate to the location where you extracted the CSV data file to then select it and click 'Open'.
Click 'Update Now' to see a preview of the data.
## Using Tableau Desktop for Data Analysis
Guidance on how to analyse data in Tableau Desktop is beyond the scope of this documentation.
You may find the following helpful:
* [Tableau Desktop and Web Authoring Help](https://help.tableau.com/current/pro/desktop/en-us/gettingstarted_overview.htm)
* [Tableau Public How-to Videos](https://public.tableau.com/en-us/s/resources)
# Loading Data Files in Python
## Supported Formats
Python can load the following data file formats:
* Compressed CSV (`.csv.gz`)
* ORC (`.orc`)
* Parquet (`.parquet`)
## Download the Data Files
Download and save the data files to a suitable location. In the examples that follow, the data has been saved to `C:\data`.
{% hint style="info" %}
Although you could load data files directly from the data file URLs, this is not recommended because you may quickly hit usage limits or incur additional costs. We always recommend saving the files locally or to cloud storage first using the Open Data Blend Dataset UI, Open Data Blend Dataset API, or [Open Data Blend for Python](#open-data-blend-for-python).
{% endhint %}
## Loading Compressed (Gzip) CSV Data Files
You can use the below steps as a guide on how you can load compressed (Gzip) data files into Python.
Install the `pandas` module from the Anaconda prompt.
```python
pip install pandas
```
Import the `pandas` module.
```python
import pandas as pd
```
Read the compressed CSV data file into a data frame.
```python
pd.read_csv(r"C:\data\date.csv.gz")
```
## Loading ORC Data Files
You can use the below steps as a guide on how you can load ORC data files into Python.
{% hint style="info" %}
There was a [known issue](https://issues.apache.org/jira/browse/ARROW-7811) with the `pyarrow.orc` module, which prevented it from working correctly. If you experience the same issue, try upgrading the module to a newer version that includes the fix.
{% endhint %}
Install `pandas` and the last version of `pyarrow` modules.
```python
pip install pandas
pip install pyarrow
```
Import the `pandas` and `pyarrow` modules.
```python
import pandas as pd
import pyarrow.orc as orc
```
Read the ORC data file into a data frame.
```python
df_date = pd.read_orc(r"C:\data\date\date.orc")
```
Read a subset of the columns from the Parquet data file into a data frame.
```python
df_mot_results_2017 = pd.read_orc(r"C:\data\anonymised_mot_test_result\anonymised_mot_test_result_2017.parquet", columns = ["drv_anonymised_mot_test_date_key", "drv_anonymised_mot_test_result_info_key"])
```
## Loading Parquet Data Files
You can use the below steps as a guide on how you can load Parquet data files in Python.
Install the Pandas and Apache Arrow modules.
```python
pip install pandas
pip install pyarrow
```
Import the Pandas and Apache Arrow modules.
```python
import pandas as pd
import pyarrow.paquet as pq
```
Read the Parquet data file into a data frame.
```python
df_date = pd.read_parquet(r"C:\data\date\date.parquet")
```
Read a subset of the columns from the Parquet data file into a data frame.
```python
df_mot_results_2017 = pd.read_parquet(r"C:\data\anonymised_mot_test_result\anonymised_mot_test_result_2017.parquet", columns = ["drv_anonymised_mot_test_date_key", "drv_anonymised_mot_test_result_info_key"])
```
{% hint style="info" %}
When working with larger data files, it is a good practice to only read the required columns because it will reduce the read times, memory footprint, and processing times.
{% endhint %}
## Open Data Blend for Python
Open Data Blend for Python is the recommended method for ingesting Open Data Blend datasets using Python. You can use the Python package called `opendatablend` to quickly copy data files and the corresponding dataset metadata to your local machine or supported data lake storage.
Install the opendatablend module.
```python
pip install opendatablend
```
Get the data.
```python
import opendatablend as odb
dataset_path = 'https://packages.opendatablend.io/v1/open-data-blend-road-safety/datapackage.json'
# Specify the resource name of the data file. In this example, the 'date' data file will be requested in .parquet format.
resource_name = 'date-parquet'
# Get the data and store the output object
output = odb.get_data(dataset_path, resource_name)
# Print the file locations
print(output.data_file_name)
print(output.metadata_file_name)
```
Use the data.
```python
import pandas as pd
# Read a subset of the columns into a dataframe
df_date = pd.read_parquet(output.data_file_name, columns=['drv_date_key', 'drv_date', 'drv_month_name', 'drv_month_number', 'drv_quarter_name', 'drv_quarter_number', 'drv_year'])
# Check the contents of the dataframe
df_date
```
You can learn more about Open Data Blend for Python including how to use it to ingest data into supported data lakes [here](https://github.com/opendatablend/opendatablend-py).
## Using Python for Data Analysis
Guidance on how to analyse data in Python is beyond the scope of this documentation.
You may find the following helpful:
* [Pandas documentation](https://pandas.pydata.org/docs/) (suitable for small to medium-sized data files)
* [Mondin documentation](https://modin.readthedocs.io/en/stable/) (suitable for data of any size, especially very large data files)
* [Pandas API on Spark documentation](https://spark.apache.org/docs/latest/api/python/user_guide/pandas_on_spark/index.html) (suitable for data of any size, especially very large data files)
# Loading Data Files in R
## Supported Formats
R can load the following data files formats:
* Compressed CSV (`.csv.gz`)
* Parquet (`.parquet`)
## Download the Data Files
Download and save the data files to a suitable location. In the examples that follow, the data has been saved to `C:\data`.
{% hint style="info" %}
Although you could load data files directly from the data file URLs, this is not recommended because you may quickly hit usage limits or incur additional costs. We always recommend saving the files locally or to cloud storage first using the Open Data Blend Dataset UI, Open Data Blend Dataset API, or [Open Data Blend for Python](https://github.com/opendatablend/opendatablend-py).
{% endhint %}
## Loading Compressed (Gzip) CSV Data Files
You can use the below steps as a guide on how you can load compressed (Gzip) data files in R.
Reading the entire compressed (Gzip) CSV data file directly into a data frame.
```r
df_date <- read.csv("C:\\data\\date\\date.csv.gz")
```
###
## Loading Parquet Data Files
You can use the below steps as a guide on how you can load Parquet data files in R.
Install the `arrow` package.
```r
install.packages("arrow")
```
Import the `arrow` library.
```r
library(arrow)
```
Read the Parquet data file into a data frame.
```r
df_date <- read_parquet("C:\\data\\date\\date.parquet")
df_anonymised_mot_test_result_info <- read_parquet("C:\\data\\anonymised_mot_test_result_info\\anonymised_mot_test_result_info.parquet")
```
Read a subset of the columns from the Parquet data file into a data frame.
```r
df_mot_results_2017 <- read_parquet("C:\\data\\anonymised_mot_test_result\\anonymised_mot_test_result_2017.parquet", col_select = c("drv_anonymised_mot_test_date_key", "drv_anonymised_mot_test_result_info_key"))
```
{% hint style="info" %}
When working with larger data files, it is a good practice to only read the required columns because it will reduce the read times, memory footprint, and processing times.
{% endhint %}
## Using R for Data Analysis
Guidance on how to analyse data in R is beyond the scope of this documentation.
You may find the following helpful:
* [The R Manuals](https://cran.r-project.org/manuals.html)
* [Introduction to dplyr](https://dplyr.tidyverse.org/articles/dplyr.html)
# Loading Data Files in Other Tools
There are many other tools, platforms, and services that you can use to load and analyse our data files.
We recommend any of the following tools for working with our ORC and Parquet data files:
* Amazon Athena
* Apache Drill
* Apache Hive
* Apache Impala
* Apache Spark
* Azure Databricks
* Azure Synapse Analytics
* Dremio
* Google BigQuery
* Julia
* Microsoft Fabric
* Presto
# Analytics
Open Data Blend Analytics is a hosted business intelligence service that you connect to from analytical tools such as Excel, Power BI, and Tableau. The catalogue of available data is the same as the [Open Data Blend Datasets](/open-data-blend-datasets/datasets) but limited at most to [four years of history](/open-data-blend-analytics/analytics#tables) plus the latest available year.
Once connected to Open Data Blend Analytics from supported analytical tools, you can instantly analyse the data and build insightful dashboards and reports using convenient drag-and-drop interfaces. There is no need to download data files or write any code.
## In-memory Analytics
Analysis Services is the in-memory analytics engine that powers Open Data Blend Analytics. The highly optimised [Open Data Blend Analytics Model](/open-data-blend-analytics/analytics#semantic-data-model) enables you to flexibly summarise 100s of millions of rows in seconds. This makes it possible to analyse large data volumes interactively.
{% hint style="info" %}
Most tools that can connect to Analysis Services models will be compatible with Open Data Blend Analytics.
{% endhint %}
### Open Data Blend Analytics Experiences
The following short clips demonstrate the Open Data Blend Analytics drag-and-drop experience in three of the most popular BI tools that are supported.
####
#### Excel
{% hint style="info" %}
Supported by using Get Data in Excel 2016 and later.
{% endhint %}
####
#### Power BI Desktop
{% hint style="info" %}
Supported by all recent versions of Power BI Desktop but we recommend using the latest version whenever possible. Power BI Desktop is free.
{% endhint %}
####
#### Tableau Desktop
{% hint style="info" %}
Supported by Tableau Desktop Professional Edition 10 or later.
{% endhint %}
## Semantic Data Model
The Open Data Blend Analytics Model is a business intelligence semantic data model. It already 'knows' how the tables are related and what meaningful aggregations can be applied. The model hides the complexity of the underlying relationships and calculations, enabling data analysts to ask complex analytical questions through simple client tool interfaces.
There are five significant types of object in the Open Data Blend Analytics Model:
* Tables
* Columns
* Relationships
* Hierarchies
* Measures
### Tables
All table names are aligned with the friendly names that can be seen in the corresponding [Open Data Blend Datasets](/open-data-blend-datasets/datasets). Look-up tables (i.e. dimension tables) always contain the same data as the data files in the corresponding Open Data Blend Dataset.
For tables that contain a set of measurements or observations over time (i.e. fact tables), the amount of history available in the model depends on the total number of rows in the table.
| Number of Rows | History |
| ----------------- | --------------------------------------------- |
| Up to 100 million | All years |
| Over 100 million | At least four full years plus the latest year |
### Columns
All column names are aligned with the friendly names that can be seen in the corresponding [Open Data Blend Datasets](/open-data-blend-datasets/datasets). Columns, such as keys, are hidden in the model because they are not useful for data analysis. Some client tools like [Power BI Desktop](/open-data-blend-analytics/connecting-from-power-bi) can show the hidden columns.
The columns of each table are grouped into two top-level folders:
* **Source:** Columns in this folder contain the same values as in the original source data.
* **Derived:** Columns in this folder contain derived values. These are cleansed versions of the source values, values that have been inferred, or other enrichments that have been made by the Open Data Blend Team.
This has been done to make it clear where additional value has been added and to make it possible to tie back derived column values to their source values.
{% hint style="info" %}
Being able to clearly distinguish between columns that originated from the source and derived columns helps to keep the data transparent and trustworthy. You can always reason about the derived values and provide us with [feedback](https://www.opendatablend.io/get-involved) if you spot any issues.
{% endhint %}
###
### Hierarchies
Hierarchies have been included in the model to save you time and effort when building your dashboards and reports. Imagine a scenario where you need to break down a set of numbers (i.e. measures) by years, months, and dates.
Without a hierarchy, you would need to find and add the columns to your report one at a time and ensure that the levels are arranged in the right order:
* Year
* Month Name
* Date
The convenience of a hierarchy allows you to use a single object named '**Year > Month Name > Date**' that will break down the numbers in the same way with much less effort. For less obvious hierarchies, such as those for prescription medicines, it takes the guesswork out of what the hierarchies should be and the ordering of the levels.
You are free to use the hierarchies that have been pre-defined, or create your own using the manual method described above. The choice is yours.
### Relationships
The Open Data Blend Analytics Model includes pre-defined relationships for tables where one or more known relationships exists. You can get a high-level view of the relationships between tables by viewing the [relationship diagrams](/open-data-blend-datasets/dataset-ui#relationship-diagrams) in the relevant Open Data Blend Datasets or by using the [Model View](https://docs.microsoft.com/en-us/power-bi/transform-model/desktop-relationship-view) Power BI Desktop. The power of these relationships comes from the ability to break down the numbers (i.e. measures) from one table by the descriptive columns in one or more related tables.
### Measures
You'll find predefined calculations in the Open Data Blend Analytics Model that can be used to dynamically aggregate the data. These calculations are called 'measures' and they can do things as simple as summing a single column or more complicated calculations like year-on-year percentage change.
#### Calculation Table
The Calculation table is a special purpose table that only contains measures. The measures are organised into the following folder structure:
\|---- {Subject Area Name}
\|---- {Logical Table Name}
\|---- General
\|---- Time Intelligence
\|---- Month to Date
\|---- Previous Year
\|---- Previous Year Month to Date
....
\
For example, you will find measures belonging to the ‘Anonymised MOT’ Open Data Blend Dataset under the following folder structure:
\|---- Anonymised MOT
\|---- Test Item
\|---- General
\|---- Time Intelligence
\|---- Month to Date
\|---- Previous Year
\|---- Previous Year Month to Date
\|---- ....
\|---- Test Result
\|---- General
\|---- Time Intelligence
\|---- Month to Date
\|---- Previous Year
\|---- Previous Year Month to Date
\|---- ...
Below is a summary of what the General and Time Intelligence folders contain:
* **General:** All measures except for time intelligence measures.
* **Time Intelligence:** Measures that apply some form of time intelligence to a base measure.
{% hint style="info" %}
‘Base measure' refers to the version of a measure without any time intelligence applied.
{% endhint %}
####
#### Time Intelligence Calculations
All the time intelligence calculations are based on calendar dates, months, quarters, and years. In the below table, 'base measure' refers to the version of a measure without any time intelligence applied.
| Calculation Type | Measure Name Suffix | Calculation Description |
| ----------------------------- | ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- |
| Previous Year | PY | The value of the base measure in the same period a year ago. |
| Month to Date | MTD | The value of the base measure after applying a running total starting from the beginning of the month. |
| Quarter to Date | QTD | The value of the base measure after applying a running total starting from the beginning of the quarter. |
| Year to Date | YTD | The value of the base measure after applying a running total starting from the beginning of the year. |
| Year on Year | YOY | The difference between the base measure value in the current year compared to the same period a year ago. |
| Year on Year % | YOY % | The difference between the base measure value in the current year compared to the same period a year ago, expressed as a percentage. |
| Previous Year Month to Date | PYMTD | The value of the base measure after applying a running total starting from the beginning of the month and shifted back in time by a year. |
| Previous Year Quarter to Date | PYQTD | The value of the base measure after applying a running total starting from the beginning of the quarter and shifted back in time by a year. |
| Previous Year Year To Date | PYYTD | The value of the base measure after applying a running total starting from the beginning of the year and shifted back in time by a year. |
{% hint style="info" %}
Over time we may introduce more time intelligence calculation types. As we do, we will update the above table to reflect the new additions.
{% endhint %}
#### Percentage of Visual Total Calculations
You will find versions of base measures (e.g. 'English Prescriptions') that have a ' %' suffix such as 'English Prescriptions %'. This type of measure will return the percentage of the total that is visible for the base measure in your report.
#### Incorrectly Sliced Measures
If you try to slice (i.e. break down) a measure by a column from a table that is not related, you will see a result like the following.
Notice that the measure's value is repeated for every value in the column. This means there is no relationship between the table that the measure is based on and the table that the column belongs to.
To help guide you on which table columns can be used to slice which measures, we recommend checking the [relationship diagram](/open-data-blend-datasets/dataset-ui#relationship-diagrams) of the corresponding Open Data Blend Dataset. If you are connected from [Power BI Desktop](/open-data-blend-analytics/connecting-from-power-bi), you can view the relationships for the entire model in the [Model View](https://docs.microsoft.com/en-us/power-bi/transform-model/desktop-relationship-view).
# Analytics Queries
In Open Data Blend Analytics, any query that is submitted as DAX or MDX is considered an Analytics Query.
## DAX Queries
[Data Analysis Expression (DAX)](https://docs.microsoft.com/en-us/dax/dax-overview) queries look like the following example.
```
DEFINE
VAR __DS0Core =
SUMMARIZECOLUMNS (
'Anonymised MOT Test Result Info'[Test Postcode Area Name],
"Anonymised_MOT_Test_Results", 'Calculation'[Anonymised MOT Test Results]
)
VAR __DS0PrimaryWindowed =
TOPN (
1001,
__DS0Core,
[Anonymised_MOT_Test_Results], 0,
'Anonymised MOT Test Result Info'[Test Postcode Area Name], 1
)
EVALUATE
__DS0PrimaryWindowed
ORDER BY
[Anonymised_MOT_Test_Results] DESC,
'Anonymised MOT Test Result Info'[Test Postcode Area Name]
```
Examples of client tools that can generate these queries are Power BI Desktop and Power BI Report Builder.
## MDX Queries
[Multidimensional Expression (MDX)](https://docs.microsoft.com/en-us/analysis-services/multidimensional-models/mdx/mdx-query-fundamentals-analysis-services?view=asallproducts-allversions) queries look like the following example.
```
SELECT {
[Measures].[English Prescription Actual Cost],
[Measures].[English Prescription Items Total Quantity],
[Measures].[English Prescription Items],
[Measures].[English Prescription Quantity],
[Measures].[English Prescriptions]
},
NON EMPTY [Prescribing Practice].[Prescribing Practice Code (Source)].[Prescribing Practice Code (Source)].AllMembers ON ROWS
FROM [Model]
```
Examples of client tools that can generate these queries are Excel and Tableau Desktop.
## Generated Queries
Client tools such as Excel, Power BI Desktop, and Tableau Desktop will typically generate one query per visualisation (e.g. charts, graphics, tables, etc). Placing too many visuals on a dashboard or report page can cause each refresh of a page display to send several queries to the Open Data Blend Analytics service. This should be considered when making dashboards and report design decisions, as it can reduce their responsiveness.
# Analytics Users
An Analytics User is an email address that is linked to your Open Data Blend Analytics subscription that can be used to connect to and query the Open Data Blend Analytics service.
## Active Analytics Users
An Analytics User that has sent one or more [Analytics Queries](/open-data-blend-analytics/analytics-queries) in a month is considered an active analytics user in that month and takes up an 'Analytics User Seat'. From a billing perspective, the maximum number of analytics users allowed is determined by your [Plan](https://www.opendatablend.io/pricing) and whether you have an add-on for additional users.
# Connecting from Excel
{% hint style="info" %}
The instructions on this page are based on Excel 2019.
{% endhint %}
## Supported Excel Versions
The Windows version of Excel 2019 or later is required to connect to Open Data Blend Analytics.
## Installing the Analysis Services Client Libraries
You may need to install the latest version of the Analysis Services OLE DB Provider (MSOLAP). This is available [here](https://docs.microsoft.com/en-us/analysis-services/client-libraries?view=asallproducts-allversions).
## Connecting to the Open Data Blend Analytics Server
Navigate to the 'Data' tab in the Excel ribbon and click `Get Data > From Database > From Analysis Services`.
You will be prompted for the connection details.
Enter the following details and select 'Use the following User Name and Password' then click 'Next':
**Server**: link://analytics.opendatablend.io
**Username:** Leave blank.
**Password:** Leave blank.
{% hint style="info" %}
Instead of 'link://analytics.opendatablend.io' for the server, you can also use 'powerbi://api.powerbi.com/v1.0/analytics.opendatablend.io/Open Data Blend Analytics'
{% endhint %}
You will be prompted to sign into your account. Use your [Analytics User](/open-data-blend-analytics/analytics-users) details and follow the additional prompts to complete the login.
When prompted to 'Select Database and Table', the 'Open Data Blend Analytics' database, click 'Model' then click 'Next'.
{% hint style="info" %}
If you want to work with only a subset of the model, you can choose one of the perspectives (i.e. a named subset of the model). For example, if you want to focus on analysing the prescribing data you would choose 'Prescribing'.
{% endhint %}
On the 'Save Data Connection File and Finish' step, update the data file name and friendly name if needed, then click 'Finish'.
On the 'Import Data' dialog, click 'OK' to proceed with creating a PivotTable that is connect to the Open Data Blend Analytics Model.
Now you can start analysing the data.
This is what the Open Data Blend Analytics experience looks like in Excel.
## Open Data Blend Starter Files
We have created a set of Open Data Blend Starters which are files with a pre-configured connection to Open Data Blend Analytics. You can find the starter for Excel [here](https://github.com/opendatablend/starters).
# Connecting from Power BI Desktop
{% hint style="info" %}
The instructions on this page are based on the Power BI Desktop February 2021 release.
{% endhint %}
## Supported Power BI Versions
The Open Data Blend Analytics database is hosted in Azure Analysis Services. We recommend that you use the latest version of Power BI Desktop to connect to Open Data Blend Analytics.
## Connecting to the Open Data Blend Analytics Server
On the Power BI ribbon, click `Get Data > Analysis Services`.
You will be prompted for the connection details.
Enter the following details and ensure 'Connect live' is selected then click 'OK'.
**Server**: link://analytics.opendatablend.io
**Database**: Leave blank.
{% hint style="info" %}
Instead of 'link://analytics.opendatablend.io' for the server, you can also use 'powerbi://api.powerbi.com/v1.0/analytics.opendatablend.io/Open Data Blend Analytics'
{% endhint %}
You will be prompted to sign into your account. Use your [Analytics User](/open-data-blend-analytics/analytics-users) details and follow the additional prompts to complete the login.
When presented with the 'Navigator' screen, expand 'Open Data Blend Analytics', click 'Model', and then click 'OK'.
{% hint style="info" %}
If you want to work with only a subset of the model, you can choose one of the perspectives (i.e. a named subset of the model). For example, if you want to focus on analysing the prescribing data you would choose 'Prescribing'.
{% endhint %}
Now you can start analysing the data.
This is what the Open Data Blend Analytics experience looks like in Power BI Desktop.
## Open Data Blend Starter Files
We have created a set of Open Data Blend Starters which are files with a pre-configured connection to Open Data Blend Analytics. You can find the starter for Power BI Desktop [here](https://github.com/opendatablend/starters).
# Connecting from Tableau Desktop
{% hint style="info" %}
The instructions on this page are based on Tableau Desktop Professional Edition 2020.3.6.
{% endhint %}
## Supported Tableau Desktop Versions
The Windows version of Tableau Desktop Professional Edition is required to connect to Open Data Blend Analytics.
{% hint style="info" %}
We recommend using Tableau Desktop Professional Edition 2020.3.6 or later for the best experience.
{% endhint %}
## Installing the Analysis Services Client Libraries
We recommend installing the latest version of the Analysis Services OLE DB Provider (MSOLAP). This is available [here](https://docs.microsoft.com/en-us/analysis-services/client-libraries?view=asallproducts-allversions).
## Connecting to the Open Data Blend Analytics Server
In the Connect pane under the 'To a Server' section, click 'Microsoft Analysis Services'.
You will be prompted for the connection details.
Enter the following details then click 'Sign In':
**Server**: link://analytics.opendatablend.io
**Username:** Enter a blank space.
**Password:** Enter a blank space.
{% hint style="info" %}
Instead of 'link://analytics.opendatablend.io' for the server, you can also use 'powerbi://api.powerbi.com/v1.0/analytics.opendatablend.io/Open Data Blend Analytics'
{% endhint %}
You will be prompted to sign into your account. Use your [Analytics User](/open-data-blend-analytics/analytics-users) details and follow the additional prompts to complete the login.
On the data source page, select 'Open Data Blend Analytics' for the database and 'Model' for the cube. Then click 'Sheet 1'.
{% hint style="info" %}
If you want to work with only a subset of the model, you can choose one of the other cube names. For example, if you want to focus on analysing the prescribing data you would choose 'Prescribing'.
{% endhint %}
Now you can analyse the data.
This is what the Open Data Blend Analytics experience looks like in Tableau Desktop.
## Using Tableau Desktop for Data Analysis
Guidance on how to analyse data in Tableau Desktop is beyond the scope of this documentation.
You may find the following helpful:
* [Tableau Desktop and Web Authoring Help](https://help.tableau.com/current/pro/desktop/en-us/gettingstarted_overview.htm)
* [Tableau Public How-to Videos](https://public.tableau.com/en-us/s/resources)
## Open Data Blend Starter Files
We have created a set of Open Data Blend Starters which are files with a pre-configured connection to Open Data Blend Analytics. You can find the starter for Tableau Desktop [here](https://github.com/opendatablend/starters).
# Connecting from Other Tools
There are several tools that can connect to Open Data Blend Analytics.
In addition to Excel, Power BI, and Tableau, compatibility has been tested with the following tools:
* DAX Studio
* Power BI Report Builder
* SQL Server Management Studio
{% hint style="info" %}
When possible, we recommend using the latest versions of the listed tools.
{% endhint %}
Connect using the following Open Data Blend Analytics server name:
```
link://analytics.opendatablend.io
```
Or
```
powerbi://api.powerbi.com/v1.0/analytics.opendatablend.io/Open Data Blend Analytics
```
When prompted for a login, use your Analytics User details.
# Insights
{% hint style="warning" %}
The Open Data Blend Insights service currently works for Microsoft work or school accounts and Microsoft Outlook accounts. To successfully log in to access the reports, your [Analytics User](/open-data-blend-analytics/analytics-users) account needs to be associated with an email address belonging to one of these account types.
{% endhint %}
Open Data Blend Insights is a collection of business intelligence reports carefully designed to effectively visualise open data. The service is built on top of [Open Data Blend Analytics](/open-data-blend-analytics/analytics) and [Power BI](https://powerbi.microsoft.com). Open Data Blend Insights reports are called 'Insight Reports'.
The Open Data Blend Insights UI includes three types of pages:
* Report Catalogue
* Report Profile
* Report View
{% hint style="info" %}
You only need a [paid plan](https://www.opendatablend.io/pricing) and the Insight Reports add-on to load and interact with Insight Reports on the [Report View](#report-view) pages.
{% endhint %}
## Report Catalogue
The Report Catalogue lists the available Insight Reports and those to be added soon. You will see a thumbnail of the report along with a summary of its contents.
Except for reports that are coming soon, clicking on a report tile will navigate to the report's profile page.
## Report Profile
The Report Profile page serves three primary purposes:
* Help you decide whether a report is likely to meet your needs
* Provide report data and content licensing information
* Act as an entry point to view and interact with the report
The page has three sections:
* Report Summary
* Report Gallery
* Report Launcher
### Report Summary
The Report Summary provides information on the data sources and target audience for which the report has been optimised.
### Report Gallery
The Report Gallery allows you to see what the report looks like. The screenshots contain actual data, so they are an excellent way to determine whether a report can support your decision-making process ahead of signing up for a [paid plan](https://www.opendatablend.io/pricing).
### Report Usage
The Report Usage section provides examples of analytical questions that the report can answer. This can help you to think about the questions you want to answer using the report. You will also see clear licensing information so there is no confusion about how the insights can be used.
## Report View
The Report View is where you can view and interact with the live report.
Before you can view a report, you will need to sign in using your [Analytics User](/open-data-blend-analytics/analytics-users) account. Once you are signed in, you can navigate to and view other reports without being prompted to sign in again for at least an hour within the same browsing session.
{% hint style="warning" %}
If you are using Safari on macOS or any web browser on iOS to access an Insight Report, it may not load in the normal view. This is because the normal view requires cross-site tracking functionality which is disabled by default in these operating systems.
To view the report without having to change any privacy settings, you can launch the report in the fullscreen view.
{% endhint %}
After successfully signing in, the report will load and you can interact with it in the normal view or click 'Fullscreen' to open it in the fullscreen view.
# Report Drill Throughs
When exploring an Insight Report, drill throughs enable you to explore an area of interest in more detail.
For example, in the Pharma Market Insights report, if you were interested in seeing trends for Paracetamol, you could right-click the name in the Prescribing Items table to display the context menu which includes an option to drill through.
If the report has been designed to support drilling through to another detail page, the relevant drill through page options will be listed when you click the 'Drill through' menu item.
{% hint style="info" %}
It is a good idea to right-click report elements to see the available options. What can be done will depend on the type of data visualisation and the design of the report.
{% endhint %}
Upon clicking one of the drill through options, the report will navigate to another page with a focus on the item that you drilled through on. If we use the Pharma Market Insights report as an example, the action would navigate to a page with the trends for the Paracetamol British National Formulary (BNF) chemical substance.
In some reports, a data visualisation may have multiple levels of [drill down](/open-data-blend-insights/report-drill-downs). For these, there can be additional drill through options at each level.
In our Pharma Market Insights report example, the Prescription Items table has two levels: a BNF chemical substance level, and a BNF description level. If you right-click on 'Paracetamol 500mg tablets' after drilling down to the BNF description level and then click the drill through to 'Trends | BNF Description'...
...the report will navigate to a report page with the trend visualisations for 'Paracetamol 500mg tablets'.
# Report Drill Downs
Drill downs enable you to get a different perspective from the same visualisation.
For example, in the Pharma Market Insights report, when looking at the data visualisation of the prescribing items by period, you can use the drill/up controls towards the top of the graph to change the summary level from months...
...to quarters.
The controls for drilling down can differ depending on the visualisation. For example, in the Pharma Market Insights report, you can drill down from BNF chemical substance to BNF description using the +/- buttons.
# Subscription Portal
The Open Data Blend Subscription Portal lets you view and update the information associated with your Analytics Plan. You can access the subscription portal by clicking the 'Login' button on the [Open Data Blend website](https://www.opendatablend.io/).
{% hint style="warning" %}
It is important to note that the login details used for the Open Data Blend Subscription Portal are different from those used for the Open Data Blend Analytics service. This is because the person managing the subscription may not always be an Analytics User.
{% endhint %}
The Open Data Blend Subscription Portal has an intuitive interface and so this documentation will only focus on some key areas.
Once logged in, you will have access the following information:
* Contact details
* Payment details
* Invoices
* Payments made
* Statements of account
* Subscription history
* Access keys
You can perform any of the following actions:
* View subscription details (e.g. renewal date)
* View your access keys
* Cancel your subscription
* Update your contact details
* Update your payment details
* Reset your subscription portal password
* Manage analytics users
* Contact our support team
* Submit a review of our services
# Managing Analytics Users
When you sign up to an [Analytics Plan](https://www.opendatablend.io/pricing) you are given the option to specify the email addresses of the [Analytics Users](/open-data-blend-analytics/analytics-users). You are also given the option to treat the email address of the subscription account holder as an Analytics User (this is ticked by default).
You can update either of these from the subscription portal. From the subscription portal Home page, click 'View More'.
Click the edit icon in the Account Details section.
The 'Email is Analytics User' refers to the email address that is used to log into the subscription portal. You can check or uncheck this as needed.
To update the Analytics Users, simply provide a semi-colon (';') separated list of email addresses e.g. `user1@yourdomain.com;user2@yourdomain.com;user3@yourdomain.com;`
{% hint style="info" %}
The semi-colon separated list of Analytics User email addresses is like the 'To' line of an email. Each user in the list will receive an invite to Open Data Blend Analytics and their activity will be linked to your subscription.
{% endhint %}
{% hint style="danger" %}
If an email address is already linked to another Analytics Plan subscription, only the link to the active subscription that was created first will be honoured for billing purposes.
{% endhint %}
# Managing Access Keys
Access Keys are used to make authenticated requests to the [Open Data Blend Dataset API](/open-data-blend-datasets/dataset-api). Whenever you use one of your access keys to request a data file, the usage will be associated with your Analytics Plan subscription. Without an access key, the number of data file requests that you can make is [limited](/open-data-blend-datasets/dataset-api#usage-limits).
{% hint style="info" %}
When starting a new Analytics Plan subscription, you may have noticed two fields called 'Access Key 1' and 'Access Key 2' on the Analytics Plan checkout page. These are presented but should be left blank. Your access keys will only be generated after you have successfully started your subscription.
In the future, these blank access key fields will no longer be presented during the Analytics Plan sign-up process.
{% endhint %}
{% hint style="danger" %}
Access Keys are a type of secret. Keep them safe and ensure that they are not shared with any unauthorised persons. If you believe that either of your access keys have been compromised, you should raise a request to have them [rotated](/subscription-management/managing-access-keys#rotating-access-keys) immediately.
{% endhint %}
## Viewing Access Keys
You can view your access keys in the subscription portal by clicking the ’View More’ link on the Home page.
Both access keys will be visible in plain text so ensure that your screen is not visible to any unauthorised persons.
Once you have copied and saved the access keys to another secure location (e.g. a password manager or key vault), we **strongly recommend** that you remove them from the portal. This can be done by clicking the edit button in the Account Details section, blanking out the access key values, and then clicking ’Save’.
{% hint style="info" %}
Strong encryption is used to store access keys in the subscription portal database. The actual values are only visible in the portal and are decrypted on-the-fly. We only store access key prefixes and hashes in the access key authentication system.
{% endhint %}
## Using Your Access Keys
For guidance on how to use your access keys, please see the [Data File](/open-data-blend-datasets/dataset-api#data-file) section of the Open Data Blend Dataset API.
## Rotating Access Keys
If you would like to rotate your access keys from time to time to keep in-line with security best practices, please use the 'Send Email' button in the subscription portal to raise a support request. When raising the request, include your subscription number, the name of the access key number that you wish to be regenerated (i.e. 'Access Key 1' or ’Access Key 2'), and ensure that you have updated your applications to use the alternate access key, where applicable.
{% hint style="success" %}
We will always confirm the access key number to be regenerated before completing the request.
{% endhint %}
# Updating Payment Details
It is important to make sure that the payment details associated with your subscription are kept up to date to avoid any disruptions to your subscription.
You can update the payment details at any time. The payment information is sent to and processed by [Stripe](https://stripe.com/docs/security/stripe). We **do not** store your card information on our servers.
From the subscription portal Home page, click 'View More'.
Click the edit icon in the Payment Details section.
Update the payment details and then click 'Save'.